Are these data real? Statistical methods for the detection of data fabrication in clinical trials

A simple method for assessing raw medical data for fraudulent or
concocted data is by using Benford's Law.
Benford's Law is a phenomenological law stating that in series of figures
or statistical tables the frequency of the number 1 as the first
significant digit occurs more often than the 10% that might be predicted.
This law is also known as the first digit phenomenon and applies
principally though not exclusively to dimensionless or scale invariant
data in which the numerical value of data depends on the units.
The law largely applies to data obtained in a semi-random manner and is
sensitive to deliberate tampering or biasing of data after which the
profile of the frequency of the first digits of numbers obtained will
differ significantly from that predicted. For this reason its value in
financial audit and other areas is being evaluated.
To date its application in assessing numerical data from medical imaging
has not been reported.
Benford's Law (1) applies to numbers drawn from a wide range of sources
and is not restricted to scale invariate data. Demonstrating this requires
sophisticated and thorough investigations of central limit like theorems
and the mantissa of random variables. With an increase in the number of
variables the density function approaches the logarithmic distribution.

If the law applies to data that are scale invariate or that are not
dimensionless then if a probability distribution exists for these data it
must be invariable over any change of scale.
Thus P ( kx ) = f ( k )* P ( x )
If

( P ( x ) dx = 1 , then ( P ( kx )dx = 1/k and normalisation
implies
f (k ) = 1/k. Differentiating with respect to k and setting k=1 gives
x P' (x ) = -P ( x )
Although this is not a proper probability distribution since it diverges,
the laws of physics and convention impose cut-offs.
If many powers of 10 lie between the cut-offs then the probability that
the first significant digit in base 10 is D is given by the logarithmic
distribution;
ln (D+1/D) / ln 10

= ln (D+1)- ln (D)/ ln 10

= Log (1+1/D)

.
The principle of logarithmically distributed significant digits in various
scientific calculations is well known and widely exploited. An extensive
range of diverse data sets does obey this logarithmic distribution for
significant digits and a considerable amount of empirical evidence
supports the use of Benford's law.
While many data sets do not follow the distribution, combination of data
tables tend to conform more closely to the logarithmic distribution. Hill
(2,3) demonstrated in his theorem that random samples taken from random
selection of distributions will conform to the logarithmic law even though
the initial distributions themselves do not.
This theorem also implies that numerous sampling methods from random
distributions will show a trend towards the same logarithmic distribution.

However as the method is also base invariable, the same distribution
of first digits should be obtained following conversion of the data to
bases other than base 10. Similarly, probability distributions can be
calculated for the frequency of the second or other digits. These tend to
occur with more uniform frequency, e.g. approaching 10% for sixth order
significant digits.
The value of this method is that tampering or external interference with
data acquisition is readily detectable and leads to abnormal skewing or
unexpected increased frequency of first digits other than 1. Biased
sampling techniques or concocted data tend to show a predominance of the
first significant digit 6 with far fewer numbers starting with the digit
1. It may provide a convenient means for medical statisticians to assess
raw data.
Discussions on the desirability of publishing raw data used in medical
research studies have largely centred on the difficulties involved.
Publishing data on a journal website is feasible however, as is the
inclusion of data supplements in material sent to reviewers. In systemic
reviews in particular, data from the primary studies should be available
to readers of the review for analyses to be checked and where required,
investigated. Benford's law may have an application in these circumstances
in assessment of numerical data at a basic level. (5,6,7)
Further potential uses of the significant digit law might be in the
testing of computer generated mathematical models. This could have
applications in analysis of predicted data from models describing tracer
kinetics or compartmental distribution of tracers or contrast medium.
"Goodness of Fit" tests to predicted frequency of digits have been devised
and applied in audit of accounting data. (4)
Application of Benford's law therefore, may be helpful to clinical audit
departments and medical statisticians in assessment of bias in data
acquisition in medical audit or research.

References;

1. Benford F.
The law of anomalous numbers
Proceedings of the American Philosophical Society 1938: 78; 551-572.

2. Hill T.
A statisitical derivation of the significant digit law.
Statistical Science 1996: 10; 354-313.

3. Hill T. P.
The first digit phenomenon.
American Scientist 1998: 86; 358-363

4. Nigrini M,
A Taxpayer compliance application of Benford's law.
Journal of the American Taxation Association 1996: 18; 72-91.

5. Hutchon D.J.R.
Infopoints: Publishing raw data and real time statistical analysis on e-
journals.
British Medical Journal 2001; 322:530.

6. Eysenbach G., Sa E-R.,
Code of conduct is needed for publishing raw data.
British Medical Journal 2001; 323:166.

7. Information for authors. Clinical Chemistry; www.aacc.org/ccj/infoauth/stm

Competing interests:
None declared