There are apparent problems with the Al-Marzouki, et al analysis as
well as with the study being criticized.
For one thing, Al-Marzouki et al adopt the criticized-study's term,
"single-blind", in their final-digit analysis.
However, the criticism uses a DOUBLE-BLIND definition to criticize
what was reported as a "single-blind" experiment.
By standard definition, an unblinded study is one in which the
experimenter(s) and the subjects (clinical subjects) are aware of the
conditions as they are administered.
In a single-blind study, the subjects are unaware of the conditions
as they are administered, but the persons administering the trials are
aware of them.
In a double-blind study, both the persons administering the trials
and the subjects are unaware of the conditions during the trials. For
example, see Bowman, et al, "Textbook of Pharmacology" (6th ed.), Chapter
20, "Clinical trial of new drugs".
Thus, apparently, Al-Marzouki et al applied a double-blind criterion
incorrectly. If the criticized study indeed was single-blind, then the
experimenters were aware of the conditions during trials, and some bias of
the data should be expected. Thus, at least some of the statistical
inference drawn by Al-Marzouki et al was not meaningful. A Bayesian
analysis might have thrown more light on the data than the methods
actually reported.
This doesn't mean that knowing falsification did not occur. But, it
is to say that there are serious risks in applying statistics to draw
conclusions about specific occurrences -- it's the same problem as with
circumstantial evidence in court, or with epidemiology in general. One
always should err on the side of assuming improbable statistics rather
than dishonesty. The statistics should raise suspicion, but dishonesty
should be inferred only because of more direct evidence.
Competing interests:
None declared
Competing interests:
No competing interests
05 August 2005
John M. Williams
Business Owner
Markanix Co., P. O. Box 2697, Redwood City, CA 94064
Rapid Response:
Single Blind is Double Blind
There are apparent problems with the Al-Marzouki, et al analysis as
well as with the study being criticized.
For one thing, Al-Marzouki et al adopt the criticized-study's term,
"single-blind", in their final-digit analysis.
However, the criticism uses a DOUBLE-BLIND definition to criticize
what was reported as a "single-blind" experiment.
By standard definition, an unblinded study is one in which the
experimenter(s) and the subjects (clinical subjects) are aware of the
conditions as they are administered.
In a single-blind study, the subjects are unaware of the conditions
as they are administered, but the persons administering the trials are
aware of them.
In a double-blind study, both the persons administering the trials
and the subjects are unaware of the conditions during the trials. For
example, see Bowman, et al, "Textbook of Pharmacology" (6th ed.), Chapter
20, "Clinical trial of new drugs".
Thus, apparently, Al-Marzouki et al applied a double-blind criterion
incorrectly. If the criticized study indeed was single-blind, then the
experimenters were aware of the conditions during trials, and some bias of
the data should be expected. Thus, at least some of the statistical
inference drawn by Al-Marzouki et al was not meaningful. A Bayesian
analysis might have thrown more light on the data than the methods
actually reported.
This doesn't mean that knowing falsification did not occur. But, it
is to say that there are serious risks in applying statistics to draw
conclusions about specific occurrences -- it's the same problem as with
circumstantial evidence in court, or with epidemiology in general. One
always should err on the side of assuming improbable statistics rather
than dishonesty. The statistics should raise suspicion, but dishonesty
should be inferred only because of more direct evidence.
Competing interests:
None declared
Competing interests: No competing interests