Intended for healthcare professionals

Rapid response to:

Editorials

Data dredging, bias, or confounding

BMJ 2002; 325 doi: https://doi.org/10.1136/bmj.325.7378.1437 (Published 21 December 2002) Cite this as: BMJ 2002;325:1437

Rapid Response:

Epidemiology needs to be taken seriously

Dear Sir,

In their editorial "Data dredging, bias, or confounding" George Davey
Smith and Shah Ebrahim (1) describe a common problem in Epidemiology and
Public Health. Only too often even experienced epidemiologists do not
resist from using existing (large) data sets for further or new analysis’s
(often called “fishing expeditions”) which provide them with significant
associations.

Results of every epidemiological study can either show, what often is
assumed, a causal relation between exposure and outcome or it can only
show the effects of chance, bias or confounding. As Smith and Ebrahim
rightly state, the effect of chance (using p <_0.05 is="is" in="in" this="this" kind="kind" of="of" data="data" dredging="dredging" exercises="exercises" often="often" underestimated.="underestimated." following="following" their="their" suggestions="suggestions" for="for" the="the" use="use" more="more" stringent="stringent" significance="significance" levels="levels" problem="problem" might="might" be="be" reduced="reduced" or="or" controlled.="controlled." but="but" what="what" about="about" bias="bias" and="and" confounding="confounding" p="p"/> Studies used for these re-analysis studies were generally not
designed with the new study question in mind. This can be a problem
especially when controlling for possible confounding in the new analysis.
Experience shows, that in fact it is often very difficult to control for
all known confounders when a study is systematically planned and all
relevant variables known are included in the data collection. Using
existing data sets which were collected for a study with different aims
and objectives than for the new data dredging exercise does not allow to
include any previously not collected variable (e.g. a possible confounder)
in the study. This can open the door for effects of strong forms of
confounding which might even reverse the measured association (e.g.
Simpsons’ paradox) (2).

This can and will ultimately lead to the loss of trust in
epidemiology by the public. Therefore, epidemiological research needs to
be taken seriously by those who use the results for decision making, but
also by those who conduct and analyse the studies.

References:

1) Smith GD, Ebrahim S. Data dredging, bias, or confounding. BMJ 2002;
325(7378):1437-8

2) Reintjes R, de Boer A, van Pelt W, Mintjes-de Groot J. Simpson's
paradox: an example from hospital epidemiology. Epidemiology.
2000;11(1):81-3.

Competing interests:  
None declared

Competing interests: No competing interests

07 January 2003
Ralf Reintjes
professor of epidemiology and public health surveillance
Hamburg University of Applied Sciences, Germany