Meta-analysis of dietary trials: Potential perils of combining "oranges" and "apples"
Despite its popular use in combining findings from multiple studies,
meta-analysis has many well-known pitfalls (1). These include lack of
homogeneity of the studies, failure to consider important covariates,
inadequate understanding of the scientific subject in question, failure to
consider quality of the studies, and biases in including or excluding
certain studies. Some of these pitfalls are exemplified in the paper by
Hooper et al. (BMJ 2001;322:757-63), who reviewed 27 intervention trials
of fat reduction or modification. There are several problems with this
meta-analysis. First, it mixed conceptually different dietary
intervention approaches, i.e., total fat reduction vs. using unsaturated
fats to replace saturated fat. Neither epidemiology nor clinical trials
support a benefit of low-fat diets on either serum cholesterol or risk of
coronary heart disease (CHD). Substitution of vegetable oils rich in
unsaturated fats for saturated or trans fats, on the other hands, lowers
LDL and reduces cardiovascular endpoints, as indicated by several previous
systematic reviews of this topic. A second problem is that the meta-
analysis mixed studies specifically on cardiovascular disease with those
designed for other purposes (such as cancer prevention, weight loss, and
so on). It is questionable whether cardiovascular endpoints are
adequately ascertained in studies designed for other purposes.
Another problem of the meta-analysis is that it did not adequately
consider compliance of the subjects. Poor compliance to a low-fat diet is
a well-known problem in dietary trials. For example, the DART study (2)
was unable to achieve anywhere close to the goal set for total fat. This
and other trials showed minimal reduction in serum cholesterol with the
dietary intervention, demonstrating poor adherence. In two earlier trials
which showed cardiovascular benefits of unsaturated fats (3, 4), adipose
tissue fatty acid levels were used to monitor compliance. Although the
Finnish Mental Hospital Study (4) was not included in the meta-analysis as
it did not meet the subjective criteria, it did provide important evidence
for an effect of fat modification.
This meta-analysis highlights potential perils in combining “oranges”
and “apples” in aggregating dietary studies. The included trials are a
veritable hodge-podge of aims, methodologies, populations, and quality –
This same kind of approach was applied by one of the authors to
cholesterol lowering drug trials (5). The results from that meta-analysis
would predict that statin drugs would increase mortality in the 4S,
WOSCOPS, LIPID, and CARE trials, whereas the opposite in fact occurred.
Thus, similar degree of caution is needed to interpret results from either
a meta-analysis or an individual study.
Frank B. Hu, MD PhD
Frank Sacks, MD
Walter C. Willett, MD, DrPh
Dept. of Nutrition,
Harvard School of Public Health,
Boston, MA 02115
1. Bailar JC, 3rd. Passive smoking, coronary heart disease, and meta-
analysis. N Engl J Med 1999;340(12):958-9.
2. Burr ML, Fehily AM, Gilbert JF, Rogers S, Holliday RM, Sweetnam PM, et
al. Effects of changes in fat, fish, and fibre intakes on death and
myocardial reinfarction: diet and reinfarction trial (DART). Lancet
3. Dayton S, Pearce ML, Hashimoto S, Dixon WJ, Tomiyasu U. A controlled
clinical trial of a diet high in unsaturated fat in preventing
complications of atherosclerosis. Circulation 1969;40:(Suppl II):1-63.
4. Turpeinen O, Karvonen MJ, Pekkarinen M, Miettinen M, Elosuo R,
Paavilainen E. Dietary prevention of coronary heart disease: The Finnish
Mental Hospital Study. Int J Epidemiol 1979;8:99-118.
5. Smith GD, Song F, Sheldon TA. Cholesterol lowering and mortality: the
importance of considering initial level of risk. Bmj 1993;306(6889):1367-