Teleoanalysis: combining data from different types of study
BMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7415.616 (Published 11 September 2003) Cite this as: BMJ 2003;327:616All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The paper by Wald and Morris introduces a new term, teleoanalysis,
meaning complete analysis [1]. Complete analysis is not new: others have
called it multi-parameter evidence synthesis [2], generalised synthesis of
evidence [3], or the confidence profile method [4]. We believe that such
syntheses are valuable, but that they should incorporate a number of
improvements to the methods in this paper. We focus on the example of
folate supplementation and neural tube defects, which is drawn from a
previous paper by the same authors [5].
First, the available evidence should be used systematically. Wald and
Morris have carefully reviewed the effect of folate supplementation on
serum folate [5], but they use a single study to model the association
between serum folate and neural tube defects, and they do not include the
randomised evidence directly relating folate supplementation to neural
tube defects. While introducing the randomised evidence requires that
allowance is made for the distribution of pre-intervention folate levels
in the trial group and for incomplete compliance with the intervention, it
is nevertheless one of the best sources of evidence.
Second, extrapolation beyond the observed data should be avoided.
Wald and Morris' data quantify the effect of folate supplementation on
serum folate at doses up to 1 mg/day [5], yet they infer the effect of
doses up to 5 mg/day on neural tube defects.
Third, the full possibilities for the dose-response relationship
should be explored. Wald and Morris use particular linear and log-linear
model forms, and other model forms might provide an adequate or better fit
to the data while leading to very different conclusions.
Fourth, analysis should allow for the possibility of different
biological pathways. Wald and Morris assume that serum folate is a perfect
surrogate, but in fact the effect of folate supplementation may differ
from that predicted by its effect on serum folate alone.
Finally, we agree with the one-step method of analysis, but Wald and
Morris' WinBUGS program (on bmj.com) inadequately adjusts for regression
dilution bias [6]. The program appears to be written for the analysis of
individual data, but is applied to groups defined by quintiles of serum
folate. The true mean serum folate of individuals with observed serum
folate below 2 ng/mL, for example, is therefore wrongly constrained also
to lie below 2 ng/mL. Analysis should also acknowledge uncertainty in the
factor used to adjust for regression dilution bias.
A better approach to evidence synthesis is to integrate all the
evidence in a single model, and then to explore whether any elements are
inconsistent. Such an example is given by Ades [2].
[1] Wald NJ, Morris KK. Teleoanalysis: combining data from different
types of study. British Medical Journal 2003;327:616-618.
[2] Ades AE. A chain of evidence with mixed comparisons: models for
multi-parameter synthesis and consistency of evidence. Statistics in
Medicine 2003;22:2995-3016.
[3] Prevost TC, Abrams KR, Jones DR. Hierarchical models in
generalized synthesis of evidence: an example based on studies of breast
cancer screening. Statistics in Medicine 2000;19:3359-3376.
[4] Eddy DM, Hasselblad V, Shachter R. Meta-analysis by the
Confidence Profile Method. Academic Press: London, 1992.
[5] Wald NJ, Law MR, Morris JK, Wald DS. Quantifying the effect of
folic acid. Lancet 2001;358:2069-73.
[6] MacMahon S, Peto R, Cutler J et al. Blood pressure, stroke, and
coronary heart disease. Part 1, prolonged differences in blood pressure:
prospective observational studies corrected for the regression dilution
bias. Lancet 1990;335:765-774.
Competing interests:
None declared
Competing interests: No competing interests
Editor,
Teleology, the explanation of phenomena by the purpose they
serve” [Oxford Dictionary], does not explain how such associations arise.
Phenomena may be associated fortuitously, linked through unsuspected
factors or, possibly, causal. Teleology contributes to the formulation of
hypotheses that should be tested experimentally, or, in clinical matters
by controlled prospective clinical trials. In the ‘teleoanalysis’ of Wald
and Morris 1 on avoidance of folate deficiency for reduction in neural
tube risk, the fact that naturally increased folate levels, increased
dietary intake and trials of supplementation are each associated with
reduction in risk warrant this approach. 1
Readers should, however, beware being misled by equally strong
associations where the specific effects of supposedly active factors have
not been trialled. Two recent examples, affecting large proportions of the
population, illustrate the problems that can arise when benefits are
expected from interventions based on ‘intuitive’ beliefs arising from
observed associations. In the first, eating fruit and vegetables reduces
ischaemic heart disease [IHD] risk. Increased circulating beta-carotene,
associated with eating fruit and vegetables, is associated with reductions
in IHD. However, published data from some large controlled trials of
supplementation reported sufficiently marked adverse effects that two had
to be stopped. Both lung [and other] cancers were increased with
supplementation and in one trial the incidence of IHD increased.3, 4 It
now appears that other factors in fruit and vegetables may explain the
original associations. In the second example, the intuitive belief that
hormone replacement therapy could counter the increased risk of ischaemic
heart disease risk seen in post-menopausal women by increasing hormone
levels to match those in pre-menopausal women has now been tested and the
findings of prospective controlled trials suggest that the incidence of
IHD increases in long-term users of HRT.5 IHD risk will not be the only
factor influencing decisions on the use of HRT but the new findings will
contribute more effectively to decision making than the evidence quoted on
beta-carotene seems to have done; 30% of UK adults continuing to use
supplemental vitamins and anti oxidants bought in supermarkets, chemists,
pharmacies and health-food shops.5 This is probably because HRT requires
prescription in the UK whilst vitamin supplements remain unregulated.
Since interventions can have unexpected effects it would be
interesting to know whether Wald and Law take, or would take, the combined
protective drug ‘cocktail’ they hypothesise could reduce IHD risk by 80%,
on the basis of trials on each separately, 6a,b without knowing the
outcome of prospective trials of combined medication?
References
1. Wald NJ, Morris JK. Teleoanalysis: combining data from different types
of study. BMJ 2003;3276:616-18
2. Albanes D, Heinonen OP, Huttunen JK, Taylor PR, Virtamon J, Edwards
BK, Haapakoski J, Rautalahti M, Hartman AM, Palmgren J et al. Effects of
Alpha-Tocopherol Beta-Carotene Cancer Prevention study. Am J Clin Nutr
1995;62(6Suppl):1427S-30S
3. Pryor WA, Stahl W, Rock CL. Beta carotene: from biochemistry to
clinical trials. Nutr Reviews. 2000; 58(2 Pt1):39-53
4. Kritharides L, Stocker R. The use of antioxidant supplements in
coronary heart disease [Review]. Atherosclerosis. 2002;164(2):211-9
5. Herrington DM. From presumed benefit to potential harm – Hormone
therapy and heart disease New Eng J Med 2003; 349(6):519-221
6. (a). Wald NJ, Law MR. A strategy to reduce cardiovascular disease by
more than 80%. BMJ 2003;326:1419-23. 6(b). Editorial comment:- Rodgers A.
A cure for cardiovascular disease? BMJ 2003;326:1407-8]
Dr B J Boucher
Honorary Senior Lecturer; Centre for Diabetes & Metabolic Medicine
Royal London Hospital, London E11 1BB.
Email = bboucher@doctors.org.uk
Competing interests:
None declared
Competing interests: No competing interests
I must take issue with Wald and Morris's claim that they have
validated their approach. They tell us that, because their two-step and
one-step models give virtually identical results, they validate each
other. In fact, they do no such thing. Both models are based on the same
assumptions, so it is hardly surprising that they give similar results.
As Ravnskov correctly points out, the method of teleoanalysis is
based on rather shaky assumptions. Proper validation of the method would
require results from a trial that directly measures the effect that
teleoanalysis estimates, and a demonstration that the estimates from
teleoanalysis were similar to the effect observed in the trial.
Competing interests:
None declared
Competing interests: No competing interests
Wald and Morris [1] point out that teleoanalysis is different from
meta-analysis because it combines various categories of data rather than
one type of study. Teleoanalysis, therefore, is intuitively far more
reliable than meta-analysis of randomised trials as a guide of both
individual decisions and public-health policies, because the evidence
derived from those trials can be misleading, especially if their designs
are conceptually flawed.
For example, as Wald and Morris correcly observe, the effect of a
significant reduction in dietary fat on the risk of coronary heart disease
(CHD) can easily be underestimated, even when it is based on the results
of randomised trials. These investigations, indeed, by taking arbitrarily
for granted that 30% of energy as fat represents an ideal target, have
generally designed and studied "low-fat" diets containing at least 22-28%
of energy as fat. These percentages, however, are unnaturally high, not
low.
Conversely, a teleoanalysis aimed at clarifying the relation between
dietary fat and CHD will not ignore that humankind's metabolic physiology
has been genetically moulded by a low-fat nutritional environment, in
which, for millions of years, it was virtually impossible to consume the
currently advocated 30% of energy as fat, because game was very lean and
cattle-breeding, chicken farming, butter, dairy products, margarine, and
oils did not exist [2, 3].
Considering that humankind has been evolutionarily programmed for low
-fat diets [3], it is not surprising that populations consuming only 11-
13% of energy as fat are free of CHD [4]. In rural China, where the intake
of fat, per head, is less than half that of Americans and, consequently,
cholesterol levels are very low, CHD mortality is 16.7-fold lower than
that in the United States [2]. A really low-fat diet has a central role
not only in preventing CHD, but also in its reversal [5].
1. Wald NJ, Morris JK. Teleoanalysis: combining data from different
types of study. BMJ 2003;327:616-618 (13 September).
2. Baschetti R. Genetically unknown foods or thrifty genes? Am J Clin
Nutr 1999;70:420-421.
3. Baschetti R. Low-fat diets and HDL cholesterol. Am J Clin Nutr
1998;68:1143-1144.
4. Baschetti R. The low fat/low cholesterol diet. Eur Heart J
1997;18:1514-1515.
5. Ornish D, Scherwitz LW, Billings JH, et al. Intensive lifestyle
changes for reversal of coronary heart disease. JAMA 1998;280:2001-2007.
Competing interests:
None declared
Competing interests: No competing interests
Karl Popper would suggest that no hypothesis can ever be proven, for
certain. It is the lack of contradictory data that continues to strengthen
a hypothesis.
With regard to the diet-heart hypothesis, as mentioned in the article
by Wald and Morris, it would seem that teleoanalysis is a sophisticated
method of ignoring refutations to the hypothesis. 'Yes, we know that no
dietary intervention trial has ever shown any impact on CHD rates, but
this means nothing. If you analyse the results that would have come in,
from trials that heve never happened, we can see that all these earlier
trials were wrong. This is what we call teleoanalysis.'
I'm sorry, but I think I must have fallen down a rabbit hole without
realising it. I tried reading the article upside down, but it made no
sense that way either.
Competing interests:
None declared
Competing interests: No competing interests
Epidemiological
studies are useful for creating new hypotheses about disease causation. However,
to prove that a statistical association is causal demands experimentation,
because even the strongest association may be secondary to the real cause.
To bypass this elementary principle, Professor
Wald and Dr. Morris, representatives for a new branch in the medical sciences
named aetiological epidemiology, have invented a statistical instrument to
synthesise different categories of evidence.1 By teleoanalysis as
they call it, it is possible to measure the extent to which a disease can be
prevented, for instance how much ischaemic heart disease that can be avoided by
reducing dietary saturated fat. They admit that the crucial experiments, the
dietary trials, have failed, but as we “know” that saturated fat intake
increases the risk of heart disease, we can use a surrogate outcome, the
intermediate factor cholesterol as evidence, implying that we also “know”
that high cholesterol increases the risk. Say Law and Morris, the latter has
been shown in epidemiological cohort studies and in trials of cholesterol
lowering drugs.
These assumptions are false, however. The
allegation that patients with heart disease have eaten more animal fat than
healthy individuals has been falsified by at least 21 cohort studies and six
case-control studies including more than 150.000 individuals,2 and
other types of epidemiological studies have been just as unsuccessful. For
instance, dynamic population studies, including data from more than 100 time
periods in 35 countries, have found no association between changes of saturated
fat consumption and changes of coronary mortality, and in four cohort studies no
association was found between degree of atherosclerosis at autopsy and dietary
intake of saturated fat.2
The evidence from the cholesterol lowering drug
trials are just as invalid. Cholesterol lowering had no effect on heart
mortality before the statin era,3 and the effect of the statins is
most likely due to other, more beneficial effects on the cardiovascular system.
Dose-response has been claimed, but only by comparing mean values from the trials.
In all clinical and angiographic trials, where dose-response was calculated from
individual data, the effect was independent on the degree of cholesterol
lowering, a strong argument against causality.4,5
According to Wald and Morris the name
teleoanalysis was chosen to indicate thoroughness and completeness. It is
difficult to find a more misleading term considering that I have only mentioned
a few of the studies that contradict the diet-heart idea. There are many, many
more.4-7
- Wald
NJ, Morris JK. Teleoanalysis:
combining data from different types of study.BMJ
2003; 327: 616-618.[Full
text][PDF] - Ravnskov
U. The questionable role of saturated and polyunsaturated fatty acids in
cardiovascular disease. J Clin Epidemiol 1998;51:443-460. [Abstract] - Ravnskov
U. Cholesterol lowering trials in coronary heart disease: frequency of
citation and outcome. BMJ 1992;305: 15-19. [Abstract] - Ravnskov
U. A hypothesis out-of-date: The diet-heart idea. J Clin Epidemiol. 2002:1057-1063.
[Abstract] - Ravnskov
U. Is atherosclerosis caused by high cholesterol? QJM 2002; 95: 397-403. - THINCS,
The International Network of Cholesterol Skeptics; www.thincs.org - Ravnskov
U. The Cholesterol Myths. Washington DC: New
Trends Publishing, 2000.
Competing interests:
None declared
Competing interests: No competing interests
Is it teleoanalysis or teleanalysis?
My OED says "teleo- and before a vowel tele-, as in teleology" of which
many a medical student was brought up on.
Teleoanalysis of "studies that have have not been done" is a bit
suggestive of telepathy. All jokes aside, congratulations on a
magnificent paper. I prefer teleoanalysis though.
Competing interests:
None declared
Competing interests: No competing interests
How about some "negative controls" ?
Might teleoanalysis gain credibility if it replicated the "no
benefit" results of randomized trials that contradicted the rosy
predictions of earlier cohort data?
For example, what would teleoanalysis predict about the
cardiovascular and carcinogenic effects of post-menopausal estrogen +
progestin in healthy women? Would it distinguish these harmful effects
from the beneficial effects of HRT on bone-density?
And would teleoanalysis predict the uselessness of Vitamin E in the
prevention of cardiovascular disease?
Competing interests:
My competing interests are posted on the BMJ website at http://bmj.com/cgi/content/full/324/7336/539/DC1
Competing interests: No competing interests