Evaluating the causal relevance of diverse risk markers: horizontal systematic reviewBMJ 2009; 339 doi: http://dx.doi.org/10.1136/bmj.b4265 (Published 05 November 2009) Cite this as: BMJ 2009;339:b4265
- Hannah Kuper, senior lecturer1,
- Amanda Nicholson, clinical research fellow2,
- Mika Kivimaki, professor of social epidemiology2,
- Amina Aitsi-Selmi, academic clinical fellow in public health medicine2,
- Gianpiero Cavalleri, biomedical research lecturer3,
- John E Deanfield, professor of cardiology4,
- Peter Heuschmann, professor5,
- Xavier Jouven, professor of cardiology6,
- Sofia Malyutina, professor7,
- Bongani M Mayosi, professor of medicine8,
- Susanna Sans, director of monitoring and research programme on chronic diseases9,
- Troels Thomsen, consultant in preventive cardiology10,
- Jacqueline C M Witteman, professor of cardiovascular epidemiology11,
- Aroon D Hingorani, professor of genetic epidemiology2,
- Debbie A Lawlor, professor of epidemiology12,
- Harry Hemingway, professor of clinical epidemiology2
- 1Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London WC1E 7HT
- 2Department of Epidemiology and Public Health, University College London Medical School, London
- 3Molecular and Cellular Therapeutics, Royal College of Surgeons in Ireland, Beaumont Hospital, Dublin, Ireland
- 4Cardiothoracic Unit, Great Ormond Street Hospital for Children NHS Trust, London
- 5Center for Stroke Research Berlin (CSB), Charité-Universitätsmedizin Berlin, Germany
- 6INSERM, Villejuif, France
- 7Institute of Internal Medicine, Russian Academy of Medical Sciences, Novosibirsk, Russia
- 8Department of Medicine, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
- 9Instituto de Estudios de la Salud, Barcelona, Spain
- 10Research Centre for Prevention and Health, Glostrup University Hospital, Glostrup, Denmark
- 11Department of Epidemiology, Erasmus Medical Center, Rotterdam, Netherlands
- 12MRC CAiTE centre, Oakfield House, Bristol
- Correspondence to: H Kuper
- Accepted 12 July 2009
Objectives To develop a new methodology to systematically compare evidence across diverse risk markers for coronary heart disease and to compare this evidence with guideline recommendations.
Design “Horizontal” systematic review incorporating different sources of evidence.
Data sources Electronic search of Medline and hand search of guidelines.
Study selection Two reviewers independently determined eligibility of studies across three sources of evidence (observational studies, genetic association studies, and randomised controlled trials) related to four risk markers: depression, exercise, C reactive protein, and type 2 diabetes.
Data extraction For each risk marker, the largest meta-analyses of observational studies and genetic association studies, and meta-analyses or individual randomised controlled trials were analysed.
Results Meta-analyses of observational studies reported adjusted relative risks of coronary heart disease for depression of 1.9 (95% confidence interval 1.5 to 2.4), for top compared with bottom fourths of exercise 0.7 (0.5 to 1.0), for top compared with bottom thirds of C reactive protein 1.6 (1.5 to 1.7), and for diabetes in women 3.0 (2.4 to 3.7) and in men 2.0 (1.8 to 2.3). Prespecified study limitations were more common for depression and exercise. Meta-analyses of studies that allowed formal Mendelian randomisation were identified for C reactive protein (and did not support a causal effect), and were lacking for exercise, diabetes, and depression. Randomised controlled trials were not available for depression, exercise, or C reactive protein in relation to incidence of coronary heart disease, but trials in patients with diabetes showed some preventive effect of glucose control on risk of coronary heart disease. None of the four randomised controlled trials of treating depression in patients with coronary heart disease reduced the risk of further coronary events. Comparisons of this horizontal evidence review with two guidelines published in 2007 showed inconsistencies, with depression prioritised more in the guidelines than in our review.
Conclusions This horizontal systematic review pinpoints deficiencies and strengths in the evidence for depression, exercise, C reactive protein, and diabetes as unconfounded and unbiased causes of coronary heart disease. This new method could be used to develop a field synopsis and prioritise future development of guidelines and research.
Clinical guidelines for the prevention of coronary heart disease are important not only because they influence practice but also because they present a highly cited collation of evidence for a multitude of risk markers.1 2 For example, the European primary prevention guidelines published in 20031 mentioned more than 40 risk markers and have been cited more than 800 times.
A fundamental problem in developing rational clinical guidelines has been the lack of explicit, systematic comparisons of the strength of causal evidence across the diverse range of risk markers, which compete for clinical attention. Traditional vertical systematic reviews, which focus on one risk marker or a relatively homogeneous group of related risk markers, are an important influence on the development of clinical guidelines. However, individual risk markers may be championed by different experts, with few attempts at harmonising, displaying, and comparing the evidence across different markers. This may contribute to an ad hoc selection not based on strength of causal evidence, of which risk markers beyond smoking, blood pressure, and cholesterol concentration are included in guidelines. The European guidelines,1 for example, did not consider atrial fibrillation, unlike the contemporaneous American guidelines.2 The large3 and expanding array of risk markers underscores the importance of this problem, particularly since many of the markers are of uncertain causal relevance and few yet provide targets for prevention of disease.
We developed a new methodology of horizontal systematic review to assess causal relevance across a range of risk markers. We provide a high level overview of synthesised evidence, based on explicit criteria of biases and causal relevance. To show the potential of this approach we focused on four risk markers: depression, exercise, C reactive protein, and diabetes. We selected these four markers because they differ in several respects, including conceptual domain (psychosocial marker, behavioural marker, circulating biomarker, defined metabolic disease), measurement properties (presence or absence of standard instruments and internationally agreed definitions), and whether exposure was endogenous (proximal in the putative causal pathway) or exogenous (more distal). We purposively selected one risk marker—diabetes—widely accepted as having an established causal role, as well as three markers where the causal role is not universally accepted. We hypothesised that concordance of research evidence from differing research designs each with different sources of error provides the strongest evidence on the causal relevance of a putative risk factor.4 Specifically, we sought evidence from three major study designs that offer different approaches to tackling confounding and reverse causation: traditional prospective observational studies with multivariate adjustments, studies that use genetic variants as instruments to tackle confounding (so called Mendelian randomisation),5 6 and randomised controlled trials where exposure to the risk marker is experimentally manipulated. Finally, in the light of the horizontal comparison we compared the recommendations made for these four risk markers in the most recent guidelines on prevention of coronary heart disease.
The horizontal systematic review assesses causal relevance across a range of risk markers and study designs. We set out a priori eligibility criteria for studies, systematically obtained the studies, and extracted and displayed the data. Firstly, we separated information on risk markers for first coronary heart disease events in people initially free from clinical disease and prognostic factors in patients with existing coronary heart disease because of the clinical importance of distinguishing between primary and secondary prevention. Secondly, where more than one systematic review was identified we displayed the largest meta-analysis or study (which tended to be better quality according to the MOOSE,7 and QUORUM8 statements) in the tables of main results and included details of the others in the web extra appendix. Thirdly, we agreed a priori that if we could find no systematic review for any randomised controlled trials of the risk marker then we would review the largest individual study. Fourthly, we stratified data extraction and synthesis by study design (observational studies, genetic association studies, and randomised controlled trials), but analysed horizontally.
In January 2008 we searched Medline to identify meta-analyses of observational studies in healthy populations (aetiologic) and among patients with existing coronary disease (prognostic), and contacted experts. Existing coronary disease included patients with myocardial infarction or those undergoing coronary revascularisation or coronary angiography. Search terms included the expanded medical subject heading (MESH) of cardiovascular disease, meta-analysis as a MESH topic or publication type, and then the four individual risk markers as either the MESH term or text words. English and non-English language publications were eligible. Eligible outcomes were fatal coronary heart disease and non-fatal myocardial infarction (aetiologic and prognostic studies) and, for prognostic studies only, all cause mortality. Meta-analyses were only eligible for inclusion if they reported summary estimates based on longitudinal studies.
Two reviewers (HK and AAS) extracted data, with recourse to a third reviewer in the event of disagreement. We extracted summary data on prespecified items: age adjusted (or unadjusted) relative risks with 95% confidence intervals; adjusted relative risks with 95% confidence intervals; number of studies adjusting for smoking, blood pressure, and cholesterol concentration (aetiologic) or disease severity (prognostic); attenuation between age and multivariate adjusted relative risks; the prevalence of exposure in individual studies; the methods used to measure exposure; the number and type of outcomes; the level of meta-analysis (literature only, or pooled analysis of individual participant data across the studies); measure of heterogeneity; whether separate estimates were reported among people aged over 75 years, women, or non-Western populations; evidence of the presence of a dose-response relation—that is, extending beyond dichotomous comparisons; the extent to which the duration of follow-up influenced the strength of the estimates (if effects are stronger with shorter periods of follow-up this is consistent with reverse causality); and evidence of publication bias.
We searched for meta-analyses of the associations between genotype and each of the four individual risk markers through Medline using gene as the MESH term or text words or Mendelian in any field; meta-analysis as a MESH topic or publication type; and then the four individual risk markers as either the MESH term or text words. To identify meta-analyses of the association between genotype and coronary heart disease outcome we searched individual single nucleotide polymorphisms in all fields identified from recent systematic reviews for depression (SLC6A4, MTHFR, APOEw1), the CRP gene,w2 and type 2 diabetes (TCF7L2, FTO, CDKN2A/CDKN2B, PPARG, ICF2BP2, KCNJ11, HHEXIDE, CDKAL1, SLC30A8w3) together with expanded MESH headings of cardiovascular disease and meta-analysis as a MESH topic or publication type. For each variant two independent reviewers (HH and AN) extracted information on prespecified items: whether the single nucleotide polymorphism was identified from genome wide scans, the number of outcome events, the number of studies in the meta-analysis, unadjusted relative risks (95% confidence intervals), and whether there was a formal test on the use of the genetic variant as an instrumental variable.6
Randomised controlled trials and meta-analyses of these were identified through searches of Medline. Search terms included the MESH heading of coronary heart disease or CHD or myocardial infarction or MI; trial as a MESH heading; the four individual risk markers as either the MESH term or text words; and meta-analysis or systematic review as a MESH heading or review as a publication type. Given the importance of evidence from randomised controlled trials to inform guidelines, we accepted individual trials where no meta-analyses were available. We also searched through the reference list from the guideline publications to identify relevant randomised controlled trials. Only randomised controlled trials that reported coronary heart disease event outcomes were eligible (aetiologic and prognostic studies) or death in the setting of patients with coronary heart disease (prognostic studies). Two independent reviewers (HK and AAS) extracted details on the nature of the intervention, the number of studies in the meta-analysis, the number and type of end points, the relative risk (95% confidence interval) of coronary heart disease or death, and whether the intervention had an effect on the risk marker.
Selection of guidelines
To make contemporaneous comparisons with our evidence review we included guidelines only published in 2007 as this was the most recent information available to us. We identified two guidelines, which were developed through independent processes, from the Scottish Intercollegiate Guideline Network9 and the fourth Joint European Societies10 11 (coordinated by the European Society of Cardiology, and representing nine professional organisations). Across each guideline and risk marker we compared the evidence cited, description of the causal relevance of the marker, recommendations on measurement in healthy population settings, inclusion in risk scores, recommendations for specific interventions, and target levels or goals for risk marker levels.
The figure⇓ shows the results of the meta-analyses of observational studies, genetic variants, and randomised controlled trials for depression, exercise, C reactive protein, and diabetes in relation to risk of coronary heart disease. These risk markers differed noticeably in the type and amount of evidence identified.
Table 1⇓ summarises the largest meta-analyses found for each of the risk markers. Meta-analyses of observational studies reported adjusted aetiologic relative risks of coronary heart disease for depression of 1.9 (95% confidence interval 1.5 to 2.4; 1262 events in 11 studies), for top compared with bottom fourth of exercise of 0.7 (0.5 to 1.0; 500 events in three studies), for top compared with bottom third of C reactive protein of 1.6 (1.5 to 1.7; 7068 events in 22 studies), and for diabetes in women of 3.0 (2.4 to 3.7) and in men of 2.0 (1.8 to 2.3; >4964 events in 29 studies). Meta-analyses of the association in patients with coronary heart disease (prognostic studies) reported an adjusted relative risk of further coronary heart disease events or death for depression of 1.6 (1.3 to 1.9; >525 events in 11 studies) and an adjusted relative risk of cardiac death for diabetes of 1.5 (1.1 to 2.0; 240 events in four studies). No prognostic meta-analyses were identified for the effect of exercise or C reactive protein on outcome among patients with coronary heart disease. A dose-response effect was reported for intensity of physical activity but not consistently for major depression compared with minor depression, concentration of C reactive protein, or glucose control in people with diabetes. None of the meta-analyses reported effects separately among those older than 75 years (or other age groups of older people), and only for diabetes was there evidence of consistent effects in non-Western populations. Effect estimates for C reactive protein and diabetes were presented separately for men and women, with observational evidence for diabetes showing a stronger association with coronary heart disease among women than among men. No sex differences were found for the association of C reactive protein with coronary heart disease. The exercise meta-analysis was restricted to women and the depression meta-analysis did not report effects separately in women and men.
Confounding—Adjustments for a priori confounders of smoking, blood pressure, and cholesterol concentration were found in four of 22 aetiologic studies for depression and coronary heart disease and one of three studies for physical activity. Adjustments in the C reactive protein studies (20/22) and diabetes studies (26/37) were generally more consistent and complete, although beyond smoking it was unclear which variables were included in multivariate analyses. For aetiological meta-analyses of the four risk markers, the effect on coronary heart disease was apparent after multivariable adjustments. Reporting of unadjusted or age adjusted and multivariate adjusted results was inconsistent.
Biases—Statistical heterogeneity, present in all the meta-analyses, was partly attributable to differences in measurement of exposure for depression and physical activity, and year of publication for C reactive protein. Depression was defined by 12 different methods, but relatively standardised methods were used for measurement of C reactive protein concentration and diabetes. For depression, but not for C reactive protein or diabetes, stronger effects were observed with shorter follow-up. Adjustment for severity of coronary heart disease in prognostic studies reduced the relative risk for depression by 45%. Evidence of small study bias (often indicative of publication bias or a strong association between methodological weakness and a non-null association in the expected direction in smaller studies compared with larger studies) was present for depression but absent for physical activity, diabetes, and C reactive protein.
Other meta-analyses of observational studies—The findings of the meta-analyses of observational studies included in our analysis were consistent with results of other eligible meta-analyses on this subject (see web extra appendix). For depression, adjusted relative risks ranged between 1.5 (1.4 to 1.6) and 1.9 (1.5 to 2.4). Similarly, a consistently strong association was found between diabetes and the incidence and mortality after coronary heart disease, with stronger relations identified for women than for men. Exercise was consistently protective for coronary heart disease, ranging from 0.5 (0.5 to 0.6) to 0.8 (0.6 to 1.2). Other meta-analyses confirmed the increased risk for coronary heart disease for people in the highest third of C reactive protein concentrations.
Table 2⇓ summarises the results of our search for meta-analyses of genetic variants, indexing differences in the risk markers of interest. Meta-analyses for two genetic variants were identified, which have been investigated in relation to both depression and coronary heart disease: MTHFR12 and APOE.13 14 The MTHFR variant was positively associated with both depression and coronary heart disease, whereas the ε2 APOE genotype was linked to a reduced risk of depression and of coronary heart disease.12 15 w4 No replicated genetic variant for physical activity was identified that could be assessed in relation to coronary heart disease. We identified two syntheses of a genetic variant in the CRP gene in which the expected relation for coronary heart disease events under a causal model, in the light of its effect on C reactive protein levels, was not observed.16 w2 Despite the large number of meta-analyses showing genes associated with diabetes, none examined the association of these variants with coronary heart disease.
Randomised controlled trials
We identified four randomised controlled trials of different interventions for the treatment of depression among patients with coronary heart disease (table 3⇓). None of these trials showed a beneficial effect on death or cardiovascular events.w5-w8 No randomised controlled trials were identified on the effect of treating depression in healthy populations (or on the effect of prevention of depression) in relation to risk of coronary heart disease. No randomised controlled trials were found of interventions that specifically increased physical activity (in isolation from, for example, improvements in diet, or compliance with drugs) in healthy populations, whereas exercise based rehabilitation reduced the risk of mortality among people with coronary heart disease (0.7, 0.6 to 1.0).w9 Currently there are no interventions that specifically lower C reactive protein levels, and hence no randomised controlled trials that could test the causal importance of C reactive protein in coronary heart disease. Randomised controlled trials (four meta-analyses and 70 individual trials) of different hypoglycaemic agents in patients with diabetes without or with manifest coronary heart disease provided some, but not consistent, support that lowering glucose concentrations reduced the rate of coronary heart disease events.w17-w21
Neither the Scottish Intercollegiate Guideline Network nor the European guidelines adopted an explicit method for displaying and comparing evidence across risk markers (table 4⇓). The Scottish but not European guidelines reported a level of evidence for some statements. Both guidelines made clear, albeit differing, statements on the importance of depression in the onset and progression of coronary heart disease. Neither the only randomised trial with statistical power to detect differences in event rates (ENRICHD)w5 nor the Cochrane meta-analysis of psychological interventions17 was cited in the executive summary of the European guidelines. Post hoc subgroup analyses were cited.18 The Scottish guidelines cited neither trials nor meta-analyses but did cite a previous position statement, which itself cites only an older narrative systematic review. Neither guideline suggested that C reactive protein was an important risk marker: for the Scottish guidelines it was neither mentioned nor any rationale given as to why not. Observational studies and Mendelian randomisation studies were cited for the European guidelines meta-analyses, and the association was stated as “often seriously confounded.”
The horizontal systematic review is a new method to compare the evidence on diverse risk markers in a unified explicit framework of the largest available syntheses of the most important forms of evidence. This approach highlighted differences and deficiencies in the evidence of causal relevance across the four selected risk markers: psychosocial, behavioural, biomarker, and metabolic disease. The evidence that depression, low physical activity, or C reactive protein concentration causes coronary heart disease seems less strong than that for diabetes. Randomised trials of specific interventions are lacking for C reactive protein and null for depression, and although they support the role of exercise in the secondary prevention of coronary heart disease they are not available to test the causal hypothesis. Neither the European nor Scottish guidelines gave explicit criteria for assessing evidence to enable prioritisation of the impact of individual risk markers. However the emphasis given to the causal and clinical relevance of depression in the guidelines was inconsistent with the available evidence.
Closing the translational gap
Discordance exists between the large number of markers that are associated with coronary heart disease3 and the small number of targets for intervention. This highlights the need for our approach, which aims to prioritise targets. Our approach is horizontal in two senses: a comparison was made across diverse markers and a comparison was made across different forms of evidence. We selected four risk markers as examples; the approach is scaleable to all risk markers.
Three complementary designs for obtaining causal evidence
Associations between putative risk markers and coronary heart disease are easy to show in observational studies but may be confounded, as has been shown by negative trials of hormone replacement therapy and vitamins on coronary heart disease.19 We marshalled three approaches, which aimed, with varying limitations, to mimic the ideal, unconfounded experiment; prospective observational studies (multivariate adjustment for confounders), genetic studies (which utilise genetic variants that influence the modifiable exposure and that are assigned at random and can therefore be used as an instrument for the unconfounded and unbiased association of the genetic variant with the outcome of interest), and randomised trials (where the investigator influences exposure). Low density lipoprotein cholesterol provides an example with converging evidence from all these approaches: robust associations between high concentrations of low density lipoprotein cholesterol and coronary heart disease shown in observational cohort studies20; genetic variants that relate to lower concentrations of low density lipoprotein cholesterol (for example, in PCSK9,21 the APOE,22 and LDL receptor gene)23 also found to be associated with a reduced risk of coronary heart disease; and trials on low density lipoprotein cholesterol lowering confirmed the protection against coronary heart disease.24 We excluded study designs that may be associated with lower validity, such as individual observational studies that have not (yet) been synthesised, non-randomised trials, and studies on biological mechanisms. Such studies have been the basis for guideline recommendations.25 We included the forms of large scale evidence, which aims to evaluate the causal hypothesis, with a low tolerance for false positive findings. Our approach could be extended to incorporate small scale experimental studies in humans (as part of a “teleoanalysis” approach)26 and, with due caveats,27 experimental studies in animals. Such extensions to other forms of evidence should acknowledge that studies in the discovery phase have a higher tolerance for false positive findings, as the aim is not to abandon a potentially important risk marker prematurely.28
Risk markers were associated with relative risks from 1.5 (C reactive protein) to over 3 (diabetes), but adjustment for established risk factors of smoking, blood pressure, and cholesterol concentration was least common among studies of depression and exercise. These psychosocial (depression) and behavioural (exercise) factors were also more prone to information biases, with multiple instruments used to determine exposure. There was evidence of reverse causality and publication bias for depression. We found no meta-analyses in a prognostic setting of C reactive protein or exercise. Given that the guidelines make recommendations in secondary prevention and since aetiological markers may not necessarily be prognostic, this lack of synthesised evidence is important. Thus, for example, meta-analyses of body mass index in the prognosis of patients with coronary disease suggest no adverse effect for obesity,29 whereas those for aetiological associations show an increased risk.30
Genetic studies using Mendelian randomisation have been more frequently applied to assessing C reactive protein31 w2 than for our other three risk factors. Despite being relatively underpowered, the emerging evidence does not suggest an important role for C reactive protein in causing coronary heart disease. A new and large collaboration should provide a more definitive answer in the near future.32 For depression, exercise, and diabetes, evidence from Mendelian randomisation for their causal role in risk of coronary heart disease was limited. The robust positive associations of MTHFR with both depression and coronary heart disease could indicate a causal effect of depression on coronary heart disease but is more likely to reflect folate intake and metabolism as a causal marker for both outcomes. The emergence of whole genome association studies and complete genome sequencing is improving our understanding of the genomic architecture underlying complex traits. Mendelian randomisation may offer a powerful tool to understand causality, particularly for risk marker traits that are controlled by a limited number of genetic variants of relatively strong effect.
Randomised controlled trials
Successful treatment of depression in patients with established coronary heart disease in randomised controlled trials does not show benefits in subsequent death or rates of coronary heart disease events. This provides no support for the causal hypothesis that avoiding depression is important in the secondary prevention of coronary heart disease, but it is a matter of debate whether it provides evidence against. For example, it might be argued that it is the intervention rather than the hypothesis that is wrong. Trials were lacking for the effect of C reactive protein concentration or physical activity among healthy populations, whereas there was evidence that glucose control may provide some reduction in coronary heart disease events. Interpretation was difficult in the situation of trials with pleiotropic effects; for instance, the evidence that exercise in combination with other aspects of rehabilitation reduced the risk of death among patients with coronary heart disease.w9 A new specific C reactive protein inhibitor drug is being used to determine the functions of C reactive protein in the experimental setting and will be tested in the setting of acute coronary events. However, the lack of oral bioavailability and short half life currently precludes its use in long term prevention trials in humans.33 Randomised controlled trials of lipid lowering statins are a non-specific test of the role of C reactive protein and coronary heart disease, because of the major effect on low density lipoprotein concentrations.34 It has been shown that false findings from observational studies continue to be influential, despite being contradicted by randomised trial evidence.35 Null randomised trials have led to revisions in the causal and mechanistic hypotheses—for example, the finding that positive inotropic agents do not prolong life in heart failure, refocused attention away from a mainly haemodynamic model of heart failure.36
Clinical implications and consistency of the guidelines
By using horizontal systematic reviews, clinicians, guideline developers, funders of research, public health policy makers, and journal editors and their peer reviewers might be aided in making more consistent and less biased decisions. The graphical summary of evidence may serve a practical purpose in guideline groups, facilitating more explicit debate of the importance of risk markers across the multiple fields of interest of contributors. The two most recent guidelines on primary prevention of cardiovascular disease refer to a wide “penumbra” of risk markers beyond smoking, blood pressure, and cholesterol concentration; over 50 markers in the most recent European guidelines.1 The guidelines cite more than 1100 references (joint societies) and 315 references (Scottish Intercollegiate Guideline Network) but do not provide a systematic comparison of the quality or strength of evidence across the risk markers that were included. There was inconsistency in the conclusions reached by the two guidelines across the four risk markers that we evaluated. For instance, C reactive protein was not considered by the Scottish Intercollegiate Guideline Network. Depression was accorded higher prominence in the European guidelines than in the Scottish guidelines and within the European guidelines was accorded higher prominence than C reactive protein, which is not consistent with the evidence in the horizontal systematic review. Depression is worth treating in its own right, irrespective of any causal relation with coronary heart disease; but the same is true for other conditions, such as chronic obstructive airways disease, which are not mentioned in the guidelines.
Limitations of this horizontal systematic review
There are important limitations in this initial illustration of a horizontal systematic review. Firstly, the method depends on the availability and quality of large scale syntheses of evidence. These are more commonly available for blood based markers than for behavioural or psychosocial markers; horizontal systematic reviews may stimulate research groups to raise or defend the profile of research in their subdisciplines. A range of measures of effect were included in the reviews, and where the confidence intervals for the effect estimates were wide this precluded reaching firm conclusions. Increasing use of horizontal systematic reviews may provide an impetus to improving the number and quality of meta-analyses, particularly those using individual participant data. Secondly, the horizontal systematic review is narrative, without novel methods for data analysis, offering no explicit ranking of causal relevance nor attempting to posit a decision threshold above which a marker might be considered causal.
Research implications and need for unbiased field synopses
Further research is required to develop the method of horizontal systematic review. Firstly, methods could be developed to derive relative weights of evidence building on the judgments of groups of experts,37 Bayesian methods could be used for the synthesis of evidence, or models could be developed to combine features from different studies to derive quantitative estimates.38 Secondly, extension is required to the whole range of risk markers that are included in guidelines, thus providing a systematic synopsis of the specialty. Thirdly, extension to other chronic diseases should be explored—for example, in the specialty of cancer, horizontal systematic reviews could build on the assessment of causality used by the International Agency for Research on Cancer. Fourthly, horizontal systematic reviews should be regularly updated as evidence changes to minimise the lag time between the generation of evidence and the development of guidelines and could take advantage of continually updated databases of genetic studies in this process. The conclusions of our review are not altered if the publication year of evidence is truncated two years before the publication of guidelines—that is, 2005. Fifthly, there are important considerations beyond causal relevance when developing guidelines, such as economic considerations and the additional deleterious effects of the risk markers (for example, the impact of depression on quality of life), and the framework could be extended to encompass these considerations. Sixthly, it should be noted that non-causal markers can be used in risk prediction (for example, socioeconomic position) and this requires distinct consideration in the observational evidence.
Horizontal systematic review in which the causal relevance of diverse risk markers is compared in an explicit framework helps clarify the relative standing of each risk marker. Field synopses, expanded to include the whole range of risk markers considered of potential clinical or public health relevance, should be developed to prioritise research efforts and to focus recommendations on those markers most likely to be causal.
What is already known on this topic
Diverse psychosocial, behavioural, and biological markers are claimed to be independently associated with coronary heart disease and are included in guidelines
Traditional vertical systematic reviews focus on one risk marker and one research design at a time
Horizontal comparisons across different types of risk markers, incorporating different research designs each with differing limitations, are lacking
What this study adds
Observational evidence from horizontal systematic review was strongest for diabetes and C reactive protein concentration as risk markers for coronary heart disease
Evidence from Mendelian randomisation was present for C reactive protein only, which did not support a causal association with coronary heart disease
Randomised trial evidence was lacking for C reactive protein and did not show a protective effect in coronary heart disease from treating depression; for no risk marker did it provide strong support for the causal hypothesis
Cite this as: BMJ 2009;339:b4265
This work was developed during meetings of the Nucleus of Epidemiology and Public Health, European Association of Cardiovascular Prevention and Rehabilitation, European Society of Cardiology in Paris 2007 and Sofia Antipolis 2008 (HH chair, members GC, JED, PH, XJ, SM, BMM, SS, TT, JCMW, ADH, and DAL).
Contributors: The writing group included HK, AN, MK, and HH. HK, AN, and MK contributed equally to the study. HK, HH, AN, and AAS carried out the searches and extracted the data. All authors took part in the discussion group where the new methodology was developed, commented on early drafts of the manuscript through to revisions, approved the final draft, had full access to all the data, and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding: MK is supported by the Academy of Finland. HK is supported by a grant from the Wellcome Trust. DAL is supported by a UK Department of Health Career Scientist Award and works in a centre that receives support from the UK Medical Research Council. ADH is supported by a British Heart Foundation senior research fellowship (FS 05/125).
Competing interests: ADH is a member of the editorial board of Drug and Therapeutics Bulletin and has acted as an adviser to GlaxoSmithKline and London Genetics. He has received honorariums for speaking at educational meetings sponsored by the pharmaceutical industry and has donated all or most of these to charity.
Ethical approval: Not required.
Data sharing: No additional data are available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.