Tillmann, et al. [1]’s analysis of the effects of education on coronary heart disease (CHD) makes some strong claims about the causality of their estimated effects. These claims are based on the presumption that Mendelian randomisation removed confounding in the relationship between education and CHD. There are at least two good reasons to be sceptical of this assumption and we strongly contend that these should not be viewed as causal effect estimates. First we’ll examine the author’s assumptions about confounding, then we’ll examine the results that they present.
Mendelian randomisation is an application of instrumental variable analysis, which economists have long used to study causal effects where the potential for interventional research is limited, such as education [2]. As with any other instrumental variable analysis, the validity of Mendelian randomisation depends critically on the absence of a relationship between the instrument (genes) and the outcome (coronary heart disease), other than that which operates through the exposure (education). Richards and Evans [3] break this assumption down further into an absence of direct effects (“assumption 2”) and an absence of association with confounders (“assumption 3”).
In most Mendelian randomisation studies, the prime suspect for violation of assumption 2 is genetic pleiotropy—multiple actions of single genes. Tillmann, et al. [1] go to great lengths to test for genetic pleiotropy but this is misdirected, as we shall explain. Richards and Evans [3] propose that assumption 3—association with confounders—is unlikely to be violated by “all or even most of the 162 genetic determinants of educational attainment” (p.2). We counter that it is highly likely that every single ‘genetic determinant of educational attainment’ breaks one or other of these assumptions. The reason for this is simply that there is no plausible explanation presented by the authors or conceivable to us which doesn’t. Every possible way that a gene could be associated with educational attainment involves either mediators that are themselves potential confounders of the relationship between education and CHD, or confounders of the gene–education relationship that may also confound the education–CHD relationship. Richards and Evans [3] dismissal of potential confounding by cognitive ability ignores the fact that something must fill the gap between genes and education, because genes cannot possibly affect education directly.
The most plausible mechanism for explaining a relationship between genes and education is an effect of the genes on brain function, whether it be academic intelligence, mental health, behaviour or temperament (Figure 1). Each of these is likely to influence educational attainment and could conceivably also influence CHD. What’s important to note is that these do not represent genetic pleiotropy; they require only a single effect of a gene on a physiological mediator, and it is the physiological mediators that in turn affect both educational attainment and CHD. The authors do discuss the literature on twin studies, which may reduce this type of confounding, but Mendelian randomisation cannot.
The other plausible explanations for associations between genes and education is what Tillmann, et al. [1] acknowledge as ‘dynastic effects’, such as “when parental genes associate with parental behaviours that directly cause [both education and] a health outcome in the child”. Mendelian randomisation studies often rely on huge samples to overcome the limitations of genetic instruments that are only weakly correlated with the exposures of interest. When the exposure and outcome variables are both correlated with ethnicity (as education and CHD are), then dynastic effects may be a significant limitation. In the absence of ethnic or cultural control, genetic instruments may simply be detecting cultural confounding. The authors’ argument about the weak relationship between parents’ and offspring’s education is not directly relevant because it is the relationship between parents’ genes and offspring’s education that is of most concern.
Figure 1 Alternative causal diagram to explain associations between genes, education and coronary heart disease (CHD). Note: there can be no direct effect of genes on education. https://ibb.co/fNtxJa
Now let’s examine the estimates presented. Given the depressingly modest effect sizes seen in most dietary and lifestyle interventions that target reductions in CHD specifically, the large effect size produced by this analysis should have raised a red flag. An even bigger red flag is raised by examining the relative effect sizes estimated by the authors in their ‘observational’ and ‘causal’ analyses; the Mendelian randomisation estimates are stronger than the minimally adjusted regression estimates. If the Mendelian randomisation estimates are truly causal effects, then the unobserved confounders in the regression estimates (e.g. socioeconomic status) must be inversely correlated with education and health. This is generally not the case; confounders like socioeconomic status affect education and health in the same direction and bias regression estimates away from the null. Thus, the authors’ own results provide evidence to indicate that this is an example of an invalid instrument yielding biased results.
Mendelian randomisation is an extremely valuable analysis tool, but this application has stretched it beyond its useful limits. The gap in the causal chain between genes and educational is too great. Mendelian randomisation should be used to explore modifiable factors closely related to physiological phenotypes and environmental factors far removed should be left for other designs. Strong claims of causality cannot be justified when the assumptions required for instrumental variable analysis are so easily violated. For further discussion of the sensitivity of education effects to instrumental variable selection, see Blundell, et al. [2] and for a more general guidance on instrumental variable analysis and its limitations, to Angrist and Pischke [4].
James Doidge*, Administrative Data Research Centre for England, University College London; Centre for Population Health Research, University of South Australia
Lorraine Dearden, Institute for Fiscal Studies; UCL Institute of Education, University College London
*Correspondence to J.Doidge@ucl.ac.uk, 222 Euston Road, London NW1 2DA, UK
References
1. Tillmann T, Vaucher J, Okbay A, et al. Education and coronary heart disease: Mendelian randomisation study. BMJ 2017;358 doi: 10.1136/bmj.j3542
2. Blundell R, Dearden L, Sianesi B. Evaluating the effect of education on earnings: Models, methods and results from the National Child Development Survey. Journal of the Royal Statistical Society: Series A (Statistics in Society) 2005;168(3):473-512. doi: 10.1111/j.1467-985X.2004.00360.x
3. Richards JB, Evans DM. Back to school to protect against coronary heart disease? BMJ 2017;358 doi: 10.1136/bmj.j3849
4. Angrist JD, Pischke J-S. Instrumental variables in action: Sometimes you get what you need. Mostly harmless econometrics: An empiricist's companion: Princeton University Press 2009.
Competing interests:
No competing interests
11 September 2017
James C Doidge
Academic
Lorraine Dearden (Institute of Fiscal Studies; and UCL Institute of Education, University College London)
Administrative Data Research Centre for England, University College London; and Centre for Population Health Research, University of South Australia
Rapid Response:
Beyond the limits of Mendelian randomisation
Tillmann, et al. [1]’s analysis of the effects of education on coronary heart disease (CHD) makes some strong claims about the causality of their estimated effects. These claims are based on the presumption that Mendelian randomisation removed confounding in the relationship between education and CHD. There are at least two good reasons to be sceptical of this assumption and we strongly contend that these should not be viewed as causal effect estimates. First we’ll examine the author’s assumptions about confounding, then we’ll examine the results that they present.
Mendelian randomisation is an application of instrumental variable analysis, which economists have long used to study causal effects where the potential for interventional research is limited, such as education [2]. As with any other instrumental variable analysis, the validity of Mendelian randomisation depends critically on the absence of a relationship between the instrument (genes) and the outcome (coronary heart disease), other than that which operates through the exposure (education). Richards and Evans [3] break this assumption down further into an absence of direct effects (“assumption 2”) and an absence of association with confounders (“assumption 3”).
In most Mendelian randomisation studies, the prime suspect for violation of assumption 2 is genetic pleiotropy—multiple actions of single genes. Tillmann, et al. [1] go to great lengths to test for genetic pleiotropy but this is misdirected, as we shall explain. Richards and Evans [3] propose that assumption 3—association with confounders—is unlikely to be violated by “all or even most of the 162 genetic determinants of educational attainment” (p.2). We counter that it is highly likely that every single ‘genetic determinant of educational attainment’ breaks one or other of these assumptions. The reason for this is simply that there is no plausible explanation presented by the authors or conceivable to us which doesn’t. Every possible way that a gene could be associated with educational attainment involves either mediators that are themselves potential confounders of the relationship between education and CHD, or confounders of the gene–education relationship that may also confound the education–CHD relationship. Richards and Evans [3] dismissal of potential confounding by cognitive ability ignores the fact that something must fill the gap between genes and education, because genes cannot possibly affect education directly.
The most plausible mechanism for explaining a relationship between genes and education is an effect of the genes on brain function, whether it be academic intelligence, mental health, behaviour or temperament (Figure 1). Each of these is likely to influence educational attainment and could conceivably also influence CHD. What’s important to note is that these do not represent genetic pleiotropy; they require only a single effect of a gene on a physiological mediator, and it is the physiological mediators that in turn affect both educational attainment and CHD. The authors do discuss the literature on twin studies, which may reduce this type of confounding, but Mendelian randomisation cannot.
The other plausible explanations for associations between genes and education is what Tillmann, et al. [1] acknowledge as ‘dynastic effects’, such as “when parental genes associate with parental behaviours that directly cause [both education and] a health outcome in the child”. Mendelian randomisation studies often rely on huge samples to overcome the limitations of genetic instruments that are only weakly correlated with the exposures of interest. When the exposure and outcome variables are both correlated with ethnicity (as education and CHD are), then dynastic effects may be a significant limitation. In the absence of ethnic or cultural control, genetic instruments may simply be detecting cultural confounding. The authors’ argument about the weak relationship between parents’ and offspring’s education is not directly relevant because it is the relationship between parents’ genes and offspring’s education that is of most concern.
Figure 1 Alternative causal diagram to explain associations between genes, education and coronary heart disease (CHD). Note: there can be no direct effect of genes on education. https://ibb.co/fNtxJa
Now let’s examine the estimates presented. Given the depressingly modest effect sizes seen in most dietary and lifestyle interventions that target reductions in CHD specifically, the large effect size produced by this analysis should have raised a red flag. An even bigger red flag is raised by examining the relative effect sizes estimated by the authors in their ‘observational’ and ‘causal’ analyses; the Mendelian randomisation estimates are stronger than the minimally adjusted regression estimates. If the Mendelian randomisation estimates are truly causal effects, then the unobserved confounders in the regression estimates (e.g. socioeconomic status) must be inversely correlated with education and health. This is generally not the case; confounders like socioeconomic status affect education and health in the same direction and bias regression estimates away from the null. Thus, the authors’ own results provide evidence to indicate that this is an example of an invalid instrument yielding biased results.
Mendelian randomisation is an extremely valuable analysis tool, but this application has stretched it beyond its useful limits. The gap in the causal chain between genes and educational is too great. Mendelian randomisation should be used to explore modifiable factors closely related to physiological phenotypes and environmental factors far removed should be left for other designs. Strong claims of causality cannot be justified when the assumptions required for instrumental variable analysis are so easily violated. For further discussion of the sensitivity of education effects to instrumental variable selection, see Blundell, et al. [2] and for a more general guidance on instrumental variable analysis and its limitations, to Angrist and Pischke [4].
James Doidge*, Administrative Data Research Centre for England, University College London; Centre for Population Health Research, University of South Australia
Lorraine Dearden, Institute for Fiscal Studies; UCL Institute of Education, University College London
*Correspondence to J.Doidge@ucl.ac.uk, 222 Euston Road, London NW1 2DA, UK
References
1. Tillmann T, Vaucher J, Okbay A, et al. Education and coronary heart disease: Mendelian randomisation study. BMJ 2017;358 doi: 10.1136/bmj.j3542
2. Blundell R, Dearden L, Sianesi B. Evaluating the effect of education on earnings: Models, methods and results from the National Child Development Survey. Journal of the Royal Statistical Society: Series A (Statistics in Society) 2005;168(3):473-512. doi: 10.1111/j.1467-985X.2004.00360.x
3. Richards JB, Evans DM. Back to school to protect against coronary heart disease? BMJ 2017;358 doi: 10.1136/bmj.j3849
4. Angrist JD, Pischke J-S. Instrumental variables in action: Sometimes you get what you need. Mostly harmless econometrics: An empiricist's companion: Princeton University Press 2009.
Competing interests: No competing interests