Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health
BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c4739 (Published 13 September 2010) Cite this as: BMJ 2010;341:c4739
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Tugwell et al.[1]reference the Commission on Social Determinants of
Health's recommendation of assessment of health equity effects of public
policy decisions and attempt to provide guidance for such assessment. But
the authors' effort suffers from failure to recognize certain fundamental
measurement issues.
Virtually all health inequalities research to date has suffered from
a similar problem. In consequence, at least when it has endeavored to
appraise changes over time or otherwise appraise the size of inequalities
in different settings, such research has been of questionable value. The
problem lies in the failure to recognize the way that standard measures of
differences between outcome rates are affected by the overall prevalence
of an outcome, as I have explained in over 100 references made available
in the Measuring Health Disparities page of jpscanlan.com.,[2] and, for
example, references 3-6. For reasons inherent in the shapes of the
underlying distributions of factors related to experiencing an outcome,
the rarer the outcome the greater tends to be the relative difference in
experiencing it and the smaller tends to be the relative difference in
avoiding it. Thus, as mortality and morbidity decline, relative
differences in those outcomes tend to increase, while relative differences
in avoiding those outcomes tend to decrease; as appropriate healthcare
rates increase, relative differences in receiving such care tend to
decrease, while relative differences in failing to receive such care tend
to increase. Absolute differences between rates are also affected by the
overall prevalence of an outcome, though in a more complicated way.
Roughly, as uncommon outcomes become more common, absolute differences
between rates tend to increase; as common outcomes become even more common
absolute differences tend to decrease. Thus, whereas reliance on relative
differences may result in different interpretations as to directions of
changes over time depending on which relative difference is examined,
reliance on absolute differences may result in different interpretations
as to directions of change over time depending on the rate ranges at
issue. While not necessarily agreeing with my views on these issues in
all or most respects, others, mainly in Europe, are increasingly
recognizing the patterns by which measures between outcomes rates are
affected by the overall prevalence of an outcome and certain implications
of such patterns.[7-11]
The Commission on the Social Determinants of Health's recommendation
that policy decisions be influenced by perceptions of effects on health
inequalities dramatically increases the importance of these issues. For a
long time, while much that was said about patterns of changing
inequalities was fundamentally flawed, the discussion was largely
academic. That is, it did not very much affect particular policy choices.
Bauld et al.[10] recently observed with regard to setting and monitoring
health inequalities reduction goals that by ignoring the pattern of
relative differences described above, governments "run the risk of
guaranteeing failure, largely for conceptual and methodological reasons
rather than social welfare reasons." But even in that context, the
failure to properly interpret patterns of changes over time would not
necessarily have substantial concrete implications.
The potential for concrete implications of the failure to properly
interpret health inequalities measures increased somewhat when researchers
starting considering the effects of Pay for Performance on health
inequalities and suggestions were made that performance measurement should
be tied to effects on inequalities. The aims of such programs could be
significantly thwarted absent sound measures to evaluate whether
inequalities had increased or decreased.[11,12]
The Commission's recommendation that there be assessments of the
health equity effects of policy decisions may cause authorities and
institutions to make substantial policy choices based on their perceptions
about the ways various courses are likely to affect one or another measure
of inequality. Those choices could be seriously misguided if influenced
by mistaken interpretations of data on the effects on health equity.
But in discussing measurement issues, the authors merely note that
both relative and absolute inequalities should be presented. They do not
address the issue of which relative difference (in the favorable or the
unfavorable outcome should be examined) or show a recognition that the two
tend to change in opposite directions. The fact is that neither relative
differences (in either outcome) nor absolute differences can provide
useful information unless one distinguishes between distributionally-
driven patterns and patterns that reflect something more meaningful.
The best approach is to employ a measure that is unaffected by the
overall prevalence of an outcome such at that described in references 13
and 14. That measure has its imperfections. But it is still superior to
reliance on standard measures without consideration of the implications of
overall prevalence. In any case, societies should proceed with caution in
tying important policy decisions to interpretations of data on
inequalities without being sure that the interpretations are sound.
References:
1. Tugwell P, Petticrew M, Kristjansson E., et al. Assessing equity
in systematic reviews: realising the recommendations of the Commission on
Social Determinants of Health
2. Measuring Health Disparities page of jpscanlan.com:
http://jpscanlan.com/measuringhealthdisp.html (Accessed 28 Sept 2010)
3. Scanlan JP. Can we actually measure health disparities? Chance
2006:19(2):47-51:
http://www.jpscanlan.com/images/Can_We_Actually_Measure_Health_Dispariti...
(Accessed 28 Sept 2010)
4. Scanlan JP. Race and mortality. Society 2000;37(2):19-35:
http://www.jpscanlan.com/images/Race_and_Mortality.pdf (Accessed 28 Sept
2010)
5. The Misinterpretation of Health Inequalities in the United
Kingdom, presented at the British Society for Populations Studies
Conference 2006, Southampton, England, 18-20 Sept 2006:
http://www.jpscanlan.com/images/BSPS_2006_Complete_Paper.pdf
6. Scanlan's Rule page of jpscanlan.com:
http://jpscanlan.com/scanlansrule.html (Accessed 28 Sept 2010)
7. Carr-Hill R, Chalmers-Dixon P. The Public Health Observatory
Handbook of Health Inequalities Measurement. Oxford: SEPHO; 2005:
http://www.sepho.org.uk/extras/rch_handbook.aspx (Accessed 28 Sept 2010)
8. Houweling TAJ, Kunst AE, Huisman M, Mackenbach JP. Using relative
and absolute measures for monitoring health inequalities: experiences from
cross-national analyses on maternal and child health. International
Journal for Equity in Health 2007;6:15:
http://www.equityhealthj.com/content/6/1/15 (Accessed 28 Sept 2010)
9. Eikemo TA, Skalicka V, Avendano M. Variations in health
inequalities: are they a mathematical artifact? International Journal for
Equity in Health 2009;8:32: http://www.equityhealthj.com/content/pdf/1475-
9276-8-32.pdf (Accessed 28 Sept 2010)
10. Bauld L, Day P, Judge K. Off target: A critical review of
setting goals for reducing health inequalities in the United Kingdom. Int
J Health Serv 2008;38(3):439-454. (Accessed 28 Sept 2010)
11. Pay for Performance sub-page of Measuring Health Disparities
page of jpscanlan.com:
http://www.jpscanlan.com/measuringhealthdisp/payforperformance.html
(Accessed 28 Sept 2010)
12. Scanlan JP. Interpreting patterns of changes in absolute
differences between rates when common outcomes become even more common.
BMJ Dec. 7, 2008 (responding to Ashworth M, Medina J, Morgan M. Effect of
social deprivation on blood pressure monitoring and control in England: a
survey of data from the quality and outcomes framework. BMJ
2008;337:a2030): http://www.bmj.com/cgi/eletters/337/oct28_2/a2030
(Accessed 28 Sept 2010)
13. Solutions sub-page of Measuring Health Disparities page of
jpscanlan.com:
http://www.jpscanlan.com/measuringhealthdisp/solutions.html (Accessed 28
Sept 2010)
14. Scanlan JP. Measuring Health Inequalities by an Approach
Unaffected by the Overall Prevalence of the Outcomes at Issue, presented
at the Royal Statistical Society Conference 2009, Edinburgh, Scotland, 7-
11 Sept 2009:
http://www.jpscanlan.com/images/Scanlan_RSS_2009_Presentation.ppt
(Accessed 28 Sept 2010)
Competing interests: No competing interests
Problems with assessing equity will continue as the generative mechanisms are still missing
In response to authors' intention "to stimulate methodological
development, discussion, and practice in this field" I'd like to make few
comments which will be useful for all involved in conducting studies for
interventions on populations, both primary and systematic reviews, as well
as for all users of such studies.
1. "The Commission on Social Determinants for Health has recommended
assessment of health equity effects of public policy decisions."
Yes, it's a good wish, but the CSDH did not provide a clue how to do
this.
2. Systematic reviews are about examining the Evidence, however,
neither the CSDH nor the experts in this paper could see, let alone
explain, that Evidence for interventions on populations is relative and
depends on the distribution of the benefit at local level in dynamics.
The Problem
The CSDH makes its recommendations based on a lot of empirical
observations. However, as Connelly (2001, 2005) points out, without proper
theoretical framework based on critical realism, where the causal pathways
are "if A then always B", interventions based on empirical observations
might not "work" or can be inappropriate at local level. In confirmation
of this , Kelly (2006) notes that "different segments of the population
respond very differently to identical public health interventions" and "in
relation to health inequalities it appears from the evidence that in many
cases the causal pathways are not always so clear and the covering laws
(he obviously means the generative mechanisms) are not known at all".
The Solution
With any intervention on populations there are always winners -
people who benefit of it, and losers - people who benefit less.
Distinguishing between "relative losers", "absolute losers" and "double
losers", Panayotov (2008, 2009) has established that evidence for
interventions on populations is relative and depends on the distribution
of the benefit at local level in any specific case. He has developed a
model - Panayotov Matrix, which identifies eight combinations of this
distribution leading to very different results regarding average health
status and health inequalities. Therefore, an appraisal of any
intervention (especially when assessing equity) should start with
analysing the distribution of the benefit at local level in dynamics.
In conclusion
Panayotov Matrix is a model which explains the generative mechanisms
which create, widen, or diminish health inequalities. Based on critical
realism (if A then always B) the model provides universal explanations and
predictions. Panayotov Matrix offers a useful tool for assessment of
health equity effects of public policy decisions and helps both users and
authors of systematic reviews of interventions on populations.
References:
Connelly J., Critical realism and health promotion: effective
practice needs an effective theory, Editorial, Health Edu Research,
Vol.16, No2, 115-120, Oxford University Press 2001
Connelly J., More public health theory please - but make it adequate,
Editorial, Journal of Public Health, Vol.27, No.4, p. 315, 2005
Kelly M., et al., The development of the evidence base about the
social determinants of health, Measurement and evidence knowledge network,
CSDH, WHO, May 2006
Panayotov, J., "Public Health and Average Health Status: Do Health
Inequalities Matter?, ICARE, August 2008
available at http://icare.academia.edu/JordanPanayotov/Papers
(Accessed 10.10.2010)
Panayotov, J., "Evidence in Public Health and Health Impact
Assessment", ICARE, February 2009
available at http://icare.academia.edu/JordanPanayotov/Papers
(Accessed 10.10.2010)
Panayotov, J., Presentations at different forums 2006 - 2010
available at http://icare.academia.edu/JordanPanayotov/Talks
(Accessed 10.10.2010)
Competing interests: No competing interests