Beware of the Texas sharp shooter in calculating rate ratios of progression
Glasziou et al’s method of calculating rate ratios of progression
(stable unchanging condition before vs. change shortly after the
intervention) is appealing, but in applying it we need to be wary of a
‘Texas sharp shooter’ effect. This effect is usually associated in
epidemiology with the problem of interpreting apparent ‘clusters’ of
disease in space, where the geographical unit of analysis - a town, a
borough, a few streets, the top half of one street etc - may have been
chosen post hoc in such a way as to maximise the apparent density of cases
(the sharp shooter metaphor comes from a joke about a Texan firing bullets
into the wall of a barn and then drawing the targets around the bullet
holes as a demonstration of his shooting prowess).
There is the potential for an analogous problem when trying to
calculate rate ratios in the manner described in this article, although
here the sharp shooting is in time not in space. To take the authors’
example of the mother’s kiss, the time period used is 10 seconds which
gives a rate ratio of progression of 1440. Perhaps, however, the bead
dislodged after only 8 seconds – if a somewhat sharper shooter had used 8
seconds as a time frame, this would have given a rate ratio of 1440/0.8 =
1800. Alternatively, if the bead had taken 15 seconds to dislodge, the
GP, nurse and mother might still reasonably have felt that they should
take the credit for the this happy outcome. The authors’ might
correspondingly have stretched their time interval to 15 seconds
(associated rate ratio 960) or included two 10 second intervals, just as
they included 3 (rather than one) month in their portwine stain example
(associated rate ratio 720). The point is that one needs to make an a
priori decision about the post-intervention time frame you will use –
presumably based on the maximum length of time after the event during
which, if improvement occurs, you are prepared to attribute it to your
intervention.
Rapid Response:
Beware of the Texas sharp shooter in calculating rate ratios of progression
Glasziou et al’s method of calculating rate ratios of progression
(stable unchanging condition before vs. change shortly after the
intervention) is appealing, but in applying it we need to be wary of a
‘Texas sharp shooter’ effect. This effect is usually associated in
epidemiology with the problem of interpreting apparent ‘clusters’ of
disease in space, where the geographical unit of analysis - a town, a
borough, a few streets, the top half of one street etc - may have been
chosen post hoc in such a way as to maximise the apparent density of cases
(the sharp shooter metaphor comes from a joke about a Texan firing bullets
into the wall of a barn and then drawing the targets around the bullet
holes as a demonstration of his shooting prowess).
There is the potential for an analogous problem when trying to
calculate rate ratios in the manner described in this article, although
here the sharp shooting is in time not in space. To take the authors’
example of the mother’s kiss, the time period used is 10 seconds which
gives a rate ratio of progression of 1440. Perhaps, however, the bead
dislodged after only 8 seconds – if a somewhat sharper shooter had used 8
seconds as a time frame, this would have given a rate ratio of 1440/0.8 =
1800. Alternatively, if the bead had taken 15 seconds to dislodge, the
GP, nurse and mother might still reasonably have felt that they should
take the credit for the this happy outcome. The authors’ might
correspondingly have stretched their time interval to 15 seconds
(associated rate ratio 960) or included two 10 second intervals, just as
they included 3 (rather than one) month in their portwine stain example
(associated rate ratio 720). The point is that one needs to make an a
priori decision about the post-intervention time frame you will use –
presumably based on the maximum length of time after the event during
which, if improvement occurs, you are prepared to attribute it to your
intervention.
Competing interests:
None declared
Competing interests: No competing interests