Rapid Responses to:

EDITORIALS: Timothy David Noakes and Catherine E Draper Preventing spinal cord injuries in rugby union BMJ 2007; 334: 1122-1123 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

A Statistical Caution

Roy J. Shephard   (3 July 2007)

A Statistical Caution 3 July 2007
Roy J. Shephard, Professor Emeritus of Applied Physiology, University of Toronto Brackendale, BC V0N 1H0 Send response to journal: Re: A Statistical Caution	Has the New Zealand Rugby Union found the magic bullet that will curb the problem of spinal injuries in their sport? The results of Quarrie and colleagues (1) certainly merit further investigation, but I would approach the interpretation of their data a little more cautiously than Professor Noakes appears to do, for several reasons. As Dr. Noakes acknowledges, there is firstly the problem that the data are derived from a simple "before and after" study, with no control group for comparison. Many things may have happened in parallel with introduction of the RugbySmart programme, such as changes in the rules of play and their enforcement, and a decrease in the number of scrums per game (2). There is also the issue that the data lack a denominator; this is a common problem in injury studies. The only figure available for analysis is the total number of disabling spinal injuries sustained during a quinquennium, without reference to the number of players or the hours of practice and competition. However, the most important limitation of the study is its recourse to data mining of what seem to be some rather doubtful statistics. The hypothesis tested by Quarrie et al. (1) was that the introduction of the RugbySmart programme would reduce the number of disabling spinal injuries seen over a quinquennium. The figure they have used for their comparison is an average of 18.9 disabling spinal injuries per quinquennium, apparently for the period 1976-2000; however, the authors' published graph and thir stated total of 69 such injuries during 25 years suggest that the correct average is 13.8. During the period 2001-2005,they observed a total of eight disabling spinal injuries. Given the small total number of events, this does not differ significantly even from the erroneously high number of 18.9, and it is certainly not significantly less than the true comparative figure of 13.8 injuries per quinquennium. Thus, the authors' initial hypothesis is disproven. One of the earliest lessons in Canadian statistics courses is that in such a situation, it is wrong to proceed with post-hoc data mining. However, Quarrie et al. go on to break down their injuries into sub-groups for scrums, tackles, rucks and mauls, where group sizes are even smaller. In this manner, they claim to show a significant effect of their programme upon the incidence of spinal injuries during scrums. Plainly, there remains a need for a large, well-controlled experiment, addressing a clear hypothesis that an educational programme can reduce spinal injuries either in all play or in scrums. References 1. Quarrie KL, Gianotti SM, Hopkins WG, Hume PA. Effect of nationwide injury prevention programme on serious spinal injuries in New Zealand rugby union: ecological study. BMJ 2007; 334: 1150-1153. 2. Quarrie KL, Hopkins WG. Changes in player characteristics and match activities in Bledisloe Cup rugby union from 1972 to 2005. J Sports Sci 2007; 25: 895-903. Competing interests: None declared