Re: Increased mortality associated with weekend hospital admission: a case for expanded seven day services?
Dr Goldacre appears to believe that sharing the 'analytic code' would in some way improve science. Although clearly not an intended consequence, this suggestion may unfortunately be an example of bad science for the following reasons:
First, replication is key to the scientific activity; when analysing our randomised trials for example, our Standard Operating Procedures (SOPs) direct us to replicate the findings completely. This specifically includes the programming steps, to provide independent verification of our findings. We asked a senior colleague to look over our two published papers[1,2] (our paper in the BMJ simply being an encore of the one in JRSM) of this observational study. His report stated that our analyses were described in sufficient detail in the published literature for others to replicate them. Indeed we are aware from our peer review activities that others have successfully replicated our methods in whole or in part for weekend analyses. Other researchers have also made different (and quite reasonable) decisions on how to handle some issues (for example, deprivation). Any differences in the findings between our findings and those of others will be addressed in the public domain. At a professional level we expect to have correspondence with others working in the area and have indeed received and entered into such communication.
The second challenge is that the 'code' is actually a rather extensive catalogue of SQL and SAS programs which, when submitted, create the data sets and then run the analyses. This would cover many pages of information. Given that the code is partly machine dependent, it is unclear how it could be of use to a third party who may be conducting analysis in other environments such as Stata, R, Genstat, etc. If we limit ourselves to the code for the survival analysis itself then the request becomes anodyne; the SAS manual describes very thoroughly how to specify survival analyses with time dependencies and a suitably qualified and experienced statistician would not rely upon our code.[3] It may be worth noting that when we submitted the paper for publication we described it as a 'complex survivorship analysis' but the BMJ statistical referee correctly pointed out it is not really very complex at all (the complexity coming from the application in the large data set and the degree of processing required).
1. Freemantle N, Ray D, Mcnulty D, Rosser D, Bennett S, Keogh BE, Pagano D. Increased mortality associated with week-end hospital admission: a case for expanded seven-day services? BMJ, 2015; 351: h4596
2. Freemantle N, Richardson M, Wood J, Ray D, Khosla S, Shahian D, Roche WR, Stephens I, Keogh B, Pagano D. Weekend hospitalization and additional risk of death: An analysis of inpatient data. Journal of the Royal Society of Medicine, 2012; 105: 74-84.
3. https://support.sas.com/documentation/cdl/en/statug/63347/HTML/default/v...
Competing interests:
No competing interests
07 February 2016
Nick Freemantle
Professor of Clinical Epidemiology & Biostatistics
University College London
Department of Primary Care and Population Health, UCL Medical School (Royal Free Campus), Pond Street, London NW3 2PF, UK
Rapid Response:
Re: Increased mortality associated with weekend hospital admission: a case for expanded seven day services?
Dr Goldacre appears to believe that sharing the 'analytic code' would in some way improve science. Although clearly not an intended consequence, this suggestion may unfortunately be an example of bad science for the following reasons:
First, replication is key to the scientific activity; when analysing our randomised trials for example, our Standard Operating Procedures (SOPs) direct us to replicate the findings completely. This specifically includes the programming steps, to provide independent verification of our findings. We asked a senior colleague to look over our two published papers[1,2] (our paper in the BMJ simply being an encore of the one in JRSM) of this observational study. His report stated that our analyses were described in sufficient detail in the published literature for others to replicate them. Indeed we are aware from our peer review activities that others have successfully replicated our methods in whole or in part for weekend analyses. Other researchers have also made different (and quite reasonable) decisions on how to handle some issues (for example, deprivation). Any differences in the findings between our findings and those of others will be addressed in the public domain. At a professional level we expect to have correspondence with others working in the area and have indeed received and entered into such communication.
The second challenge is that the 'code' is actually a rather extensive catalogue of SQL and SAS programs which, when submitted, create the data sets and then run the analyses. This would cover many pages of information. Given that the code is partly machine dependent, it is unclear how it could be of use to a third party who may be conducting analysis in other environments such as Stata, R, Genstat, etc. If we limit ourselves to the code for the survival analysis itself then the request becomes anodyne; the SAS manual describes very thoroughly how to specify survival analyses with time dependencies and a suitably qualified and experienced statistician would not rely upon our code.[3] It may be worth noting that when we submitted the paper for publication we described it as a 'complex survivorship analysis' but the BMJ statistical referee correctly pointed out it is not really very complex at all (the complexity coming from the application in the large data set and the degree of processing required).
1. Freemantle N, Ray D, Mcnulty D, Rosser D, Bennett S, Keogh BE, Pagano D. Increased mortality associated with week-end hospital admission: a case for expanded seven-day services? BMJ, 2015; 351: h4596
2. Freemantle N, Richardson M, Wood J, Ray D, Khosla S, Shahian D, Roche WR, Stephens I, Keogh B, Pagano D. Weekend hospitalization and additional risk of death: An analysis of inpatient data. Journal of the Royal Society of Medicine, 2012; 105: 74-84.
3. https://support.sas.com/documentation/cdl/en/statug/63347/HTML/default/v...
Competing interests: No competing interests