Association between day of delivery and obstetric outcomes: observational study

I wish to express my concerns about why a paper capable of causing such huge media impact has been accepted for publication without being peer-reviewed, and would be grateful if the reason for this could please be clarified.

Another aspect which I am interested to know is on what basis was this study commissioned.

I would welcome a response from the authors.

The findings and conclusions of this study are based on several premises that are open to question.

The core of the study is the complicated and complex statistical analysis of a very large data. The authors have done us all a signal service by assembling the data set. But if it is to be exploited fully then we need to go beyond just the one presentation based on an analysis that can be, and already has been, called into question.

The best way to address these criticisms would be for the authors to make their complete anonymised dataset available in the public domain for other research to re-run the analysis using different assumptions and thereby test the conclusions.

This approach of 'reproducible research' has noe become the norm in the analysis of large data sets, where in the data the computer program code for the data cleaning and modification and the function calls that perform the analysis are all published together in an externally verifiable form [Ref1]

References.

1. Peng RD. Reproducible research in computational Science. Science. 2011 Dec 2; 334(6060): 1226–1227.
doi: 10.1126/science.1213847

We were deeply saddened to read the article by Palmer et al. [1] and note that a number of people have already submitted responses that make note of the flawed methodology of comparing days of the week to a Tuesday baseline, as well as obstetric-specific concerns. We would further like to raise concerns about the methodology in terms of data characterisation, choice of outcome, choice of p-value threshold and failures in statistical rigour.

In particular:
1. It is stated that data were "categorised by day of admission (for maternal indicators) or birth (for neonatal records)". Given that the duration of labour could extend beyond day of admission, this potentially means that a Sunday admission could result in a Monday birth. Therefore a potential explanation in the low perinatal mortality on a Monday is actually due to improved care at the weekend; the complete opposite conclusion to that of the authors. Even with normal neonatal outcomes, the 90th centile of admission to 10cm dilation can be 20 hours in nulliparous women [2].

2. Use of a 3 day outcome for a mother admitted on a Sunday is confounded by effects of Monday and Tuesday obstetric care and, similarly, a 3 day outcome for a mother admitted on a Friday is confounded by weekend care. Thus the raised weekend mortality could in fact be an issue with care of obstetric patients on a Thursday and Friday.

3. As noted by other responders to this paper, defining the weekend as starting at midnight on a Friday and finishing at midnight on a Sunday is wholly unrealistic and likely to have an impact on the estimates of weekend effect.

4. We do not feel that the authors have adequately declared their reasons for Tuesday being the reference point. Choosing a midweek point, e.g. Wednesday, particularly if direct comparison to the equally flawed Freemantale [3] paper is to be made, seems more intuitive. Given the arbitrary nature of this point and the potential for this to substantially change the representation of data, it is surprising that a sensitivity analysis using alternative points in the week have not been shown. This is absolutely critical as if one reference weekday provides a different answer to another one, then clearly this is a poor choice of statistical method or an intentional attempt to misrepresent. We prefer to give the authors benefit of the doubt that it is the former.

5. The authors define perinatal mortality as "rate of still births plus in-hospital deaths within 7 days". It seems disingenuous to compare day-by-day effects using a 7 day outcome without any discussion of the potential distribution of deaths within that 7 day window, particularly as this represents a composite outcome that includes stillbirths. Furthermore, these data were obtained from discharge documentation, yet it is unclear whether day of week has impact on the recording of it.

6. Perhaps more problematic is puerperal sepsis defined as all cases that occur within 42 days. Given that day-by-day comparisons are being made, how is it possible that the authors are able to remove confounding effects of care during the week (or even the next week) with a 42 day outcome? Puerperal sepsis can have late-onset manifestations (after 1 week) [4] and a failure of recognition of sepsis during the week could, for example, be incorrectly attributed to the weekend. This outcome is also confounded by pre-admission risk factors, such as the presence of group B streptococcal infection bacteriuria during pregnancy, and does not take account of the potential for non-maternal sources of sepsis. It is quite difficult to justify attributing all or most of these outcomes to the day of admission, which any discussion of a “weekend effect” relies upon.

7. A study of this size and of potential impact in terms of policy and of inducing panic in patients and their families ought to be held to a high standard, including appropriate statistical methodology. The authors have chosen an alpha level of 5%, yet have failed to make any adjustment for multiple comparisons: a Bonferroni or Sidak correction using estimates calculated from the reported data with 7 tests results in much wider confidence intervals with only perinatal mortality and neonatal injury remaining significant (data not shown; see next point).

8. We would further like to draw the authors attention to the work of Johnson et al. that states “in terms of classical hypothesis tests, these evidence standards mandate the conduct of tests at the 0.005 or 0.001 level” [5]. A choice of an individual-test alpha of 0.001 or a family-wise alpha of 0.005 results in none of the investigated odds ratio being significantly different from 1 (see attached table). Family-wise alpha levels of 0.05 might be appropriate for small trials, but not a large scale study that has the potential to influence policy. It is surprising that we still seem to universally accept an alpha of 0.05 since its arbitrary choice in 1925 [6].

9. Regardless of choice of alpha, a study of this size should be sufficiently powered to detect very small changes and so we are left with the issue of whether statistically significant differences are in any way clinically meaningful.

10. Goodness-of-fit for logistic regression is not as straightforward as it is for general linear models, yet it is still possible to make assessments of model fit that extend beyond simple sensitivity analyses, such as k-fold cross-validation or Hosmer-Lemeshow [7].

11. The authors do not provide detailed information on sensitivity analyses

12. There is a distinct absence of model fitting methodology, including whether different adjustments were made for each model fitted.

We would welcome response from the authors and also would like to propose that the BMJ formulates new statistical standards for publishing, even if it simply requires authors to make consideration of multiple comparison testing and whether or not 0.05 is an appropriate cut-off for a p-value when insisting on using frequentist statistical methods. We also support the view that, unless there are individual patient confidentiality issues, the underlying data in studies such as these should be published alongside the work in the interest of repeatability, peer review and meta-analysis.

[1] Palmer et al. (2015). Association between day of delivery and obstetric outcomes: observational study. The British Medical Journal 351:h5774
[2] Zhang et al. (2010). Contemporary Patterns of Spontaneous Labor With Normal Neonatal Outcomes. Obstetrics and Gynecology 116(6):1281–1287.
[3] Freemantle et al. (2015). Increased mortality associated with weekend hospital admission: a case for expanded seven day services? The British Medical Journal 315:h4569
[4] Parks et al. (2012). Invasive streptococcal disease: a review for clinicians. British Medical Bulletin 115(1):77-89
[5] Johnson, V.E. (2013). Revised standards for statistical evidence. Proceedings of the National Academy of Sciences USA 110(48):19313-19317
[6] Fisher et al. (1925). Statistical methods for research workers. Edinburgh: Oliver & Boyd. ISBN 0-05-002170-2
[7] Stoltzfus, J. C. (2011), Logistic Regression: A Brief Primer. Academic Emergency Medicine 18: 1099–1104.

We Have read with Interest this article and the research question is valid in developing Countries where the cultural and social connotations at times demand a particular day of delivery specially in large parts of rural india.The Present study has alot of strong Points Numbers, Valid records, Strong Analysis and the only limiting factor is the examination and further analysis of transcripts of clinical notes which could have added a perspective as to whether there was a clinical correlation to Outcomes in terms of Pregnacy history and progress of labour. It is however better tohave a fixed cohort from different locations to track the natural history to outcomes which is slightly difficult. What was the Odds estimation for different days of week and Outcomes.We congratulate the author and would assure we will attempt a community based enquiry within Ethical and logistic considerations.