Statistical Report

I have several important reservations about this trial, in relation to all aspects: design, analysis, presentation, and interpretation.

Methods

1. The inclusion criteria for participants in the trial are not clearly stated.
2. It is noted that 11 patients refused surgical treatment and 3(? – unclear on my copy) refused nonsurgical treatment. I suspect that this means that they were not willing to be randomised. They could not ‘drop out’ of the trial before they were entered into it. Likewise the four who dropped out before random assignment. These comments assume that the researchers sought informed consent from the participants; this is not mentioned at present but should be.
3. How was randomisation done? Specifically, how was the allocation sequence generated (‘by a third party’), and what mechanism was used to implement it (such as sealed envelopes). In what way was allocation made equal in the two groups?
4. Three patients could not have the operation as specified. Were they included or excluded from the analysis. The former (intention to treat analysis) is preferred.
5. Baseline characteristics are given on briefly and for the groups combined. Details of the two groups should be presented in Results, with information given separately for each treatment group.
6. Why was the operational technique changed after the first three patients? Does it matter? (If not, why change it?)
7. It is not clear what the duration of the diets were (was it the same in the two groups?), nor what frequency of follow up was intended
8. Which time point after randomisation was intended to be the focus of the analysis? What were the pre-specified primary outcome measures?
9. It is commendably honest to say that the data collection was performed without blinding. But this is a sure weakness of the study. Even in a nonblind trial it is desirable for outcomes to be assessed blind if at all possible.
1. The sample size calculation is not fully clear.
2. The authors replaced missing data by interpolated values when only one value was missing, which apparently was in 29 of 393 evaluations (p353). This is not the best way to handle missing data – for example, it leads to an underestimate of uncertainty as the true values would be more variable than the interpolated ones.
3. However, of much greater concern is the handling of data for patients with more than one weight missing. It is clear from the figures that there was considerable attrition. Such losses to follow up will not be random and thus may lead to bias.
4. There is no mention of Ethics Committee approval for the trial.

Results

5. It seems that the main focus of the analysis is the maxim maximum weight loss. This measure will lead to an overoptimistic assessment of the effects of both interventions. It is also hard to interpret as it occurs at different times for different patients and is not sustained (later weight losses are by definition less good).
6. In the text there is a reference to the median of the maximum weight loss. It may be that the figures also show medians but this is not stated. However, the approach is not consistent and in the same paragraph there are references to both medians and mean weight loss.
7. There are repeated mentions of the mean weight loss difference being less than 9.5 kg – this seems to be based on a confidence interval but this is not clear. Both limits of confidence intervals should be shown, not just one. (And methods for constructing confidence intervals should be detailed.)
8. Fig 1 is misleading as it excludes individuals after they have lost the maximum they will lose. Why are three patients missing? Numbers of patients at each time point are not shown.
9. None of the figures indicates whether mean or median weight loss is shown. In Figs 1 and 2 the bars represent interquartile ranges – is the same true for Fig 3?
10. Confidence intervals should be given for the difference between groups. It is wrong to present separate confidence intervals for the changes within each group.
11. A serious problem with the study is the attrition. This is seen most clearly in fig 3, from which it seems that only 26 patients were still followed at 2 years.
12. More information is needed on when and why participants were lost to follow up (by group).
13. Fig 3 seems to me the only figure that displays useful information. But, as noted, the loss to follow up complicates interpretation.
14. There should be some discussion of compliance to the low calorie diets. What attempts were made to ascertain whether patients did indeed comply? It could be interesting to compare results for compliers and non-compliers in each group.
15. The interpretation is further complicated by the multiple significance tests at five or six time points (not consistent across the figures). Any significant results need to be assessed in the context of several tests.
16. The only significant P values quoted have P<0.05. Does this imply that all were >0.01? Ambiguity here and elsewhere can be resolved by giving actual P values; do not use ‘NS’
17. Finally, a better method of analysis is analysis of covariance, in which post treatment weight is regressed on initial weight and treatment group. This method is preferable to change from baseline.

Discussion

18. The nonsignificant baseline difference is irrelevant in a randomised trial.
19. I do not see why the two groups received different diets. The trial compared surgery and diet A with diet B. Any observed differences could be due either to the surgery or the diet or both. Such a ‘confounded’ trial is not amenable to a clear interpretation even without the other problems outlined above.

Abstract

20. The abstract states that the trial was carried out with 60 participants followed for 2 years. In fact while 60 were randomised figure 3 indicates that only 26 were followed for 2 years.
21. The conclusion is more positive that the text (Discussion) suggests.

Minor points

22. The scale breaks in the figures are unnecessary