Sample review (written 16.6.01) of a paper by Andersen et al that was
published in N Engl J Med 1984;310:352-6

General comment on this retrospective review: There are many defects in the presentations of this paper when judged by standards current in 2001, so readers may be surprised that NEJM accepted it. However, it is unfair to judge a 1984 paper by 2001 standards, because many of the defects noted below were not generally recognised 17 years ago. Also, this paper is a follow-up of a preliminary report in Int J Obes 1981 in which some essential information was given, but not repeated here. It may be that the editors of NEJM did not permit republication of these data, so the fault did not lie with the authors.

Originality and importance: For all its defects this paper was original and important to clinicians at the time of publication. It was a first attempt to conduct a large randomised controlled trial, with a long follow-up, of two methods for treating severe obesity which were widely used at the time. The attempt failed, because the task was impossible (see below), but if I had been asked in 1984, I would have advised the editors to publish the data, but not in this form.

1. Study design

The original intention was that severely obese patients presenting to the specialist obesity unit at Hvidovre Hospital, nr Copenhagen, should be recruited to an RCT of gastroplasty vs. a very-low-calorie diet (VLCD), and followed for 2 years. This aim was laudable, but impossible to implement in practice. In a placebo-controlled drug trial a consenting patient is given a capsule to take which may or may not contain the active drug. The patient may or may not take the capsule, but (unless the side effects are obvious) compliance is likely to be similar in both arms of the trial.

Gastroplasty and VLCD are unsuitable for this type of design, since the side effects of both are very obvious, and therefore it is impossible to blind either patients or researchers. Also, patients in the VLCD arm can stop at any moment they choose, but those with gastroplasty need a surgeon to reverse it. The correct design (which was adopted in later trials) is to compare the progress of patients who opt for gastroplasty with that of those who opt for VLCD.

Between September 1979 and May 1981 there were 128 patients admitted to the unit, of whom only 78 met the selection criteria (which are not clearly stated in the NEJM paper). Of these 18 refused randomisation, and the remaining 60 were divided between the two arms "following the instructions of a third party" (neither the party involved, nor the instructions, are described, so the reader would be unable to replicate the study).

The design involved clinic attendance weekly for 3 months (12 visits) then every 2 weeks to 2 years (another 42 visits), which is too much to expect even from the most compliant Danes. We are not told what proportion of these scheduled visits were kept. Those on VLCD "were allowed to take diethylpropion… only if hunger prevented compliance with the diet." The diet in the VLCD group was stopped "when no weight had been lost for two months" – we are not told if this was part of the original design, or merely an acknowledgement that any patient who does not lose weight in 2 months has clearly already stopped adhering to the VLCD. No mention is made of what happened if gastroplasty patients ceased to lose weight.

2. Dropout rate. This is stated to be 4%, but that cannot be true. Figure 3 shows weight change after 24 months in 14/27 gastrostomy patients and 12/30 VLCD patients, which implies a drop-out rate >50%.

3. Statistical analysis. There is no statistician among the authors (TA and FQ are physicians, OB and KS are surgeons, all personally known to me). the conclusion that there is "no significant difference" between the weight losses of the two groups is based on the rather crude analysis of the two-tailed Mann-Whitney test for non-paired data, evidently taking the data at 3, 6, 12, 18 and 24 months as independent sets, which they certainly are not. It is not explained how allowance was made for different dropout rates between the two groups. Better statistical advice should have been taken and acknowledged.

4. Conflict of interest. No statement is made about this. The text on p355, col 1, reads rather like an advertisement for "our" VLCD and it is not clear if any of the authors had a commercial interest in the proprietary VLCD used in this study, or in Oluf Moerk Bio-Chemie who are thanked for supplying it. The statements on p355 that "food rich in protein must be selected" and that artificial nutrient based VLCD are "more effective on an outpatient basis" are not supported by any evidence in this paper (or anywhere else, to my knowledge).

5. Ethical committee approval. This is not mentioned, and would not be given if a similar protocol was submitted today. However, this research (which included needle biopsy of liver as a screening procedure) was probably considered acceptable in 1984.

6. Results. These are not clearly reported, partly owing to the confusing study design. The entry criteria included >60% overweight by local standards (published in Danish) but weight change is shown in kg. It is difficult to calculate how close to "desirable" weight the more successful patients came. We are never shown the gender, age, weight and height of those who entered the two treatment groups, and those completed the 24 month study at baseline and again at completion. Contrary to the statement on p355, col 2, it is not necessary to separate the data for weight loss and regain, because a plot of net weight loss at each time point suffices. The term "immediate success rate" is used, but not clearly defined. On p355, col 2, some post-hoc analysis is introduced to find subgroups in whom gastroplasty gave better results than diet, despite the general conclusion that the difference in weight loss between groups was "NS". There may have been a difference in height between the two groups (never mentioned in the text) because although the initial median weight of the gastroplasty group was greater than VLCD (120 vs 115kg, see abstract), the median loss of excess weight was also greater (0.57 vs 0.46 – see p354, col1). This may be due to the large drop-out between 18-24 months (see fig 3).

Could the paper be salvaged by re-writing?

The paper would not be acceptable today, because the ethical situation is unclear, and the results have been overtake by subsequent better studies. However, if these obstacles are ignored, as an exercise in reviewing, the following changes might be suggested to the authors:

A. Title. This is rather misleading: the method of randomisation is not clearly explained, and one of the treatments was not "diet alone", but a prescribed very-low-calorie diet with diethylproprion if required. A more informative and accurate title would be "Weight loss in severely obsess patients two years after either gastroplasty or a very-low-calorie diet and diethylproprion".

B. Abstract. This is unstructured and unhelpful. It should clearly explain the Objectives, Design, Setting, recruitment of Subjects, Interventions, Outcome measures, Results and Conclusions. The functions of the Authors, the source of Sponsorship, and any Conflict of interest should be specified. A homily on type I and II errors is inappropriate in an abstract, even if it is exciting news to the authors.

C Methods. These are not well described. International standards (eg. BMI) would be used to assess overweight, not local standards not easily accessible to the reader. The authors state that the diets in the two groups were isoenergetic, but this is clearly untrue, as the weight-change data show. No-one knows to what extent the actual diet of an outpatient corresponds to the prescribed diet. The number of recall visits is unrealistic, but cannot be changed retrospectively, but at least DNA rates could be reported. Ideally, a clinically significant weight loss should be defined, and a power calculation made to show the number of cases required to detect this difference. Statistical analysis of groups from whom there is dropout should be carefully explained. Ethical committee approval for the study should be reported.

D. Results. Measures of "success rates" should be defined, and the same units used to calculate overweight and weight loss (ie. kg). It is not necessary to report weight gain and loss separately. Post-hoc analysis of subgroups should be avoided. Tabulation of data suggested in para 6 above would be helpful to the reader.

E. Discussion. This should set out the extent to which the results of the study achieve the stated objectives, and how these results fit in with other research in the area. It is not the place for unsupported claims for the treatments used. It should also deal with the limitations of the study, one of which is that mere change in body weight is not a valid measure of success in treating obesity.