When professional opinion is not enoughBMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39161.403218.AD (Published 19 April 2007) Cite this as: BMJ 2007;334:831
- Tom Treasure, professor of cardiothoracic surgery1,
- Martin Utley, reader in operational research2,
- Ian Hunt, specialist registrar in thoracic surgery1
- 1Guy's Hospital, London SE1 9RT
- 2Clinical Operational Research Unit, University College London, London
- Correspondence to: T Treasure
- Accepted 10 February 2007
New treatments are usually thoroughly evaluated before they enter clinical practice, but much of what doctors do is based on experience rather than evidence. However, just because a practice is widely accepted within the profession does not guarantee that it is effective; there are many historical examples, such as the practice of copious and repeated blood letting, which persisted from antiquity well into the mid-19th century.1 In recognition of this, last year the UK National Institute for Health and Clinical Excellence (NICE) launched an initiative to identify interventions delivered by the NHS that do not benefit patients.2
Of course, to reject the experience and insight of generations of clinicians as “low grade evidence” and subject everything we do to randomised controlled trials would be hugely wasteful and impractical. Many existing practices should be retained and new ideas introduced because the benefits are large and evident without further study.
The question that arises is what standard of evidence we are willing to accept to persist with a particular practice. Anyone proposing a trial or a cost effectiveness study of treatment of broken legs or cataracts would quite rightly have their calculator and clipboard taken from them. However, for conditions such as cancer, the benefit is less clear cut: practice is justified in the belief that the dying die more slowly than would have been the case and that a handful may live to die of something else. In some areas of our practice we have little evidence on which to base this belief. We discuss one example, resection of lung metastases in patients who have had surgery to remove a primary colorectal cancer, where benefit is questionable and NICE guidelines are not based on good evidence.
Lung metastases from colorectal tumours
Patients with lung cancer who have metastases are not usually offered resection of the primary tumour because it will not produce a cure.3 Resection of the primary tumour alone rarely achieves a net improvement in symptoms; it adds morbidity from the operation itself and the loss of pulmonary function.
Management of the primary tumour in colorectal cancer is fundamentally different. Even if survival is already determined by metastatic spread, local control of the colorectal tumour may be justified to pre-empt potential consequences such as obstruction, bleeding, perforation, tenesmus, fistula, and pelvic pain. Metastases therefore have become a potential surgical target in the management of colorectal cancer in a way that they are not in the management of lung cancer.
Patients with pulmonary metastases from colorectal cancer are increasingly referred to thoracic surgeons for consideration of excision of the secondary cancer.4 This follows recommendations included in the 2004 NICE guidelines that patients with metastases confined to limited areas of the liver or lung should be referred to the appropriate specialised multidisciplinary team for an opinion.5 The expected benefits are set out as follows: “Surgery for patients with metastases confined to the liver or lung can be curative when carried out by specialists with experience of this type of work. Although such resection is only appropriate for a minority of patients, it can improve five year survival from close to zero to over 30%.” The reference cited in support of this statement is a 1994 case series analysis about untreated liver metastases.6 Regardless of whether this paper is sufficient evidence for the removal of liver metastases, it contains no data on patients with lung metastases, resected or not.
Knitting with sand
A recent systematic review of evidence for removal of liver metastases found no randomised controlled trials.7 The authors found 529 studies but 470 were excluded from the analysis on the basis of specified criteria such as small numbers (<100) short follow-up (<year), or lack of survival data. Of the 59 retained, only two studies included all patients presenting with liver metastases and only three reported on all patients who had liver surgery—that is, the denominator from which the case series was drawn was specified for less than 1% of studies. The data available relate entirely to survival, but no quantitative meta-analysis was possible. “No data [were] found on comorbidity, quality of life, or the cost effectiveness of liver resection.”7 This is a sorry state of affairs. The work in this publicly funded systematic review was enormous, but the quality of surgical reporting on which it relies is inadequate for the purpose.8 Further case series will not answer the clinical question whether to resect metastases, and it is time to stop producing clinical reports that do not add to existing knowledge.9
For liver resection after control of primary colorectal cancer, the qualitative systematic review concludes: “Around one third of patients will survive for 5 years or more. These outcomes in highly selected patients exceed those normally associated with colorectal cancer.”7 Whether metastasectomy contributes to this difference in survival, rather than it being solely due to selection of biologically less aggressive cancers is not addressed.
One argument used for hepatic metastasectomy is that the liver is a first filter. But once the disease has reached the lungs it has by definition escaped the first filter. Extrapolation from hepatic resection to lung resection is not justified by the evidence, the rationale that underlies management, or putative mechanisms. Thus thoracic surgeons must come to a collective view.
We are in the process of conducting a systematic review of this literature. Without prejudging the findings, we can say that the evidence for resecting pulmonary metastases from colorectal cancer is not strong and that the practice is widespread. We have found heterogeneous case series but no randomised controlled trials. The working group of the European Society of Thoracic Surgery (in process at present), which is an international panel of experts, has not identified any trials in progress. Evidence based analyses of different approaches to pulmonary metastasectomy concur: there are no randomised controlled trials.10 11
The argument that pulmonary metastasectomy is beneficial is spuriously based on identifying factors associated with longer survival among patients who had metastasectomy. The factors identified are the number of metastases, disease-free interval, and the level of tumour markers. The patients are sorted according to how many of these factors they have, and the Kaplan-Meier plots move stepwise up the page.12 13 The conclusion that surgery confers benefit is concerning since each factor is an index of the biology of the cancer. The fallback defence for resection is that as a group resected patients do better than would be expected based on the survival of the patient population as a whole—but they are all clinically selected. Also, judgments on lung resection often require data on more than survival since pulmonary resection is inescapably associated with substantial postoperative morbidity and permanent loss of respiratory function.
A way forward
Unsurprisingly, we conclude that we may need a trial. The systematic review for hepatic metastasectomy included the statement “Randomised trials comparing surgical resection with non-surgical treatments are not now possible.”7 What none? Ever? Is this view based simply on reverence for existing practice? Randomised controlled trials are indeed difficult in surgery but not impossible.
In this case the difficulty of discerning any benefit of lung resection is compounded because surgery is often combined with other therapeutic strategies. The surgery itself is an inescapable source of risk, perioperative morbidity, and permanent reduction in lung function. So how do we know if surgery is responsible for any perceived improvements? We outline below a trial design that we think would be feasible because it relies on this clinical uncertainty. Consider the following two extreme ends of the clinical spectrum:
Scenario 1—The patient develops a single metastasis four years after resection of the primary colorectal tumour when intervening investigations had all been clear. The patient is fit and the metastasis is easy to remove from the periphery of the lung with minimal access.
Scenario 2—A radiograph taken on first follow-up a month after colorectal resection, which was poorly tolerated, shows a total of 20 metastases distributed through every pulmonary lobe.
All of the thoracic teams we know would offer to resect the single metastasis in scenario 1 but not the multiple metastases in scenario 2. Every team may therefore have a zone of uncertainty if the decision is to offer surgery under some circumstances and to advise against surgery under other circumstances. Teams could prospectively identify scenarios where they were uncertain about the benefit of resection. They would recruit such patients into a multicentre trial with outcome measures including survival and measures of quality of life. When there is uncertainty unbiased allocation is justified and is as rational as either alternative. The informed public understands that.14
Resection of lung metastases is widespread in patients with colorectal cancer
The evidence for benefit of resection is poor with no randomised controlled trials
Resection is unlikely to eradicate the cancer, other than in a lucky minority
Removal of the metastases causes appreciable morbidity
The practice should be formally evaluated
Contributors and sources: TT is a practising thoracic surgeon and has an interest in evaluation of interventions. This article arose because of a marked increase in the referral from the colorectal to the lung multidisciplinary team. MU is a mathematician who often sits in on lung cancer multidisciplinary meetings and has an interest in the role of evidence in clinical decision making. IH has worked on NICE guideline development and is the lead worker on a systematic review of colorectal lung metastasectomy. The authors have worked together researching the existing evidence on this question, discussing the issues involved, and writing the manuscript. TT is guarantor.
Competing interests: None declared.