Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Whats the Point?
- girish chawla (7 January 2005)
-
Randomised trials in surgery are valueless?
- Richard G Fiddian-Green (8 January 2005)
-
Expertise or performance?
- Gautham K Suresh (8 January 2005)
-
Shortfalls of expertise based design
- Eric Lim (8 January 2005)
-
More questions than answers
- Andrew Renaut, MS(Lond) FRCS(Gen) FRACS FDSRCS (8 January 2005)
-
Very interesting
- Sanjay Dalmia (11 January 2005)
-
surgical expertise is not a status quo
- javaid ahmad BUTT (12 January 2005)
-
Surgical research shares many similarities with psychotherapy research
- Simon Hatcher (19 January 2005)
-
The varying effectiveness of surgeons
- Simon G Thompson, Katherine J Lee (10 February 2005)
-
Expertise based randomized trial design - more perspectives and questions
- Swaroop S. Vedula, Nagarjun Rayapudi (26 March 2005)
|
|
|||
|
girish chawla, sho Worcester royal hospital WR5 1EP
Send response to journal:
|
Dear sir, Being a surgical SHO, I am a very junior person to comment but i sometimes personally feel that surgery is an art and as every artist has its own creativity, so does every surgeon has his own style of doing things, so realistically speaking it is not possible to compare or randomise. Moreover in Surgery when you find that a procedure is having good outcome, you automatically change your practise accordingly, so if someone is asked to do a procedure say by two methods A & B randomly and he finds that he is more comfortable with say for eg. Procedure A , he would automatically try doing more of those and with development of expertise will come to conclusion that procedure A is better. On the other hand if there is another surgeon who liked procedure B , will say that procedure B is better, will develop it and if you tell him that research says that procrdure A is better, he will not be keen to change is practise. Therefore the point i am trying to say is that, if a research will not change practise, what is the point unless there are gross differences in outcome. Thanking you,
Competing interests: None declared |
|||
|
|
|||
|
Richard G Fiddian-Green, FRCS, FACS c/o Sanders, Temple Gdns, Moor Park.
Send response to journal:
|
It is refreshing to see surgical expertise being considered in the design of randomised controlled trials (1). "Strategies to achieve this goal will [however]include selecting qualified surgeons who have attained a specified level of post training experience". What does that mean? Certification, recertification, years in practice, seniority, number of cases done, or established performance? There can be huge intrainstitutional and interinstitutional variations in outcome (2). What is more outcomes from operations performed for the same disease, such as pancreatic cancer, by largely the same surgeons and in the same institution can improve progressively with the incorporation of incremental improvements in technique(3). What of variations in the standard of case selection, some physicians tending to refer cases much later in the course of a disease than others, and supportive perioperative care (4)? Doing an ileoanal using a Soave technique in a female teenager with ulcerative colitis referred within weeks of onset before she develops transmucosal ulceration (5), for example, is a completely different kettle of fish from doing it in an adult male with severe and long-standing transmucosal ulcerations referred years later. The same applies to the manner in which a proctectomy is performed for this disease, morbidity being greatly reduced by using a intrpshinctric technique (6) and avoiding the use of perineal drains. There are also important differences in minor technical preferences, such as the choice of incision, suture material, and the use of drains. A more recent variation is laparoscopy, the relative short and long term benefits of open and laparoscopic techniques being far from established (7). Clearly the only meaningful measure of expertise is the outcomes achieved. There are, however, huge variations in referral patterns and in the cases of the less common diseases, such as ulcerative colitis, it can take years to define the level of expertise even if cases have been preferentially directed to the individual in question. That is why personal experiences have traditonally formed the basis of the outcomes reported in the literature. Herein lies the rub, using these data to make their clinical decisions for performed by surgeons with lesser skills. High volume cases, such as inguinal hernias and CABGs, are the only ones in which prospective randomised studies can be meaningfully applied in a timely manner. For most cases, therefore, certification, recertification, years in practice, seniority, and number of cases done are the only practical means for determining expertise. But they are largely meaningless. What then is the solution? The Mayo Clinic one, appointing the most promising juniors in one's own training program and feeding them cases? But that could promote inbreeding and hinder progress. At Groote Schuur, for example, decades of inbreeding has lead to the evolution of different standards of care for gastric, oesophageal and pancreatic diseases. Paradoxically it is not the surgeons but the referring specialist physicians that appear to have been largely responsible for they have controlled the selection and flow of cases. Separating cardiothoracic surgery from general surgery has, however, clearly been another factor. It is highly probable that subspecialisation and especially superspecialisation (8) have been additonal contributing factors in the UK and the US. I have been accustomed to having cases referred to me by conservative gastroenterologists usually when they have failed a trial of conservative treatment. In many cases I have felt that earlier referral would have been more appropriate, but I have valued the break especially as I participated in their conferences and givene and been given feed-back. It may have made me less discriminating of case selection. In retrospect it would appear that I was referred less suitable cases as a premeditated means of testing my competence. What was particlarly misleading was the referral of such cases by clinicians whose assessments I had come to value and not question. Having key information deliberately withheld from me and dysinformation fed to me seemingly in a delibrate attempt to lead me astray was not something I would ever have imagined possible in my wildest of dreams. But perturbation of normal practice with premeditated political intent is hardly the norm. Had I been warned of the possibility I might have made different decisions but then I was often critical of established practices and constantly looking for novel ways to improve outcome. It is a sad commentry on the state of medical practice to conclude that the possiblity that these means might be used to confound the outcomes of prospective randomised studies should always be considered. But that is a poor reason for being intimidated into not attempting to do a prospective study or rejecting a free market model of healthcare for improving outcomes. 1. P J Devereaux, Mohit Bhandari, Mike Clarke, Victor M Montori, Deborah J Cook, Salim Yusuf, David L Sackett, Claudio S Cinà, S D Walter, Brian Haynes, Holger J Schünemann, Geoffrey R Norman, and Gordon H Guyatt Need for expertise based randomised controlled trials BMJ 2005; 330: 88 2. Sosa JA, Bowman HM, Gordon TA, Bass EB, Yeo CJ, Lillemoe KD, Pitt HA, Tielsch JM, Cameron JL Importance of hospital volume in the overall management of pancreatic cancer. Ann Surg. 1998 Sep;228(3):429-38 3. Crist DW, Sitzmann JV, Cameron JL. Improved hospital morbidity, mortality, and survival after the Whipple procedure. Ann Surg. 1987 Sep;206(3):358-65. 4. Mythen MG, Webb AR. Perioperative plasma volume expansion reduces the incidence of gut mucosal hypoperfusion during cardiac surgery. Arch Surg. 1995 Apr;130(4):423-9. 5. Coran AG, Sarahan TM, Dent TL, Fiddian-Green R, Wesley JR, Jordan FT. The endorectal pull-through for the management of ulcerative colitis in children and adults. Ann Surg. 1983 Jan;197(1):99-105. 6. Zeitels JR, Fiddian-Green RG, Dent TL. Intersphincteric proctectomy. Surgery. 1984 Oct;96(4):617-23. 7. Fiddian-Green RG. Open versus laparoscopy assisted colectomy. Lancet. 2003 Jan 4;361(9351):74; author reply 75-6. 8. Silen W. Super-specialization fellowships in gastrointestinal surgery: an unrealistic dream. Surgery. 1992 Apr;111(4):479-80. Competing interests: None declared |
|||
|
|
|||
|
Gautham K Suresh, Assistant Professor and Neonatologist Medical University of South Carolina, Charleston, SC 29425, USA
Send response to journal:
|
To the Editor, Devereaux et al (1) make a convincing case for expertise-based randomized trials. However, it is important to realize that the interventions that are tested in such trials usually are combinations of several component sub-interventions, such as the exact surgical technique, the technology used in performing the procedure, and the performance of the operator or surgeon. Expertise is only one component of performance. A variety of other variables, well known to those who work in the field of human factors engineering(2), can affect how well a physician performs an invasive procedure or surgery. These include variables such as fatigue, emotional state, physical deficiencies (such as poor vision), the physical characteristics of the the operating room, and situational distractions. Therefore in designing the intervention, great care should be taken to standardize as many of these variables as possible. Hopefully, randomization will ensure that patient level co-variates that can affect performance and outcomes, such as obesity or unusual anatomy, will be evenly allocated across intervention and control groups. Also, the authors propose dichotomizing surgeons into experts and non -experts in a given surgical procedure. Isn't expertise more likely to exist on a continuous scale, with different surgeons possessing different degrees of expertise, than on a dichotomous scale with just two types of surgeons-expert and non-expert ? Even if we were to try to categorize surgeons, is there an accurate way to measure expertise, so we can confidently say that a given surgeon is an expert in a procedure? 1. Devereaux P J,Bhandari M, Clarke M, et al. Need for expertise based randomised controlled trials. BMJ 2005; 330: 88 2. de Leval MR. Human factors and surgical outcomes: a Cartesian dream. Lancet 1997; 349: 723 - 725 Competing interests: None declared |
|||
|
|
|||
|
Eric Lim, Specialist Registrar Papworth Hospital
Send response to journal:
|
Editor, There are problems with the proposal of Devereaux and colleagues on the use of expertise based design to enhance the validity and applicability of randomised trials in surgery. 1. The use of expertise based designs does not necessarily enhance the validity of a surgical trial. Surgical outcome does not depend solely on the operation; other influences include postoperative management, the surgical team and equipment. A different bias is introduced by the expertise based design, the influence of the overall performance of surgeon A versus B, and in this regard, expertise based design is not necessarily a more valid comparison of operation A versus B. Although differential expertise can be reduced by the expertise based design, surgeons that pioneer new or technically more challenging techniques may have better overall skills in compared to surgeons who only use conventional techniques. Both of whom may be regarded as `experts’, but the results may be potentially biased in favour of the new treatment arm. The different factors that can influence the results of an operation performed by two groups of surgeons are heterogenous and immeasurable and can be minimised by simple randomisation stratified by surgeon (that is if one considers that the results of an operation is influenced by more than the technical procedure per se). The negative impact of differential expertise is based on the assumption that an operation is performed best by a surgeon with extensive expertise, but how about surgeons with less expertise but much greater surgical skill? A more pragmatic approach is to evaluate the results of an operation performed by surgeons with a range of surgical skills and familiarly with a new procedure, as it would be more reflective of real practice. The fact that surgeons are unblinded to treatment allocation and may potentially influence the outcome of an intervention is not specific to surgical trials, the same criticism applies to any unblinded medical trial. In expertise based trials, there is also the possibility that surgeons performing a new intervention may subconsciously be more susceptible to influence the results of a new intervention as opposed to the group of surgeons performing the standard procedure. 2. The use of expertise based designs does not necessarily enhance the applicability of a surgical trial. If there is a clear benefit from a different operation, then pragmatically, surgeons should consider adopting the superior intervention as opposed to stopping their practice and not seeing any more patients that may require the inferior operation. Expertise based designs ignores any learning curve, but few surgeons can become experts overnight. Surely the initial acquisition of the necessary skills in surgeons that subsequently become experts was by the refinement and performance of the operation on numerous patients. The number of experts for a particular procedure must be renewable, but how do surgeons acquire skills to a similar level without going though the same process? No amount of simulator training or animal work can be equivalent to performing the new procedure on a regular basis. Invariably the initial rate of adverse outcomes will be higher whilst a surgeon refines an operative technique and acquires new expertise, even for established operations[1] never mind a new one. This would not be reflected in results of surgical trials using an expertise based design, and the trial results in general would not be applicable to the standard population of surgeons. Reference 1. Bridgewater B, Grayson AD, Au J, Hassan R, Dihmis WC, Munsch C, Waterworth P. Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeons. Bmj. 2004;329:421. Competing interests: None declared |
|||
|
|
|||
|
Andrew Renaut, MS(Lond) FRCS(Gen) FRACS FDSRCS, Colorectal & Endoscopic Surgeon, Director New Zealand Institute of Advanced Laparoscopic Surgery The Oxford Clinic, Christchurch, NZ
Send response to journal:
|
This is a spectacularly good article. Along with the some of the excellent responses it highlights the major pitfalls in attempting to assess the efficacy of a particular surgical technique over another using randomized controlled trials. As with all good articles it raises more philosophical questions than it answers, including moral and legal issues. Expertise is of course delivered by an expert, but until we define the latter how are we to make any progress? Seniority is certainly not a pre-requisite – quite the opposite in most cases. A surgeon who performs just one type of operation can be expected to become very good at it, but there is still likely to exist some variation between individual practitioners even at this level - so who is to judge the expert? Should individual surgeons whose results do not match those of the expert in the trial be forced to hang up his/her scalpel? In the current climate, lawyers will have a field day if this premise is applied to ‘malpractice’. I have long held the view that the majority of surgical trials are not worth the paper they’re written on. I have instead relied upon a ‘best practice’ approach using stringent personal outcome audits within a relatively restricted surgical practice. I certainly know what I’m good at and when I fall below what is quoted then I’ve promised myself and my patients that I will head for the nearest trout stream, rod in hand, rather than to the OR. Competing interests: None declared |
|||
|
|
|||
|
Sanjay Dalmia, Registrar, Surgery Royal Shrewsbury Hospital
Send response to journal:
|
Surgical practice has traditionally been described as something learnt by apprenticeship.It takes a variable period of time to learn a surgical technique.Evidence based medicine on the other hand evaluates new techniques and procedures. As a Surgical Registrar since last six years I had the good fortune of seeing a wide range of surgical procedures performed in a wider range of approaches.Surgeons can be critical of a different approach, at times strongly. Expertise bias has always been there in Surgery and will probably remain so.As a surgeon I would like to carry out a procedure safely that has produced good results in my patients over the years. A well designed trial can atleast tell us that procedure A is not worse than procedure B and hence is of some use.A balance between expertise bias and new techniques is probably the answer. Competing interests: None declared |
|||
|
|
|||
|
javaid ahmad BUTT, lecturer in general surgery St.James hospital.Dublin.
Send response to journal:
|
surgical expertise is changing constantly with new inovations and technologies.I being a surgeon of many years of experience have constantly changed every surgical procedure I ever performed starting from a simple hernia to aortic aneurysm repair.Intention of this change is better outcome for the patients,therefore the argument for randomising the patient to 'an expert' for any surgical intervention lose its credibilty if that procedure is bound to change following month. Competing interests: None declared |
|||
|
|
|||
|
Simon Hatcher, Senior Lecturer in Psychiatry University of Auckland, 1
Send response to journal:
|
Of course the expertise based randomised trial is the norm in psychotherapy research when comparing two different psychotherapies. There has been a similar debate in the psychotherapy literature on the interpretation of such trials. Surgery and psychotherapy research share other similarities beyond having to account for practitioner expertise. There is the issue of blindness - hard to achieve for both patient and doctor in these areas; plus the "why test it, it's obvious it makes a difference" argument. Both disciplines could learn from each other about the design and analysis of clinical research. Competing interests: None declared |
|||
|
|
|||
|
Simon G Thompson, Director MRC Biostatistics Unit, Cambridge, CB2 2SR, Katherine J Lee
Send response to journal:
|
Randomised trials of surgical procedures can be more difficult to undertake and interpret than trials of pharmacological interventions, as Devereaux and colleagues explain [1]. One issue that they do not discuss is the need to account for the variability in effectiveness between surgeons or surgical teams, irrespective of the method of randomisation. Not all surgeons will undertake a procedure with equal skill, and so the outcomes for patients will differ between surgeons [2,3]. This variability between surgeons needs to be allowed for in the analysis of data from trials [4]. Also, since this variability decreases the precision of the treatment effect estimate, sample sizes have to be inflated to maintain power [5]. In a conventional surgical trial, surgeons with more expertise in procedure A than procedure B will tend to get better results with procedure A [1]. Conversely, surgeons with more expertise in B than A will get better results with B. Hence there may be a substantial treatment-surgeon interaction, which needs to be acknowledged in the analysis and will lower the precision of the estimated average difference between the treatments [4]. In the expertise based trial, as advocated by Devereaux and colleagues [1], surgeons with expertise in procedure A only undertake procedure A, and similarly for B. Nevertheless variability in outcomes between surgeons will still lower the precision of the overall treatment effect [4], but maybe not to such a large extent as in a conventional trial since surgeons now only undertake the procedure that they are familiar with. Thus an additional advantage of the expertise based trial may be that the effect of the variability in outcomes between surgeons is reduced. 1. Devereaux PJ, Bhandari M, Clarke M et al. Need for expertise based randomised controlled trials. BMJ 2005; 330: 88-91. 2. Williams AC, Sandy JR, Thomas S et al. Influence of surgeon's experience on speech outcome in cleft lip and palate. Lancet 1999; 354: 1697-8. 3. Witt PD, Wahlen JC, Marsh JL et al. The effect of surgeon's experience on velopharyngeal functional outcome following palatoplasty: is there a learning curve? Plastic and Reconstructive Surgery 1998; 102: 1375-85. 4. Lee KJ, Thompson SG. Clustering by health professional in individually randomised trials. BMJ 2005; 330: 142-4. 5. Hoover DR. Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects. Stat Med 2002; 21: 1351-64. Competing interests: None declared |
|||
|
|
|||
|
Swaroop S. Vedula, Research Associate Cochrane Eyes and Vision Group, US Project; Brown University; Providence; RI, USA 02912, Nagarjun Rayapudi
Send response to journal:
|
We congratulate the authors on the well-built argument about validity, applicability, feasibility and ethical integrity of the design of expertise based randomized controlled trials (1). Below we present a few comments on the different aspects of the design that were discussed and raise a few questions: In figure 1 of the article, showing the potential for differential expertise in a trial with different proportions of participants in each arm managed by “experts”, which factor is more important for differential expertise bias – differential expertise of the participating surgeons for their respective procedures in each arm or differential proportion of participants in each arm managed by “experts” in that particular procedure? How does “expertise” discussed in this paper, relate to standardization of surgical procedure and post-operative care protocols in a conventional randomized trial comparing two surgical interventions? Don't standardization and adequate adherence to the protocol by participating surgeons adequately minimize the proposed role of “expertise”? As discussed by McLeod (2), standardization of procedure, peri-operative care and post-operative care is important for surgical trials and may determine whether the trial is explanatory or pragmatic. Regarding the “learning curve”, mentioned in your paper - we note from a comprehensive review by Ramsay, et.al (3), that there is no consensus on the variables that reflect “learning” and “expertise” and on statistical methods to analyze any such variable. This highlights the methodological challenges to implement expertise based study designs. Regarding the issue of differential technical challenge for surgeons in the two trial arms, would not “expert” surgeons for a given procedure "perceive" different levels of technical challenge than those who are not? And would not this perception, ideally, be equal among the “expert” surgeons in both trial arms for their respective procedures thus annulling the proposed differential technical challenge? Our main concern about your argument is that it discussed situations “indicating the potential” and drew attention to “possible bias” in conventional randomized trial design. However, it fails to provide the “evidence” to show that such bias exists, the magnitude and direction of such bias and that the proposed “expertise based design” minimizes it. At the same time, we realize that this may be due to limited data to support such arguments. As a sequel to the above, we wish to ask - what is “expertise”? How is it defined? Can it be quantified to verify the existence of differential expertise bias in the conventional design and its absence in the expertise based design? How can it be quantified? Relating this to your argument about “applicability” of the expertise based design, how does a surgeon in the community know whether he/she has acquired the same skill set and "expertise" as those involved in an explanatory trial based on this design? In summary, we think that there are quite a few methodological issues that need to be established/refuted about the proposed design. References: 1. Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, Sackett DL, Cinà CS, Walter SD, Haynes B, Schünemann HJ, Norman GR, Guyatt GH. Need for expertise based randomised controlled trials. British Medical Journal. 2005; 330: 88 2. McLeod RS. Issues in surgical randomized trials. World Journal of Surgery. 1999; 23: 1210-14 3. Ramsay CG, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of the learning curve in health technologies. International Journal of Technology Assessment in Health Care. 2000; 16 (4): 1095-1108 Competing interests: None declared |
|||