Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Are experimental treatments truly (on average) superior to standard treatments?
- Benjamin Djulbegovic, Heloisa Soares, Ambuj Kumar and Fadila Serdarevic (28 May 2004)
-
Ask the wrong question - Get the wrong answer
- Tzippora Shochat (18 June 2004)
-
The Uncertainty Principle
- Jeremy Weinbren (19 June 2004)
-
Does this tell us much?
- Bakhtyari Arash (22 June 2004)
-
Prior beliefs and the uncertainty principle
- Waseem Sharieff (23 June 2004)
-
Whose Equipose matters?
- Robert C Kane, MD (27 June 2004)
-
Are cancer trials frequently overpowered?
- Dr Georg Roggla, Dr Sandra Fortunat (28 June 2004)
|
|
|||
|
Benjamin Djulbegovic, H. Lee Moffitt Cancer Center & Research Institute, University of South Florida 12902 Magnolia Dr, Tampa, FL 33612, USA, Heloisa Soares, Ambuj Kumar and Fadila Serdarevic
Send response to journal:
|
Several years ago we postulated that there is a predictable relationship between the uncertainty principle, that is, the moral principle, upon which trials are based and the ultimate outcomes of clinical trials [1]. This hypothesis predicts that, on average, one would expect that equal proportion of innovative (experimental) and standard treatments to be successful [1]. Otherwise, the entire clinical trial system may become jeopardized since the patients would likely refuse to be randomized and request those treatments that are expected to be, on average, superior. Therefore, the accurate answer about a distribution of the outcomes in clinical trials is of profound importance. A paper by Steven Jofee and colleagues focused on ECOG and CALGB cohort of the studies performed from 1981 to 1995 (n=93) indicate that there appears slight, but nevertheless significant advantage for the experimental treatments. Of interest is, however, that we are conducting almost identical study but attempting to examine all trials conducted by all cooperative groups (COGs) sponsored by the NCI. So far, we are aware of 812 completed trials conducted by COGs since the groups were founded. Although our analyses are still ongoing, our results so far indicate that there is an equal chance for innovative and standard treatment to result in successful outcomes or in the outcomes that may not differ between two types of the treatments [2,3]. The reason for the difference between our results and Joffe's and colleagues could be a cohort-specific (we are yet to analyze ECOG and CALGB cohort of the trials), eligibility criteria (our universe of the trials includes both published and unpublished trials conducted by the COGs since their inception vs. those performed between 1981-1995), method of the analysis (we elect to meta-analyze data vs. using geometric means), the approach to quality assessment [4] and the analysis of the choice of comparator intervention [1]. Therefore, we believe that further empirical research regarding assessment of a distribution of the outcomes between innovative and standard treatments [5] should continue, and ideally should encompass the universe of all studies from a common funder (e.g. NCI in the case of cancer trials). The future of the entire clinical trial system depends on the accurate and comprehensive answer to this question. References: 1. Djulbegovic B. Acknowledgment of Uncertainty: A Fundamental Means to Ensure Scientific and Ethical Validity in Clinical Research. Current Oncology Reports 2001;3:389-95. 2. Djulbegovic B, Soares H, Daniels S, et al. Evaluation of new treatments in cancer: are they better than standard treatments? XI Cochrane Colloqium 2003, Barcelona, October 26-31: 14. 3. Kumar A, Soares HP, Wells RJ, et al. Experimental vs. control interventions in cancer: which is better? The Children's Oncology Group Experience. Proc Am Soc Clin Oncol 2004(23):521. 4. Soares HP, Daniels S, Kumar A, et al. Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ 2003;328:22-5. 5. Chalmers I. What is the prior probability of a proposed new treatment being superior to established treatments? BMJ 1997;314:74-75. Competing interests: None declared |
|||
|
|
|||
|
Tzippora Shochat, statistical consultant Israel
Send response to journal:
|
The above article attempts to check adherance to the uncertainty principal in cancer trails. The authors suggest this principal is upheld if, on the average, experimental procedures are not found to be significantly better than standard ones. If this were indeed the case, huge sums of time, money and other resources are beeing spent in vain. In fact, in a ideal world, all experimental drugs and procedures should be found to be significantly better, i.e. the researchers hunch and the results from animal studies would always add to humans well-being. The uncertainty principal does not state that the experimental procedure should not be found significant, but rather that not enough prior knowledge or evidence exsist before a human trail is conducted. In baysian terms - the prior probalities are, lacking knowledge, assumed equal. But one should hope that the postier probabilities (i.e. after a study is conducted) are if favour of the new and promising drug. In order to check adherance to the "uncertainty principal" one could interview ethics boards and docters involved in enrollment of patients. The above article gives no relavent information as to adherance to this principle. Thankyou. Competing interests: None declared |
|||
|
|
|||
|
Jeremy Weinbren, Anaesthetist Hillingdon Hospital, Uxbridge, UB8 3NN
Send response to journal:
|
I read with interest the paper on Uncertainty & Equipoise. It has long been an unanswered question in my mind as to whether the Uncertainty Principle ever applies in well designed Randomised Controlled Trials (RCTs). My reasoning is as follows: In designing an RCT, the competent researcher decides on a sample size for the study. This is based upon either the expected proportions of positive responses in each of the groups, or the expected means in the randomised groups. Also factored into the equation is the desired power of the study and the level of statistical probability which will be deemed significant. Ignoring the fact that the calculations are based on parametric statistical methods (even if the subsequent results are only appropriate for non-parametric methods), it becomes clear that there must be some expectations of approximate values of results or proportions in order to generate sample size numbers. Otherwise, every RCT would require infinite numbers if there was no assumed difference in the groups, as presented to the various sample size calculation equations. Therefore, I would suggest that the Uncertainty Principle is violated to an extent every time a RCT is designed. This probably stems from the fact that statistical methods have the primary purpose of analysing and describing results which have already been obtained, but can only supply probabilities when it comes to future events. Statistics is by nature a retrospective set of mathematical techniques. Power calculations have been mathematically rearranged into sample size calculations. This is algebraically easy, but requires temporary suppression of Uncertainty and Equipoise for numerical convenience. Competing interests: None declared |
|||
|
|
|||
|
Bakhtyari Arash, Clinical Pharmacology Physician Nottingham
Send response to journal:
|
Although it is reassuring that the authors did not uncover any bias in terms of outcome favouring the experimental or established treatments, every proposed trial will have to be assessed on its own merit regarding whether there is enough uncertainty to a) proceed with the trial and b) to proceed with enrolling a patient in the trial. The underlying assumption however is that each trial is independent of what has gone on before. A little like the likelihood of having a healthy boy by an expectant couple if they have had say a healthy son already. Most poeple estimate the probability still in the region of 0.5. The independence is not always of course the case. If a couple have had a child affected by C.F. they have a different probability of having their second child affected by the condition than if their first child was healthy. Competing interests: None declared |
|||
|
|
|||
|
Waseem Sharieff, PhD Candidate The Hospital for Sick Children, University of Toronto, 555 University Avenue # 8259, M5G 1X8
Send response to journal:
|
Editor: Dr. Joffe and colleagues raise an important subject for discussion; the ethical basis of conducting a trial given the prior belief in favor of the effectiveness of a new treatment (1). Although, they showed an effect size favoring new treatment over standard, it was small in magnitude and reflected the prior uncertainty upon which trials were conducted. However, as authors pointed out given the limitations of their methods, it was not possible to elicit how strong the prior beliefs were, as a strong prior belief in favor of the new treatment would violate the uncertainty principle (2). I have the following comments: First, no matter how strong a clinician believes in the new treatment, in the absence of supporting evidence his/her prescribing the new treatment would be considered a deviation from standard practice; thus the uncertainty principle may be re-worded as "A patient can be entered [in a trial] if, and only if, [based on the existing evidence], the responsible clinician is substantially uncertain which of the trial treatments would be most appropriate for that particular patient......." Second, the clinician's prior belief in a new treatment is usually formed from a biological theory that evolves into a hypothesis which needs to be tested, and without which there would be no grounds for conducting a trial (3). Third, from a Bayesian perspective it is possible to predict outcomes for new trials provided enough data are available from previous trials (4,5); this does not hold for new regimens on which no previous trial data are available. 1. Joffe S, Harrington D, George S, Emanuel E, Budzinski L, Weeks J. Satisfaction of the uncertainty principle in cancer clinical trials: retrospective cohort analysis. BMJ 2004; 328:1463-6. 2. Peto R, Baigent C. Trials: the next 50 years. Large scale randomized evidence of moderate benefits. BMJ 1998;317:1170-1. 3. Lyman GH, Kuderer NM. A Primer for Evaluating Clinical Trials. Cancer Control 1997;4(5):413-418. 4. Chalmers I. What is the prior probability of a proposed new treatment being superior to established treatments? BMJ 1997;314:74-5. 5. Campbell MJ. Statistics at Square Two. Understanding modern statistical applications in medicine. London: BMJ Publishing Group, 2001. Competing interests: None declared |
|||
|
|
|||
|
Robert C Kane, MD, recovering practitioner Rockville, MD 20850
Send response to journal:
|
A valuable contribution, yet it is ultimately the equipose of the participating doctors and patients which results in the accrual to the trial. The authors acknowledge that this study might be viewed as assessing a "surrogate endpoint" for that outcome. The trials might be expected to have a modest positive balance, on average, since they were able to be completed. The rest of the story may reside in those group trials not completed due to poor accrual, in turn due to a perceived lack of equipose by patients or doctors; thus some method of adding in the trials closed for poor accrual might provide a more complete, or "intention to trial" population from which these trials might be viewed as a "subset analysis." Competing interests: None declared |
|||
|
|
|||
|
Dr Georg Roggla, Department of Internal Medicine, General Hospital of Neunkirchen A-2620 Neunkirchen, Austria, Dr Sandra Fortunat
Send response to journal:
|
We would like to draw attention to data the authors did not discuss in Joffes retrospective cohort analysis on satisfaction of the uncertainty principle in cancer clinical trials (1). The author reports, that statistical power was 0.85 to 0.89 in 15 of 94 studies (16.0%) and > 90 in 14 of 94 studies (14.9 %). Conventionally, a value of 0.80 is the target value for statistical power, representing a likelihood that four times out of five a study will reject a false null hypothesis. Jeffes data implicate that a considerable proportion of cancer trials seems to be overpowered. Careful sample size calculation is normally performed in order to prevent the possibility of a type II error and missing an association which exists (‘underpowering’) (2). Very little attention is paid to the pitfall of overpowering and thereby making a type I error more likely: finding an association which is not clinically important (but only statistically significant). Horrobin wonders in a Lancet personal paper whether pharmaceutical industry deliberately uses overpowered trials as a way of keeping competitors out of a particular subject. If a firm can recruit several times more patients than necessary for a trial especially in less common cancers, then they will gain a clear competitive advantage by making it more difficult for rivals to recruit (3). Based on Joffes report, we suggest that considerably more concern on the danger of overpowering cancer trials seems justified. References (1) Joffe S, Harrington DP, George SL, Emanuel EJ, Budzinski LA, Weeks JC. Satisfaction of the uncertainty principle in cancer clinical trials: retrospective cohort analysis. BMJ 2004;328:1463 (2) Halpern SD, Karlawish JHT, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA 2002; 288: 358-62. (3) Horrobin D. Are large clinical trials in rapidly lethal diseases usually unethical? Lancet 2003; 361: 695-97 Competing interests: None declared |
|||