Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Patients can't use pMDI's in real life
- Brian J Lipworth (21 October 2001)
-
Cheapest is best?
- Ian Williams (25 October 2001)
-
Meta-analysis: more stress on analysis please
- Stephen Senn (27 October 2001)
-
factual and methodological errors
- Richard Spiers (1 November 2001)
-
Systematic reviews of asthma inhalers – not generalisable without subject details
- Patrick T White (1 November 2001)
-
Meta-Madness
- Alyn H Morice (16 November 2001)
-
Oh Dear
- Mark Everard (16 November 2001)
|
|
|||
|
Brian J Lipworth, Professor of allergy and pulmonology University of Dundee ,Ninewells Hospital
Send response to journal:
|
The systematic review of Brocklebank et al is a good example of how meta-analysis can be misleading in terms of what we already know from everyday practice.The most common reason in my clinic for failure to respond to inhaled corticosteroid is inabilty to use a pMDI .Moreover even after educational input on correct use,most patients are still unable to correctly use a pMDI.The pMDI was cutting edge technology in the 1960's ,but nowadays there are several other portable breath actuated pressurised and dry powder inhalers,which are much easier to use,require less educational input and provide better lung depostion as well as having an actuation counter.All of these facotrs are likely to result in better long term compliance .In this respect the data from clinical trials using pMDI's will not reflect real life because patients were usually selected on the basis of being able to use a pMDI properly as an inclusion criteria -ie the results would always represent the best possible scenario for the clinical efficacy of pMDI as compared to other devices.In everyday practice one has to take into consideration the indirect costs of more intensive patient education and treatment failures when using pMDI's as compared to using the direct costs of using more expensive breath actuated devices ,especially in the long term.Pointedly none of the included studies evaluated long term effects over several years .Most of my specialist colleages in pulmonology gave up using pMDI's several years ago and I very much doubt whether this meta-analysis will have any impact on prescribing in secondary care.Whether or not primary care prescibers are more gullible to this type of analysis is open to debate.Sometimes one has to stand back from so called evidence based data and use a bit of common sense .Technology has moved on over the past four decades and this is something we should embrace rather than going back to the bad old days where patients could not their inhalers. BJL has recieved grant,eduacational and equipment support from companies who make pMDIs and breath actuated inhalers,including AstraZeneca,GlaxoSmithkline,Schering Plough,Boehringer Ingelheim,Novartis,Innovata Biomed,Aerogen,Aventis ,Orion,Yamanouchi.The spouse and mother of BJL have shares in GlaxoSmithkline. |
|||
|
|
|||
|
Ian Williams, GP Huntingdon Cambs
Send response to journal:
|
Both of the articles in this weeks copy of the BMJ have the same inherent flaw, which has also been pointed out by Prof. Lipworth in his response. It has long been true that the most expensive inhaler is the one that sits in the patient's bedside table and is not used because of inability to use it properly, or perceived lack of response. Many publications have backed up the view that patients cannot use properly and efficiently a pMDI, and that even after correct instruction in technique their "ability" declines rapidly with time. There is a huge danger with meta-analyses such as these, that they will be quoted with glee by PCO prescribing advisers as a push to move prescribing down the price range back to pMDI, but the evidence is that asthma control will deteriorate as a result. This is acknowledged in the conclusion of both articles "....the cheapest inhaler device that patients can use adequately should be used as first line treatment", but it is sad that this essential fact does not appear either in the abstract or in the "What this study adds" (the sections most likely to be read in Primary Care)in fact the latter has the very dangerous conclusion that "Pressurised metered dose inhalers (or the cheapest device) should be first line treatment in all patients with stable asthma. Competing Interests: Dr Williams has chaired meetings and been a guest at respiratory care meetings funded by Glaxo Wellcome, and Astra Zeneca. |
|||
|
|
|||
|
Stephen Senn, Professor of Pharmaceutical and Health Statistics University College London
Send response to journal:
|
The recent meta-analysis by Brocklebank et al1 has many virtues, not least the energy and thoroughness with which it has been prosecuted. Also, in their more extensive Health Technology Assessment report, available on the web, much further helpful information is recorded by the authors in a detail that is rare outside the drug regulatory context. Nevertheless, there are inappropriate features in the analysis which, in my view, reflect inherent dangers in the Cochrane Collaboration approach to meta- analysis. Since it has been claimed by a paper2 in this journal that in asthma Cochrane reviews are superior to others, it is worth looking at the problems with the analysis by Brockelbank et al. As is common in Cochrane reviews, the authors have relied on the software RevMan. Rather than proceeding from using treatment contrasts and standard errors as the raw input for a meta-analysis this requires means, standard deviations and numbers for each treatment group. This approach is only valid for single-centre parallel-group trials and is completely inappropriate for cross-over trials, for example3. Yet the authors have forced cross-over trials into this straight-jacket claiming that it is necessary to do so because within-patient errors are not available. Where, however, treatment differences and standard errors of differences are already recorded this is not true provided one is not committed to using RevMan. The requirement of RevMan to have everything presented as a parallel group trial has led the authors further astray. For example, to enable the results of the Vidgren et al trial4, which had three arms, to be "understood" by RevMan, they have entered this twice as two separate trials but using the same control data. This means that in their overall summary of figure 2 the control data are counted twice, a very serious methodological error. (Two typos in this figure need correcting: the upper confidence limits for the first two trials have minus signs missing.) A further aspect of their approach betrays a misunderstanding of measurement in clinical trial. Contrary to what they claim, provided that the same measure of lung-function is used "absolute and improvement from baseline" measure exactly the same thing, as may be simply seen when we consider that in the perfectly balanced trial they give the same answer5. There is thus no need to use the method of standardise mean differences to combine them. Finally , out of 28 standard deviations quoted in Figure 2, 10 have a value of 100.0. Can these be real? Surely they must be imputed. It seems to me that more space could have been spent discussing this rather important point and less discussing the use of Cochrane' preferred quality assessment index, which places virtually no premium on proper analysis3. 1. Brocklebank D, Wright J, Cates C. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering corticosteroids in asthma. Br. Med. J. 2001;323:896-902. 2. Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. Br. Med. J. 2000;320(7234):537-40. 3. Senn SJ. Review is biased. Br. Med. J. 2000;321:p297. 4. Vidgren P, Silvasti M, Poukkula A, Laasonen K, Vidgren M. Easyhaler Powder Inhaler - a New Alternative in the Antiinflammatory Treatment of Asthma. Acta Ther. 1994;20(3-4):117-131. 5. Senn SJ. Baseline comparisons in randomized clinical trials. Stat. Med. 1991;10(7):1157-9. Declaration of interest. The author is a consultant to the pharmaceutical industry |
|||
|
|
|||
|
Richard Spiers, Medical Director 3M UK and Ireland 3M Health care Medical Department
Send response to journal:
|
In the systematic review by Brocklebank et al(1) , the meta-analysis of dose ratios of HFA and CFC BDP present a number of questions for healthcare professionals. The only HFA beclomethasone available in the UK is Qvar from 3M, licensed as equivalent in efficacy to CFC BDP at half the dose. If and when another HFA BDP is approved, the CMO’s guidance(2) concerning brand prescribing will ensure that there is no confusion during transition away from CFC products. We question the applicability of meta-analysis on HFA vs CFC BDP studies to meet the review’s objective in stable asthma, when only 2 of the 6 publications examined included stable patients. Busse studied patients with moderate to severe symptomatic asthma, who had undergone a steroid washout period prior to receiving treatment. Patients in the studies by Davies and Gross were symptomatic, had an oral steroid burst, and were then studied to examine maintenance of asthma control on inhaled steroids. Patients in studies by Demedts and Dahl (a cross over study) were all well controlled at study entry. Dosing frequencies and statistical methods also varied. The HFA vs CFC review also contains significant errors. The largest study reviewed was by Demedts (not Damedts). The Busse study was designed to, and demonstrated, the comparative dose responses FEV1 of CFC and HFA BDP across 100, 400 and 800 mcg. Brocklebank meta-analysed this study as four separate studies, using the secondary, patient diary recorded variable of peak expired flow. In addition to this erroneous interpretation, patient numbers and change from baseline in Figure 3 have been transposed, biasing the standardised mean difference and the subtotal plot. The review and supporting HTA also ignored the considerable correspondence(3) concerning Milanowski. Finally, the focus on efficacy, ignoring the comparative safety of HFA and CFC BDP is a significant omission. I trust that the work put into this HTA will not be devalued by errors such as those above and that a corrected assessment is forthcoming. Accurate reviews are crucial for evidence based clinical decision making. Dr Richard Spiers References: 1. Brocklebank D, Wright J, Cates C. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering corticosteroids in asthma. Br. Med. J. 2001;323:896-902. 2. CMO’s Update 17 February 1998. Propellant changes in metered dose inhalers 3. Letters to the Editor, Respiratory Medicine 2000; 94 (2) |
|||
|
|
|||
|
Patrick T White, Senior Lecturer in General Practice and Primary Care Department of General Practice and Primary Care, Guy's King's and St Thomas' School of Medicine,
Send response to journal:
|
Dear Sir The results in the two papers from the National Health Technology Assessment Inhaler Review Group may seem counter-intuitive to clinicians in practice.1,2 For many patients simple metered-dose inhalers (MDIs) are not as effective as breath activated inhalers and powder inhalers because they are too difficult to use. Why do these systematic reviews come to a different conclusion? The answer may be that the types of patient for whom the clinicians in every day practice choose to prescribe alternative devices were excluded from the clinical trials which these systematic reviews examined. The entry criteria for randomised controlled trials in asthma are likely to exclude patients who have difficulty learning, or patients who have difficulty in keeping diaries (this may be an aspect of learning difficulty), or particularly patients who have poor reversibility with a simple MDI. Patients with learning difficulties may find learning to use an MDI impossible. Patients who have poor reversibility with a simple MDI may have excellent reversibility with an alternative device if they can not use a simple MDI effectively. As many as 30% of patients in a general practice setting may have poor technique with an MDI.3 The generalisability of the results of randomised controlled trials begins with the entry criteria for patients in those trials. It is interesting that in the same issue of the British Medical Journal the generalisability of randomised controlled trials is discussed in relation to recombinant human activated protein C in sepsis in intensive care.3 In that article the importance of knowing the details of the subjects in a trial and their comparability with subjects in a clinical setting is highlighted. So also is it essential to know the details of the subjects in these papers on the clinical effectiveness of asthma inhalers before a judgement can be made. No details of the subjects in the original trials are given and the possible relevance of the trial subjects is not considered in the discussion. I am disappointed to conclude from these two papers that they can add little to my clinical practice. Systematic reviews can only offer limited insights if they fail to report the characteristics of the subjects in the trials they review. Yours sincerely, Patrick White
1. Brocklebank D, Wright J, Gates C on behalf of the National Health Technology Assessment Inhaler Review Group. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering corticosteroids in asthma. BMJ 2001;323:896-900. 2. Ram FSF, Wright J, Brocklebank D, White JES on behalf of the National Health Technology Assessment Inhaler Review Group. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering 2 agonist bronchodilators in asthma. BMJ 2001;323:901-5. 3. Hilton S. An audit of inhaler technique among asthma patients of 34 general practitioners. Br J Gen Pract 1990;40:505-506. 4. Padkin A, Rowan K, Black N. Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ 2001;323:923-6. |
|||
|
|
|||
|
Alyn H Morice, Professor of Respiratory Medicine Castle Hill Hospital, HU16 5JQ
Send response to journal:
|
On the 20th of October the BMJ published two striking examples of meta-madness(1; 2). The systematic review industry continues to pick over the corpses of long dead studies hoping to resuscitate silk purses despite good evidence that the whole flawed process is a little better than the toss of a coin when it comes to deciding treatment options(3). These recent studies however plumb new depths in data distortion. The combined studies look at over 100 randomised controlled comparisons of metered dose inhalers and a variety of dry powder devices. The authors conclude that metered dose inhalers are the most cost effective treatment and should be continued as the first line delivery devices. This is an astonishing conclusion considering none of the studies they quote used cost effectiveness as an end point. To design an experiment which demonstrates the dose relationship, never mind the cost effectiveness, of two different types of inhaler in asthma is extraordinarily difficult. The main problem is that conventional doses of both beta-agonists and inhaled steroids are near the maximum of the dose response curve. Thus, in their classic paper demonstrating the two to one efficacy of the turbohaler over the metered dose inhaler Agertoft and Pederson(4) found that half the children failed to exacerbate on reduction of their budesonide dose i.e. they were being over treated. In those patients who did exacerbate turbohaler at half the dose was more effective in clinical measures than the full conventional dose delivered by an MDI. Even this carefully designed study did not truly examine the cost effectiveness of the two different treatment strategies but merely demonstrates that this particularly inhaler was clinically more effective at half the dose. Any such studies which may have hinted at relative efficacy are drowned out by the large volume of studies without such careful entry criteria. Why have the majority of these comparative studies been performed? They are undertaken to demonstrate the equivalence of an MDI and a dry powder inhaler. This is because the regulatory authorities insist on novel inhalers being 'equivalent' to the standard preparation, almost invariably the MDI. Dry powder inhalers are deliberately 'dumbed down' to provide equivalence since much less clinical information is required for such an application. Devices which claim higher efficiency such as Qvar or the turbohaler are made to undergo many more costly studies. These bizarre regulations, which are almost entirely irrelevant clinically because asthma medication should be adjusted according to the individual patient response, have been previously criticised by myself and others(5). There are a number of manoeuvres that can be used to minimise the risk of demonstrating a difference in such licensing application studies. If high doses are used maximum clinical effect will be achieved whatever the device. It is interesting to note that the authors comment that ten of the twenty studies in adult asthma using inhaled corticosteroids were at doses so high they not reflect clinical practice. Power is obviously kept to a minimum in order to maximise the chance of a type II error. Such studies do not add to our knowledge of the relative clinical effectiveness of inhalers. However, their inclusion in a meta-analysis significantly degrades the power of that analysis to detect any true effect. One of the great joys of performing a prospective clinical trial as opposed to a retrospective analysis is that you do not know the answer before you start. In this regard it is perfectly reasonable to pick relative change from baseline or absolute change from baseline as your measure of efficacy in a particular study. Inconveniently for the authors three studies demonstrated greater efficacy with dry powder inhalers. These studies are subjected to post hoc analysis resulting in them becoming insignificant. This is data dredging of the worst sort; picking studies which do not fit your hypothesis and manipulating them until they do. At the very least any post hoc analysis should have been applied evenly over the whole data set. In all studies an entry criteria will have been that the patients were able to use the inhaler devices correctly. Problems with MDI training are well documented, indeed, legions of nurses are employed specifically for this purpose (how cost effective is that?). Thus the studies will have excluded the many patients for whom an MDI is inappropriate and yet the authors generalise their speculative conclusion to the whole asthmatic population. Finally, I was astonished to read that the authors felt that they had no competing interests. The Department of Health recommends metered dose inhaler usage on the erroneous and unsupported belief that it is more cost effective. The NHS Health and Research Development Health Technology Assessment Programme receives considerable funding from the Department of Health and pays the salary of some of the authors. It is not just big pharma which butters bread. Yours sincerely PROFESSOR ALYN H MORICE
Competing interests AHM has been reimbursed for attending symposia, received fees for speaking, consulting as well as receiving research funding from Astra Zeneca, Boehringer Ingelheim, Boots, GlaxoSmithKline, Merck Sharp Dohme, ML Pharmaceuticals, Novartis, Proctor & Gamble, Schering Plough, 3M. Reference List 1. Brocklebank D, Wright J, Cates C. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering corticosteroids in asthma. Br Med J 2001;323:896 2. Ram FSF, Wright J, Brocklebank D, White JES. Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering beta 2 agonists bronchodilators in asthma. Br Med J 2001;323:901 3. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N.Engl.J.Med. 1997;337:536-542. 4. Agertoft L, Pedersen S. Importance of the inhalation device on the effect of Budesonide. Arch Dis Child 1993;69:130-133. 5. Morice AH. CFC transition. Thorax 1901;56:501-502. |
|||
|
|
|||
|
Mark Everard, Consultant in Paediatric Respiratory Medicine Sheffield children's Hospital
Send response to journal:
|
Editor ‘Oh dear….’ The 'systematic' review of inhalers for delivering inhaled corticosteroids (ICS) by Brocklebank et al1 is a worthy, relatively comprehensive, ultimately disappointing and indeed misleading example of secondary research. Leaving aside the validity of their conclusions for the moment the concept of undertaking systematic reviews of ‘effectiveness’ of therapy with no consideration of safety is at best naïve and will certainly lead to breaches of our commitment to 'do no harm'. One of the principle reasons for developing holding chambers in the early 1980's was to improve the 'therapeutic index' of beclomethasone in response to the overt side effects such as thinning of the skin and purpura seen in the elderly. Conventional beclomethasone remains the most widely prescribed ICS in the UK and it is clear that the ‘therapeutic index’ changes considerably if a spacer is not used. A systematic review of the effectiveness of therapy for morning sickness would almost certainly indicate that thalidomide would be a valid cost effective option. As is so often is the case if you do not ask the right question you do not get the right answer. Perhaps more concerning is the authors and, to an even greater extent, the journals lack of insight into the 'generalisability' of the results. If secondary research is to be of value and influence clinical practice it must consider the characteristics of the studies that may influence the 'generalisability' of the results. There are two important factors which are not considered pertaining to inhaled therapy. The journal stresses that ' pMDIs should be the first line in all patients with stable asthma'. However the vast majority of studies utilise a pMDI and spacer as it is widely accepted that many patients cannot use a pMDI effectively. pMDIs are simple to used but difficult to use effectively. More importantly, there is no consideration of the inclusion criteria used for each study. Though I have not had the opportunity to review all the papers I suspect the majority, if not all, of the studies have as an inclusion criteria the ability to use the devices under consideration. In many cases it may be necessary to approach the original investigators for this information. It is entirely predictable that we will see only small differences if we review studies which only include subjects who can effectively use both devices at the commencement of the study and who are reviewed at frequent intervals over the short period of the study. This does not represent practice in the UK where lack of competence with devices, particularly in the elderly, and contriving to use a device ineffectively (contrivance), as when patients choose not to use the spacer, are both very common2. Moreover few patients receive regular intensive training. The authors do note however the very high presumably supra-maximal levels of ICS used in so many of the studies which will mask any significant differences between devices in terms of effectiveness as the dose used in clinical practice should, almost certainly, be ‘the lowest dose that works’. As a consequence the findings of this piece of secondary research are of little relevance to clinical practice. The journal’s bullet points represent another example of taking one piece out of a complex jigsaw and over interpreting the results. Though there are proponents of DPIs these are again not without their problems. At present we do not have the evidence on which to make blanket recommendations as the appropriate trials have not been undertaken. The choice of inhaler for ICS is very important – if a patient chooses to comply with therapy it is our duty to provide them with a device that will reliably and effectively deliver the ICS to the lungs. More over the ‘therapeutic index’ of the device/drug combination should be acceptable. At present there is no convincing evidence that any one ICS is safer than another, if used with an appropriate delivery system and is used at the ‘lowest dose that works’. The choice of delivery system for agonists is a lifestyle choice as these are administered at supra-maximal doses and patients can titrate the dose against response both of which will compensate for poor technique The reality is that none of the current devices are ideal and that there is still a need to develop devices that are intuitive to use so that all patients can and will use them effectively. In the meantime choice of delivery system for ICS is dependent upon selecting an appropriate device that a patient can and will use effectively. 1. Brocklebank D, Wright J, Cates C. Systematic review of clinical effectiveness of pressurissed metered dose inhalers versus other hand held inhaler devices for delivery of corticosteroids in asthma. BMJ 2001; 323: 896-900 2 Everard ML. CFC transition: the Emperor’s new clothes. Thorax 2000; 55: 811-14 |
|||