Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Issues on prospective method for estimating rates of adverse events
- Dr Ali Baba-Akbari Sari (26 January 2004)
-
Prevention : the next step in the measurement of adverse events
- Guy Haller, Just Stoelwinder (5 February 2004)
-
Other methods for rates of adverse events?
- Roberto Natangelo (20 February 2004)
-
Comparison of Paired Proportions
- Sanjeev Sarmukaddam, Pune-411001, INDIA (1 February 2009)
|
|
|||
|
Dr Ali Baba-Akbari Sari, Postgraduate Researcher University of York
Send response to journal:
|
This study provides very useful information on the methodology of detecting and measuring adverse events. However, there are some potentially important issues that might need more consideration. The reported proportion of adverse events in this study is significantly greater than many previous studies 1-4. This is partly because the authors have used a combination of methods (review of medical records and interviews with medical team) to detect adverse events. This is similar to the capture-recapture technique, which is usually used when there is no single reliable source of information. The possibility of a Hawthorne effect should also be considered when using the prospective method. In this project, clinicians knew the study patients from the beginning of their admissions, so it was possible to both treat the study patients and complete their medical records differently, and this might change the rate of adverse events. Adverse events can happen anytime between the admission, discharge and post-discharge, (i.e. surgical wound infection). The prospective method could miss those adverse events that might happen after the interviews. In patients who had two or more adverse events (i.e. adverse event A and B), each prospective or retrospective method might detect adverse event A but miss adverse event B and vice versa. It is possible that the two methods detect similar number of patients with adverse events; however, the number and type of adverse events detected by each method could be different. Therefore, comparing the prospective and retrospective methods based on the number and types of adverse events may provide some potentially useful information as well. Reference List 1. Brenan TA, Leap LL, Laird NM, Hebert L, Localio AR, and Lawthers AG. Incidence of adverse events and negligence in hospitalised patients. N Engl J Med 1991; 324, 370-376. 2. Gawande AA, Thomas EJ, Zinner MJ, and Brennan TA. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 1999; 126, 66-75. 3. Vincent Ch, Neale G, and Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001; 322(7285), 517-519. 4. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, and Hamilton JD. QAHCS_The Quality in Australian Health Care Study. Med J Aust 1995; 163, 458-471. Competing interests: None declared |
|||
|
|
|||
|
Guy Haller, Anaesthetist-PhD scholar Monash University Department of Epidemiology 3181 Melbourne/Australia, Just Stoelwinder
Send response to journal:
|
The cornerstone of any quality strategy is prevention. If preventable adverse events can be reliably detected, improvement strategies can be designed. Unfortunately, none of the methods tested in the study by Michel et al 1 gives a definite answer to this issue. The prospective method detects preventable adverse events more effectively in medicine. The retrospective method is equally effective when surgical cases are added. This warrants further comments. The gold standard in assessing the preventability of an adverse event was for a long time the so called ‘Bolam Test’: the judgement of a responsible body of medical peers. The validity of the Bolam Principle was questioned by the House of Lords in the Bolitho case 2. It resulted in a major controversy without definite answer. There is no real consensus on the definition of a preventable adverse event. Secondly, judgement on the preventability of an adverse event changes according to the kind of data that is collected. From related studies, it appears that preventable adverse events are found more often in incident reports than in chart review processes.3,4 Finally, information on adverse events is mostly qualitative. Classifying such events for standardised comparison and reliably assessing their preventability, represents the real challenge. The excellent study by Michel et al 1 opens the door for validating methods of adverse event measurement; prevention has still a long way to go. REFERENCES 1 Michel P, Quenon JL, de Sarasqueta AM, Scemama O. Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. Bmj 2004; 328: 199. 2 Samanta A, Samanta J. Legal standard of care: a shift from the traditional Bolam test. Clin Med 2003; 3: 443-6. 3 Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med 2003; 18: 61-7 4 O'Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan TA. Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 1993; 119: 370-6. Competing interests: None declared |
|||
|
|
|||
|
Roberto Natangelo, Retired physician Cittadinaza attiva. V Mecenate 25 20138 Milan Italy
Send response to journal:
|
What do you think of linking methods for estimating rates of adverse events and patients complaints? In Italy, the National Health Service does not support an independent agency or patient representative. In Milan, we carried out a survey to explore if an independent patient’s agency could facilitate local learning and action to improve the quality of health services from users’ perspectives. We analysed 140 forms of complaints people reported to our voluntary citizen’s association (Cittadinanza attiva). Every complaint enclosed one or more clinical records. Our aim was to compare allegations contained in the written complaints and the results of a retrospective case records review. The main causes of the complaint (according to the patients) were: *A delay in diagnosis and treatment. *The failure or a complication in the technical performance of an indicated operation or invasive procedure. *Lack of care or attention, failure to attend, lack in monitoring of a patient. According to the physician record review, 51 out of 140 complaints were associated with a preventable adverse event (mainly occurring in hospital); 9 of them were serious and preventable. We know that there are a number of potential shortcomings in our study. Nevertheless, the survey show what kind of useful information about substandard medical care we can gather also from this type of documentation. The complaints handled by an independent agency will good represent the views of the people in this field. Roberto Natangelo
Competing interests: None declared |
|||
|
|
|||
|
Sanjeev Sarmukaddam, Consultant in Biostatistics MIMH, B.J.Medical College & Sassoon Hospital, Pune-411001, INDIA
Send response to journal:
|
Many times in the field of medical/bio sciences the data are collected meticulously with tremendous efforts but finally analyzed hurriedly by wrong choice of method. One very often faced situation is ‘comparison of paired proportions’. It is necessary to make or understand very clearly whether the data are from paired samples. Even when observations are paired in some way, (ex. two proportions are measured on the same individuals on two occasions say pre/post type or from studies/trials in which a matched paired design has been used) comparison of two proportions is very often done by two independent samples ‘normal test’. It is not to claim that anything suggested below is new (except simplification of formulae), the purpose is just to highlight this fact in view of its importance in ‘epidemiology’ and medical education. The material covered and the solutions offered are, to a large extent, known in the literature. Though a case of comparison of two proportions is discussed here, when there are more than two related/paired proportions, appropriate test for comparison is Chochran’s ‘Q’ test rather than simple, usual Chi-Square test. That test is a sort of extension of McNemar test details of which can be found in literature1. Published application of ‘Q test’ can also be seen in literature2. ‘Normal test’ specifically applicable to compare two paired proportions is also described in details in earlier quoted book1. Analyzing data from crossover design with binary response and exact test on matched pair proportions based on Binomial distribution is discussed with examples in another exclusive text book on Medical Biostatistics3. Case of two paired proportions: The method is described here with the help of one numerical example. In a study of ‘nutritional supplement’ efficacy, 300 children were examined before start of the supplement and also at the end of the study. About 50% children found to be under-nourished at baseline. This proportion came down to 43.33% at the end. The data can be displayed in a 2×2 table as follows : Table : Comparison of two paired proportions ¾ same 300 subjects were examined at both times (a) Correct layout (b) Incorrect layout Pre stage Normal Under-nourished Total Result Pre stage Post stage Total Post stage Normal 150 (=r) 20 (=s) 170 Normal 150 170 260 Under-nourished 0 (=t) 130 (=u) 130 Under-nourished 150 130 340 Total 150 150 300 Total 300 300 600 For this example, r=150, s=20, t=0, and u=130. It would be incorrect to arrange the data as in (b) part of the table and to apply the standard chi-squared test, (which is same as two independent samples ‘normal test’) as this would take no account of the paired nature of the data, namely that it was the same 300 subjects examined twice and not 600 different ones. Then p1=0.5000, p2=0.5667, the difference is 0.0667, together with its standard error, can be estimated from the numbers of discordant pairs, s & t, and the total number of pairs, n. Difference between paired proportions = [(s - t) / n] and approximate estimated Standard Error = 1/n * Ö{s+t-[(s - t)2 / n]}. Then test statistic Z is the ratio of difference and SE which cannot be estimated unless r, s, t, & u are known (not from p1, p2, and n alone). Nevertheless, it is possible to estimate others if any one of r, s, t, u is known along with p1, p2, and n. Lack of this information often limits the expected practice of making “paired-unpaired” differentiation while dealing with proportions. For our dataset SE= 0.0144 and Z=4.653, is highly significant. If you look at the other-side i.e. if you consider proportions of “under-nourished” instead of “normal”, the end result will remain same except that Z will bear negative sign. “t” can assumed to be a ‘structural’ zero reasonably since we expect change only in one direction (however, ‘t’ cannot be assumed as structural zero if we expect change in both the directions), then Standard Error = 1/n *Ö{ s-(s2/ n)} becomes approximately equal to Ö{(p2 - p1) / n} taking s = (p2- p1)*n. For our example it is 0.0149 which is slightly more than the actual. Note that ‘t’ zero (frequency of that cell being zero) is not same as ‘structural zero’. McNemar's chi-squared test (c2paired) which is based on the numbers of discordant pairs, s and t, is same (apart from rounding error) as the square of the ‘Z’ value indicating that the two tests are mathematically equivalent. It may be noted that two independent samples ‘normal test’ or usual Chi-square test yields non-significant result with these data (c2 = 2.6785 and Z =1.6429). Another best way is to use the computer software accompanying an excellent book by Altman et al.4 called CIA, to estimate 95% Confidence Interval for difference (by recommended method due to Wilson for ‘paired’ samples) and check whether this CI includes zero as inclusion of ‘0’ indicates ‘non-significance’ (or exclusion of ‘0’ in CI indicating significance) at 5% level of significance. For these data CI is 0.0377 to 0.0953. Another method (called traditional) in CIA also yields exact SE from which Z can be estimated. This will help overcome another limitation for making “paired-unpaired” differentiation while dealing with proportions namely unavailability of computer software. However, while using CIA, the set-up has to be correct (as shown in part ‘a’ of above table). For data as in part ‘b’ of above table (incorrect layout) CI by Newcombe’s method in CIA is from -0.0131 to 0.1452 which includes zero. Sample size required for such studies is also discussed with examples in literature5. References 1. Sarmukaddam SB. Fundamentals of Biostatistics. New Delhi: Jaypee Brothers Medical Publishers Ltd.; 2006 (pages 117 & 109). 2. Parlikar V., Sarmukaddam S., Agashe M., and Weiss M. Diagnostic concordance of neurasthenia spectrum disorders in Pune, India. Soc Psychiatry Psychiatr. Epidemiol. 2007; 42: 561-572. 3. Indrayan A. and Sarmukaddam SB. MedicalBiostatistics. New York: Marcel Dekker, Inc.; 2001 (page 353). 4. Altman, DG., Machin, D., Bryant, TN., and Gardner, MS. Statistics with confidence. London: BMJ Books; 2000 (page 50). 5. Sarmukaddam SB. and Kharshikar AV. Sample Size for Pre-Measure and Post -Measure Panel Type Prevalence Study, In : Statistics in Health and Nutrition, Hyderabad: National Institute of Nutrition 1990, pp 402-406. Competing interests: None declared |
|||