Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Misleading inferences costing money and time
- Adrian Midgley (1 January 2000)
-
Conviction by Mathematical Error
- A R W Forrest (2 January 2000)
-
Mathematical errors
- Ken Norman (3 January 2000)
-
Re: Registration for Civil Court experts needed too
- Penny Mellor (6 January 2000)
-
Law Reform Essential
- John P Heptonstall (7 January 2000)
-
Use of estimate might have been correct
- Mark Friston (7 January 2000)
-
Conditional probability should be applied
- Wai-Ching Leung (7 January 2000)
-
Registration for Civil Court experts needed too
- Brian Morgan (7 January 2000)
-
Re: Conviction by Mathematical Error: expectations grate
- Ed Cooper (10 January 2000)
-
Conviction by Probability Theory
- John P Heptonstall (11 January 2000)
-
Perhaps we should own up to our relative ignorance
- Mark Browne (31 January 2000)
-
statistics of 'cot death'
- John P Heptonstall (11 May 2000)
|
|
|||
|
Adrian Midgley, GP Exeter UK
Send response to journal:
|
Probability theory and statistics are widely understood poorly, and an example which will bear upon more and more GPs is the practice of comparing Practices on costs of particular treatments, use of particular investigations and similar easily derived figures. Presenting these as a clear inference that practices at one end of the distribution are good doctors, and at the other are bad doctors (usually the cost is the most striking correlated variable) is becoming common and PCGs talk openly of applying sticks and carrots. I have yet to see, despite periodically requesting, a proper statistical analysis even to the extent of applying calculations of standard deviation, and an indication of the reliability of an assumption that a practice at one end of the distribution is actually from a different distribution (and therefore might be good or bad compared to the others, rather than reflecting well-known statistical variations) Small practices are disadvantaged by this process of course, as long as all the doctors in a practice are lumped together, since the variation of samples of doctors decreases the larger the number of doctors in the sample. (A practice with 13 parnters will be expected to be closer to the mean more often than a practice of 1 or 2 doctors) The requirements of understanding and presenting such data properly are less onerous than those of understanding the evidence in evidence- based medical practice. If a pharmaceutical company representative were to produce figures with so poor a statistical treatment in an effort to persuade GPs to adopt a particular course of action they would expect to be reported to the ABPI, and then hounded out of the industry. Should clinical governance apply to the public health doctors and pharmacists most commonly involved in providing a gloss of professional respectability to these poor-quality published data? |
|||
|
|
|||
|
A R W Forrest, Professor of Forensic Toxicology Department of Forensic Pathology, Sheffield University , Watery Street , Sheffield S3 7ES
Send response to journal:
|
Editor, Like Dr Watkins I was surprised to learn from the media that a figure of 1 in 73 million had been quoted in Court, during the trial of Sally Clark for the murder of her two children, as the probability that two cot deaths might take place in the same family. I was even more surprised not to learn of any report that this evidence had been challenged in Court. Prosecution expert evidence is now disclosed to the Defence well in advance of a criminal trial and if this figure had been quoted in a statement disclosed to the Defence, they would presumably had had an opportunity to obtain their own expert opinion on this statistic. Clearly, the issue will now, in all probability, be canvassed before the Court of Appeal. DNA evidence has certainly increased the statistical sophistication of the Courts and those who assist them. The Court of Appeal has given advice from on how DNA evidence should be presented to the jury. In the example quoted by Dr Watkins of the probability of a chance match of a DNA profile being 20 million to 1, the judge would probably have included in his summing up a phrase such as: "The decision you have to reach on all the evidence is whether you are sure it was the defendant who left the stain in which the DNA was found or whether it is possible it was one of that other small group of men, less than three in the United Kingdom, who share the same DNA characteristics". (1) Obviously the Courts, in particular the Criminal Courts and those accused before them, are entitled to the best possible expert evidence. The indicative register established by the Council for the Registration of Forensic Practitioners (CRFP), an organisation set up with Home Office assistence and approval, is about to accept its first applications for registration. In time it is likely to be the norm that persons, including those registered with the GMC, giving expert evidence before the Courts will be registered with the CRFP.(2) The CRFP should ensure at least basic competence in those who give expert evidence before the Courts, whether at the request of the Prosecution or the Defence. Whether or not it will be able to prevent occasional solecisms by an expert is a moot point. Hopefully, it will at least reduce them. A R W Forrest Professor of Forensic Toxicology Department of Forensic pathology University of Sheffield Watery Street Sheffield S3 7ES Declaration of Interest: I have been a member of the working party developing the Disciplinary Code of Practice that the Council for Registration of Forensic Practitioners will apply to its registrants. 1. R v. Doheny. Times Law Report,14 August 1996 2. http://www.rsc.org/members/forensic.htm |
|||
|
|
|||
|
Ken Norman, Chairman, Portia Campaign The Croft, Bowness-on-Solway, Carlisle CA5 5AG
Send response to journal:
|
I was delighted to read Dr Watkins' article, but his own statistics are not wholly reliable and under-estimate the situation. He says there are 344 cot deaths each year in Britain; this figure will have been obtained from the Foundation for the Study of Infant Deaths, which excludes deaths recorded as "open verdict" or murder. This also makes the frequency estimate of multiple cot death meaningless. Ms Michelle Horton, of Northfield, Birmingham, lost three children and this got an open verdict, last November. She was at high risk of being charged with smothering them, except that the first death occurred in hospital when the mother was nowhere near. It would seem that these were three cot deaths to the same parents, but they won't be recorded as such. And since most double-deaths are now regarded as murder, none of these go into Dr Watkins' statistics either. On the other hand, these cases are "iffy" and perhaps it makes his article all the more authoritative that they were ignored. The cases of Mrs Sally Clark and Mrs Donna Anthony (Yeovil), both of whom lost two children, will also have been excluded from the FSID figures, as will the 84 murder convictions obtained on evidence from Professor Roy Meadow over an 18-year period. Each of these, at autopsy, had been recorded as cot death, but Professor Meadow overturned these findings. No-one can possibly know how many Sudden Infant Death Syndrome cases there are, for there is no actual definition and probably millions of differing causes; some still to be discovered. Not all SIDS cases involve risk to subsequent infants to the same parents, and therefore in one or two particular categories, it may very well be that the Mendelian principle is accurate. one child in four will be "normal"; two will become carriers, like their parents; the fourth will receive a "double dose" of the defect and will die (if the genetic flaw is that severe). I have written a book, "The Lynch-Mob Syndrome," concerning the convictions of Louise Woodward, Sally Clark, Helen Stacey, Manjit Basuta. Marie Noe, Raxine Robinson, Donna Anthony, and others. In all these cases the medical evidence was highly contentious and the verdicts were reached (in some cases by 10-2 majority) by jurors who were probably unable to evaluate the evidence. With the majority verdicts, quite possibly the dissenters were the only ones able to realise that doubt existed. In cases like this, every juror should have a medical degree. A copy of the book will be sent free to anyone interested. Ken Norman, The Croft, West Common, Bowness-on-Solway, Carlisle CA5 5AG |
|||
|
|
|||
|
Penny Mellor, Child Advocate Home
Send response to journal:
|
The most worrying aspect about expert testimony given in Civil Child Care proceedings is the secrecy that shrouds them. There is never the opportunity for people outside the fields of expertise on the issues that brought everyone to the court to independently asses statistical evidence that is given. If a professional stands up in court and states that as an expert witness in let's say child sexual abuse, quotes statistics to shore up an allegation and to prove the likelyhood of further harm, it is currently not within the court's remit or inclination to bring in experts on the calculation of those statistics. This means that deeply flawed evidence remains unchallenged leaving the courts with no alternative but to remove children from the care of their parents, worse still, the "expert" remains a "flawed" expert and continues to perpetuate those statistics in courtrooms up and down the land, where it is written into case law and used as a precedent to remove further children. If ever there was an example of why the secrecy of Family Courts are unsafe, this is it. We now need clear defined outlines of an "expert" witness. A paediatrician should not be able to be an expert in statistics unless it can be proven to the courts that they have training and peer reviewed publications in this area. Given the logic applied in Sally Clarks case, can someone answer me this? Statistically speaking a certain type of paedophile gets a job where they have access to vulnerable children, gain the trust of the parents or carers and are respected members of the community. Should I suspect every paediatrician then and if an allegation is made do I then preclude guilt because statistically the markers push the odds up that they are? Of course not. |
|||
|
|
|||
|
John P Heptonstall, Director of The Morley Acupuncture Clinic and Complementary Therapy Centre West Yorkshire
Send response to journal:
|
It is inconceivable that a medical practitioner, without qualifications and training in Statistics, could be allowed to quote as 'expert' evidence to a Court of Law statistical probability unless he/she is quoting detailed and peer reviewed conclusions of a qualified statistician. I am quite sure the spirit, if not the letter, of the law requires this. In addition to this obvious 'travesty of justice', which appears to be all too common witness responses above, we are still experiencing the slow and deliberate torture of often bereaved parents who are brought before the courts accused of murderous child abuse where the possible cause of 'iatrogenic abuse' due to vaccination ADRs is avoided in medical testimony as the medical community appear to remain in total denial of this effect. John H. I have no competing interests other than a wish to see justice done in this country. I have appeared as an 'expert witness' many times and have prepared and executed case files for courts in which expert witnesses were required; I would never have entertained one who did not have relevant qualifications or experience in the particular field as I would class such evidence as his/her opinion. |
|||
|
|
|||
|
Mark Friston, Junior Counsel 6 Pump Court, Temple, London
Send response to journal:
|
Watkins in his editorial of 1 January 2000 comments on the conviction of a woman in the Crown Court sitting at Chester for the murder of her two infant children. Although I have had no involvement with the case, I understand that it was said the chance of two babies from one family dying of sudden infant death syndrome was one in 73 million. Watkins objects to the use of this epidemiological estimate on several bases. It would be inappropriate for a lawyer to comment on many of the more detailed points Watkins makes, but his major premise is, in my opinion, flawed. Watkins says: "Suspicion was drawn to Sally Clarke by the occurrence of two deaths so the probabilities should not have been squared. The odds of 1 in 73 million shrink to 1 in 8500." He states that the use of the squared figure was erroneous. His assertion is open to criticism. The squared figures could be properly used because the Defendant asserted that the deaths were innocently caused. That is, she was saying that she had not done anything wrong and that she was just a normal member of the public who had the misfortune to be affected by two rare events. By analogy, if only one child had died, it would have been appropriate to put in evidence that the chances of this happening as a result of sudden infant death syndrome are 1 in 8500: not to do so might leave the jury with the impression that sudden infant death syndrome was a common disorder. The fact that there were two deaths means that there is a confusion of possible figures, but, likewise, it would be proper to refer to the chances of two deaths occurring in a single, randomly selected family. |
|||
|
|
|||
|
Wai-Ching Leung, Senior Registrar in Public Health Medicine Epidemiology and Public Health, Newcastle General Hospital, NE4 6BE
Send response to journal:
|
It is vital to state the precise conditional and outcome events as well as the underlying assumptions in quoting probability figures. All the figures quoted by Watkins(1) in his discussion about the use of probability in the conviction of Sally Clark could be correct:- 1) The probability that two consecutive infants died of cot death in a given social class I family is 1 in 73 million, assuming no increase in recurrence rate. 2) The probability that two consecutive infants died of cot death in a given social class I family, given that the first infant had already died, is 1 in 8,500, again assuming no increase in recurrence rate.. 3) The probability that two consecutive infants died of cot death in a family is 1 in 2.75 million, again assuming no increase in recurrence rate. However, none of these are relevant to the jury. The relevant question is:- what is the probability that the causes of deaths of the infants in this family were cot deaths, given that these deaths have occurred? Surely, the answer would depend on the comparative likelihood of the two possible causes of death – cot death and homicide. For example, if we can rule out homicide completely (e.g. by several other witnesses), the probability that the causes were cot death is 1 (certainty). To illustrate the correct approach, take the incidence for Munchausen syndrome by proxy for children under 1 year of age given by McClure et al(2) of 2.8 per 100,000 (i.e. 1 in 35,714). This would greatly overestimate the probability for two completed homicides. As this is an average figure for all social classes, we should take the incidence figure for 2 consecutive cot deaths as 1 in 2.75 million (i.e. average for all social classes). Using simple rules of conditional probability, the probability that the causes of death were cot deaths = (1 / 2,750,000) / (1 / 2,750,000 million + 1 / 35714) = 0.013 (i.e. 1 in 77). The jury would be much less impressed by this figure. Furthermore, this is likely to be an underestimate, as an overestimate of the probability for two completed homicides was used in the denominator. Reference 1) Watkins SJ. Conviction by mathematical error? BMJ 2000; 320: 2- 3. 2) McClure RJ; Davis PM; Meadow SR; Sibert JR. Epidemiology of Munchausen syndrome by proxy, non-accidental poisoning, and non-accidental suffocation. Arch Dis Child, 1996, 75:1, 57-61 |
|||
|
|
|||
|
Brian Morgan, Freelance Journalist Cardiff
Send response to journal:
|
Will the register of forensic experts apply to High Court Civil and Family proceedings as well as the Criminal Court? If it is a Home Office matter rather than the Lord Chancellor's and/or the Department of Health's this suggests that there is no intention presently for medical experts giving evidence in child abuse cases to be subjected to any screening as to their expert status. It is a matter of grave concern to those like myself who have researched the conduct of medical experts in the Family Court that they are almost wholly beyond scrutiny. Unlike in the recent murder trial which resulted in an editorial and the current responses their evidence is given in secret. With the exception of the rare authorised transcript of proceedings which might have alerted them, responsible experts, such as those who have responded to your editorial can have no idea of how many abuses of the rules that the judges themselves laid down have taken place. But evidence has emerged in the form of leaked documents, reports and the testimony of the families involved. Experts have gone far beyond their legitimate field of experience or expertise, and in fact have made use of the Family Courts and judgments depending on their opinions to bolster their developing expert status. The courts have been mis-used to validate research objectives too. Readers will see, if they take to the trouble to examine the papers of certain experts, how many times their theories or research have been 'proved' and justified by a judicial decision in their favour. This is not what the courts are for, it is not what medicine is about, and it is not the way to conduct medical research. How research papers with outcomes validated only by a court judgement ever came to pass peer review and be published defeats me. . |
|||
|
|
|||
|
Ed Cooper, Consultant pediatrician Newham, East London
Send response to journal:
|
Null Hypothesis: The association of any 2 cases of sudden unexpected infant death is entirely due to chance. Let us accept the given mathematical expectation of 1 in 73 million for this conjunction without disputing its derivation. Finding from Observation: The actual incidence ratio of familial conjunction in the population far exceeds the above figure. Deduction: The null hypothesis must be rejected. The familial incidence of cases in the population is exceedingly unlikely to be explained entirely by random association. Well, that’s it for the null hypothesis. It is now dead. It is NOT the actual background expectation of familial sudden infant death. An expert witness does not bring the null hypothesis with him into the witness box as his estimate of the background incidence, and it is certainly misleading for him to tell the jury that it is the background expectation (ie prior probability of occurrence). Which medical person would have actually believed in this null hypothesis? Almost none. The null hypothesis is a notion created for purposes of significance calculation. In most clinical research or epidemiological situations the null hypothesis is an absurdity, but it gives us a standard to test a composite alternative hypothesis against. Within this composite there may be an indefinite number of competing hypotheses - the real-life situation with sudden infant death - and we will not know which single one is most likely until we either compare the probabilistic degree to which each differs from the notional null hypothesis or, much better, compare each to the others estimating their comparative effects. Bayesian statistics, which does not use a null hypothesis, may be mathematically equivalent to the Neyman or Fisher frequentist methods which do, but clinicians are prone to confuse mathematical expectation, a defined notion, with actual expectation. This happened in this case. This is reason enough to prefer the teaching of bayesian statistics to medics, as it works more like the intuitive methods we use in diagnosis and clinical judgement. |
|||
|
|
|||
|
John P Heptonstall, Director of Morley Acupuncture Clinic and Complementary Therapy Centre West Yorkshire
Send response to journal:
|
Much has been said about the niceties of probability theory and its application in the case of Sally Clark. Conviction by probability theory, with or without mathematical error, appears to set a dangerous precedent. My law studies recognised conviction for a serious crime as usually requiring proof beyond reasonable doubt of commission of an illegal act, with the intention of doing so. Probability (statistical) considerations surely introduce additional (and possibly unreasonable) problems into a defense case - not only must the latter rebut the evidence of wrongdoing but also has to find additional facts to rebut statistical evidence which probably, due to its often generalsised nature, fails to consider variables specific to the defendant. Is that fair? Watkins draws attention to this when questioning the prosecution's use of the figure 1 in 73 million rather than 1 in 2.75 million because of the family's affluence. "Social class is a complex reality of interassociated circumstances - education, work, income, lifestyle, culture, contacts, residence, opportunities, social class of origin etc.". A defendant may have to rebut the use of statistics said to represent 'his/her social class' rather than the prosecution having to first prove that he/she is accurately represented by the class it quotes statistics for. Furthermore, Ken Norman disputes the statistics noting that variables such as 'open verdicts', 'double deaths treated as murder' etc. can quickly and substantially distort a 'sample' of 344 (the figure quoted for the total number of cot deaths in Britain annually). Probability may reduce the extremities of a cohort, even if all possible variables are accounted for, but it does not eliminate them - therefore they still exist as reality. Watkins illustrates this clearly through EMERY 1986 and WOLKIND et al 1993. Is probability theory to become the new 'lax person's method of securing a conviction' without obtaining substantial direct, material or circumstantial evidence for commission of a serious offence? Will we see 'death row' populated increasingly by innocent victims of statistical error? John H. |
|||
|
|
|||
|
Mark Browne
Send response to journal:
|
Dear Editor, I was very impressed with the leading article by Stephen Watkins(1) in early January concerning the statistics involved with probability theory, and with his elegant explanations. I awaited eagerly the rush of correspondence from eminent specialists keen to defend their own proclamations in legal cases, or at least to disagree with his mathematics. When nothing appeared by your 22 January edition, I was moved to write before your 4-week deadline expired. I must admit that I didn’t read Altman and Bland’s series of Statistics Notes as it didn’t appear to carry the little blue “GP” symbol that you apply to those easier to understand articles suitable for general practitioners, and I never understood statistics anyway at school or University. Presumably, though, our more esteemed specialist colleagues either read them, or knew enough statistics to get by. Perhaps they are still chewing their pencils over the workings out concerning Watkins’ leader? I doubt that Sally Clark will be reassured by our apparent lack of understanding of probability theory. It may, therefore, be more prudent, when asked in court to give an opinion on probability, to own up to our relative ignorance and to allow the legal profession to employ expert witnesses who do? Mark Browne (1)Watkins SJ. Conviction by mathematical error?. BMJ 2000;320:2-3. (1 January) |
|||
|
|
|||
|
John P Heptonstall, Director of The Morley Acupuncture Clinic and Complementary Therapy Centre West Yorkshire
Send response to journal:
|
Editor The author's reference to the 'Sally Clark' case highlighted the uneasiness felt by many about a conviction which appears to have been gained by 'mathematical error'. 73 million to one quoted with respect to two cot deaths in one family may have been an "expert's" point of view, but the 'expert' does not appear to have quoted the statistical probability of a child (or a 'second child's death') occurring within 5 hours of vaccination, as in that case, from which comparison can be made. Surely there must be some data on such eventualities, after all we are constantly being made aware by 'vaccine experts' that such incidences are rare, or 'coincidental'; such claims cannot be justified unless statistics are available. If a child dies within a few hours of vaccination is it inevitable that a sole innocent parent or carer may be convicted in the absence of evidence quantifying 'vaccine injury' events? This is disturbing news as child deaths by vaccination is an 'acceptable risk' as far as governments are concerned, so the same governments must ensure that parents and guardians receive protection from the law should they be caught in this terrible situation. Unless the production of such statistics is made a legal requirement of vaccine suppliers or the Committee for Safety of Medicines at these trials, one does not expect purveyors of vaccines to offer such information to defence counsels and justice may not be done. Until such data is available defendants must fight with both hands tied due to government neglecting its duty of care to the public. Regards John H. |
|||