Papers

Comparability and validity of two clinical scores in the early differential diagnosis of acute stroke

BMJ 1994; 308 doi: http://dx.doi.org/10.1136/bmj.308.6945.1674 (Published 25 June 1994) Cite this as: BMJ 1994;308:1674
  1. M G Celani,
  2. E Righetti,
  3. R Migliacci,
  4. M Zampolini,
  5. L Antoniutti,
  6. C F Grandi,
  7. S Ricci
  1. Clinica Neurologica, Via E Dal Pozzo 50, 06126 Perugia, Italy
  2. Divisione Medicina, Ospedale Civile, Cortona, Italy
  3. Divisione Neurologia, Ospedale Maggiore, Trieste, Italy
  1. Correspondence to: Dr Celani.
  • Accepted 7 April 1994

Abstract

Objective: To compare two available clinical scores for the differential diagnosis of cerebral ischaemia and haemorrhage in acute stroke patients.

Design: Prospective, multicentre study of acute stroke patients evaluated with computed tomography and Allen and Siriraj scores; the scores were tested for comparability (kappa statistic) and validity (sensitivity, specificity, positive and negative predictive values, diagnostic gain). The effect of a policy of using Allen and Siriraj scores to determine pathological type of stroke before computed tomography was calculated.

Setting: Three hospitals in Italy, all participating in the international stroke trial, with different access facilities to computed tomography.

Subjects: 231 consecutive patients who were screened in the three hospitals for possible inclusion in the international stroke trial from 1 November 1991 to 31 May 1993.

Results: The prevalence of haemorrhage (diagnosed with computed tomography) was 14.7% (95% confidence interval 10.1% to 19.3%). Allen scores were “uncertain” in 44 cases and Siriraj scores in 38 cases; in the 164 cases with both the scores in the range of “certainty” kappa was 0.72. Sensitivity, specificity, positive and negative predictive values, and diagnostic gain for haemorrhage were 0.38, 0.98, 0.71, 0.91, and 0.58 for Allen scores and 0.61, 0.94, 0.63, 0.93, and 0.48 for Siriraj scores; positive predictive values for infarction were 91% for Allen scores and 93% for Siriraj scores. According to these data, of 1000 patients with acute stroke, 680 would be correctly and 70 wrongly diagnosed as “ischaemic” with the Allen score; the figures would be 671 and 48 with Siriraj score.

Conclusion: When computed tomography is not immediately available and the clinician wishes to start antithrombotic treatment (or randomise patients in a clinical trial), the Siriraj score (and possibly the Allen score) can be useful to identify patients at low risk of intracerebral haemorrhage.

Clinical implications

  • Clinical implications

  • It is not possible for all stroke patients to have a computed tomography scan immediately after admission to differentiate between haemorrhage and ischaemia

  • The Allen and Siriraj clinical scores have been proposed to help clinicians in making decisions while waiting for results of computed tomography

  • In this study, the accuracy of both scores when compared with computed tomography was 89%

  • The Siriraj score is simpler, can be used immediately after the stroke, and has a 93% positive predictive value for ischaemia

  • When clinicans wish to start antithrombotic treatment (aspirin or subcutaneous heparin) while waiting for the scan results, they can rely on the Siriraj score

Introduction

Haemorrhagic and ischaemic stroke cannot be distinguished clinically with a simple clinical evaluation, and it is virtually impossible for all stroke patients to have a computed tomography scan immediately after admission. Thus, in small district hospitals as well as in large university centres a weighted clinical score may offer some advantages to physicians who are involved in stroke management and need to distinguish between haemorrhage and ischaemia for the purpose of treatment. Two such scores are currently available.1,2 The Allen score has been validated in different European settings,3,4 and has a reasonably good (90%) accuracy when the suggested cutoffs (<4 for ischaemia and >24 for haemorrhage) are used; however, it requires several historical and clinical details to be registered (see Appendix) and cannot be used until 24 hours after the stroke. The Siriraj score is much easier to determine, and can be used immediately after the stroke; however, it has been tested only in Thailand, where the pretest probability of haemorrhage is higher than in Europe; as the positive predictive value depends on prevalence, the score needs to be validated in Europe to check the predictive values. We compared Allen and Siriraj scores in Italy.

Methods

Three centres were involved in this study: the university department of neurology in Perugia (24 hour access to computed tomography), the division of neurology in Trieste (daytime access to computed tomography), and the division of medicine in Cortona (no computed tomography in the hospital; patients sent by ambulance 30-50 km for computed tomography). All three hospitals are participating in the international stroke trial; physicians in these hospitals need to differentiate between haemorrhage and ischaemia to include patients with ischaemic stroke in that trial. From 1 November 1991 to 31 May 1993 all patients screened for potential inclusion in the international stroke trial (those with stroke onset within 48 hours) were evaluated with Allen and Siriraj scores, and submitted to computed tomography scan within 48 hours of inclusion in this study. Before being scanned, each patient was evaluated by one of the authors, who completed a form containing the variables of each score (answering yes or no to each question and reporting blood pressure) but did not calculate the scores (see Appendix). These forms, along with the computed tomography report, were sent to the neurology department in Perugia, where calculations and comparisons with the results of computed tomography were done only after all forms and reports had been received.

Several analyses were carried out. The scores were compared in terms of “certain” results (that is, percentages of cases in which the scores predicted ischaemia or haemorrhage, according to the cutoffs suggested in the original papers1,2; a result was considered to be certain when the Allen score was <4 or >24 or the Siriraj score was <-1 or >1. The kappa statistic was determined for agreement between the two scores (all cases and “certain” cases). The two scores were compared with the results of computed tomography, and sensitivity, specificity, positive predictive value, negative predictive value, and diagnostic gain (the difference between post-test and pretest probability of disease) were calculated. We used the kappa statistics program5 for comparability tests, EPISTAT for McNemar's test, CIA for 95% confidence intervals,6 and an ad hoc program, based on widely described calculations,7 for the validation tests.

Results

In 231 cases, the prevalence of haemorrhage (diagnosed with computed tomography) was 14.7% (95% confidence interval 10.1% to 19.3%). The overall comparability of the Allen and Siriraj scores was fair (k=0.4) (table I). The Allen score was uncertain in 44 cases, and the Siriraj score was uncertain in 38 (P=0.49; McNemar's test); however, the kappa statistic showed a worse comparability between the two scores in terms of certain results (k=0.23) (table II). When only the results that were within the diagnostic range with both the scores (164 cases) were considered, the agreement in diagnosing infarction and haemorrhage was high (k=0.72) (table III). When computed tomography scan results are used as a standard (tables IV and V), the diagnostic gain for detecting haemorrhage was 0.58 with the Allen score and 0.48 with the Siriraj score; however, for the detection of infarction, besides a poor diagnostic gain (0.05 and 0.08), the positive predictive value was 91% for the Allen score and 93% for the Siriraj score. In the 157 patients in whom the two scores were both certain and in agreement, when the combined result was compared with results of computed tomography, accuracy was slightly better (0.92), and the diagnostic gain was 0.6.

TABLE I

Comparison of Allen and Siriraj scores

View this table:
TABLE II

Certain and uncertain results with Allen and Siriraj scores

View this table:
TABLE III

Diagnostic agreement of Allen and Siriraj scores

View this table:
TABLE IV

Comparison of Allen score with results of computed tomogra

View this table:
TABLE V

Comparison of Siriraj score with results of computed tomography

View this table:

Discussion

Our study shows that the Allen and Siriraj scores do not diagnose the same cases as being certain infarction or certain haemorrhage. The scores should be considered comparable only when both show certainty and point to the same side of the scale. Using both scores may slightly increase the accuracy, but the Allen score can be calculated only after 24 hours, so the combined use should be restricted to few patients; furthermore, the variables used in the Siriraj score are all present in the Allen score, making an additional effect unlikely. Thus, this time consuming double procedure does not seem clinically sound. Different cutoffs might be used to increase the accuracy, or different variables might be added to obtain a better accuracy; there is some research in this direction but results have not been published, so we must rely on the published and validated cutoffs.

When the doctor needs to ascertain ischaemic stroke (to immediately start antithrombotic treatment in a clinical trial), the positive predictive value of both the scores is good, and given the characteristics of the calculations, the Siriraj score could be used at bedside. When the problem is to detect a haemorrhage (to ask for a neurosurgical opinion), the Allen score is probably better, as there is a low pretest probability of this disease. We have already suggested practical applications of these scores.4 In addition, when computed tomography is not immediately available and the doctors wish to start antithrombotic therapy (or randomise patients in the international stroke trial,8) if the Siriraj score (and possibly the Allen score) is low enough to convince them of the presence of an ischaemic stroke, one or two doses of aspirin or subcutaneous heparin can be given, or the patient randomised, while waiting for computed tomography. These drugs can be withdrawn immediately if the scan shows a small, deep haemorrhage, and a few patients with haemorrhage entered in a trial of ischaemic stroke treatment. In this way the sample size of a clinical trial can be increased and - more importantly - treatments tested in hospitals, where stroke patients are acutely admitted, and where it is rather difficult to always obtain a computed tomograph quickly. Furthermore, it would be possible to test the advantage of early treatment of cerebral ischaemia as opposed to the risk of giving a few doses of antithrombotic drugs to a patient with a small haemorrhage.

Appendix

Calculation of Siriraj score

View this table:

Calculation of Allen score

View this table:

References

View Abstract