Diagnosis of obstructive coronary artery disease using computed tomography angiography in patients with stable chest pain depending on clinical probability and in clinically important subgroups: meta-analysis of individual patient dataBMJ 2019; 365 doi: https://doi.org/10.1136/bmj.l1945 (Published 12 June 2019) Cite this as: BMJ 2019;365:l1945
- Robert Haase, resident in cardiovascular surgery1,
- Peter Schlattmann, professor of statistics2,
- Pascal Gueret, cardiologist3,
- Daniele Andreini, associate professor of cardiovascular disease4,
- Gianluca Pontone, cardiologist5,
- Hatem Alkadhi, professor of radiology6,
- Jörg Hausleiter, professor of cardiology7,
- Mario J Garcia, professor of cardiology8,
- Sebastian Leschka, professor of radiology9,
- Willem B Meijboom, imaging cardiologist10,
- Elke Zimmermann, radiology registrar1,
- Bernhard Gerber, professor of cardiology11,
- U Joseph Schoepf, professor of radiology, cardiology, and paediatrics12,
- Abbas A Shabestari, associate professor of radiology13,
- Bjarne L Nørgaard, senior cardiologist14,
- Matthijs F L Meijs, cardiologist15,
- Akira Sato, associate professor of cardiology16,
- Kristian A Ovrehus, speciality registrar in cardiology17,
- Axel C P Diederichsen, professor in cardiac imaging18,
- Shona M M Jenkins, consultant cardiologist18,
- Juhani Knuuti, professor19,
- Ashraf Hamdan, senior cardiologist20,
- Bjørn A Halvorsen, consultant cardiologist21,
- Vladimir Mendoza-Rodriguez, cardiologist22,
- Carlos E Rochitte, cardiologist23,
- Johannes Rixe, head of cardiac imaging24,
- Yung Liang Wan, professor of radiology25,
- Christoph Langer, professor of cardiology26,
- Nuno Bettencourt, cardiologist27,
- Eugenio Martuscelli, associate professor of cardiology28,
- Said Ghostine, cardiologist29,
- Ronny R Buechel, cardiologist30,
- Konstantin Nikolaou, professor of radiology31,
- Hans Mickley, professor of cardiology32,
- Lin Yang, radiologist33,
- Zhaqoi Zhang, professor of radiology33,
- Marcus Y Chen, assistant clinical investigator34,
- David A Halon, associate professor of cardiology35,
- Matthias Rief, senior radiology registrar1,
- Kai Sun, professor of radiology36,
- Beatrice Hirt-Moch, cardiologist31,
- Hiroyuki Niinuma, senior cardiology consultant37,
- Roy P Marcus, radiologist32,
- Simone Muraglia, interventional cardiologist38,
- Réda Jakamy, interventional cardiologist39,
- Benjamin J Chow, professor of medicine (cardiology and nuclear medicine) and radiology40,
- Philipp A Kaufmann, professor of nuclear medicine31,
- Jean-Claude Tardif, professor of medicine41,
- Cesar Nomura, radiologist42,
- Klaus F Kofoed, associate professor of cardiology43,
- Jean-Pierre Laissy, professor of radiology44,
- Armin Arbab-Zadeh, associate professor of medicine45,
- Kakuya Kitagawa, associate professor of radiology46,
- Roger Laham, director of structural heart center47,
- Masahiro Jinzaki, professor of radiology48,
- John Hoe, consultant radiologist49,
- Frank J Rybicki, professor and chair of radiology50,
- Arthur Scholte, consultant cardiologist51,
- Narinder Paul, professor of radiology52,
- Swee Y Tan, associate professor of cardiology53,
- Kunihiro Yoshioka, professor of radiology54,
- Robert Röhle, statistician1,
- Georg M Schuetz, resident in nuclear medicine1,
- Sabine Schueler, resident in general medicine1,
- Maria H Coenen, resident in psychosomatic medicine1,
- Viktoria Wieske, resident in radiology1,
- Stephan Achenbach, professor of cardiology55,
- Matthew J Budoff, professor of medicine56,
- Michael Laule, interventional cardiologist1,
- David E Newby, BHF Duke of Edinburgh professor of cardiology57,
- Marc Dewey, Heisenberg professor of radiology1
- on behalf of the COME-CCT Consortium
- 1Department of Radiology, Charité - Universitätsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany
- 2Institute of Medical Statistics, Computer Sciences and Data Science, University Hospital of Friedrich Schiller University Jena, Jena, Germany
- 3Department of Cardiology, Henri Mondor Hospital, University Paris Est Créteil, Créteil, France
- 4Department of Cardiology and Radiology, Centro Cardiologico Monzino IRCCS, University of Milan, Milan, Italy
- 5Centro Cardiologico Monzino, IRCCS, Milan, Italy
- 6Department of Radiology, University Hospital Zurich, Zurich, Switzerland
- 7Medizinische Klinik und Poliklinik I, Ludwig-Maximilians-Universität München, Munich, Germany
- 8Department of Cardiology, Montefiore, University Hospital for the Albert Einstein College of Medicine, NY, USA
- 9Department of Radiology, Kantonsspital St Gallen, St Gallen, Switzerland
- 10Department of Cardiology, Erasmus University Medical Centre, Rotterdam, Netherlands
- 11Department of Cardiology, Clinique Universitaire St Luc, Institut de Recherche Clinique et Expérimentale, Brussels, Belgium
- 12Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
- 13Modarres Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
- 14Department of Cardiology, Aarhus Universtity Hostipal, Aarhus, Denmark
- 15Department of Cardiology, University Medical Centre Utrecht, Utrecht, Netherlands
- 16Cardiovascular Division, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan
- 17Department of Cardiology, Odense University Hospital, Odense, Denmark
- 18Department of Cardiology, Glasgow Royal Infirmary and Stobhill Hospital, Glasgow, UK
- 19Turku University Hospital and University of Turku, Turku, Finland
- 20Department of Cardiovascular Imaging, Department of Cardiology, Rabin Medical Center, Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
- 21Medical Department, Ostfold Hospital Trust, Grålum, Norway
- 22Department of Cardiology, National Institute of Cardiology and Cardiovascular Surgery, Havana, Cuba
- 23Heart Institute, InCor, University of São Paulo Medical School, São Paulo, Brazil
- 24Department of Cardiology, Kerckhoff Heart Centre, Bad Nauheim, Germany
- 25Medical Imaging and Radiological Sciences, College of Medicine, Chang Gung University, Chang Gung Memorial Hospital at Linkou, Taoyaun City, Taiwan
- 26Heart and Diabetes Center NRW in Bad Oeynhausen, University Clinic of the Ruhr-University Bochum, Bochum, Germany
- 27Department of Cardiology, Centro Hospitalar de Vila Nova de Gaia, Vila Nova de Gaia, Portugal
- 28Department of Internal Medicine, University of Rome Tor Vergata, Rome, Italy
- 29Department of Cardiology, Centre Chirurgical Marie Lannelongue, Le Plessis Robinson, France
- 30Department of Nuclear Medicine, University Hospital Zurich, Zurich, Switzerland
- 31Department of Diagnostic and Interventional Radiology, University Hospital of Tübingen, Tübingen, Germany
- 32Department of Cardiology, Odense University Hospital, Odense, Denmark
- 33Department of Radiology, Beijing Anzhen Hospital, Beijing, China
- 34National Heart and Blood Institute, National Institutes of Health, Bethesda, MD, USA
- 35Cardiovascular Clinical Research Unit, Lady Davis Carmel Medical Center, Haifa, Israel
- 36Department of Radiology, Baotou Central Hospital, Inner Mongolia Province, China
- 37St Luke’s International Hospital, Tokyo, Japan
- 38Department of Cardiology, S Chiara Hospital, Trento, Italy
- 39Department of Cardiology, University Hospital Pitié-Salpêtrière, Paris, France
- 40University of Ottawa, Heart Institute, Ottawa, Ontario, Canada
- 41Montreal Heart Institute, Université de Montréal, Montréal, Canada
- 42Albert Einstein Hospital, São Paulo, Brazil
- 43The Heart Centre, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
- 44Department of Diagnostic Imaging and Interventional Radiology, Bichat University Hospital, Paris, France
- 45Division of Cardiology, Johns Hopkins Hospital, Johns Hopkins University, Baltimore, MD, USA
- 46Mie University Hospital, Tsu, Japan
- 47BIDMC/Harvard Medical School, Department of Cardiology, Beth Israel Deaconess Medical Center, Harvard University, Boston, MA, USA
- 48Department of Radiology, Keio University Hospital, Tokyo, Japan
- 49Department of Radiology, Mount Elizabeth Hospital, Singapore
- 50Department of Radiology, University of Ottawa, Ottawa, Ontario, Canada
- 51Department of Cardiology, Leiden University Medical Centre, Leiden, Netherlands
- 52Department of Medical Imaging, Western University, London, Ontario, Canada
- 53National Heart Centre, Singapore, Singapore
- 54Iwate Medical University, Morioka, Japan
- 55Department of Cardiology, Friedrich-Alexander University Erlangen-Nuremberg, Erlangen, Germany
- 56University of California Los Angeles, Los Angeles, CA, USA
- 57British Heart Foundation, University of Edinburgh, Edinburgh, UK
- Correspondence to: M Dewey
- Accepted 15 April 2019
Objective To determine whether coronary computed tomography angiography (CTA) should be performed in patients with any clinical probability of coronary artery disease (CAD), and whether the diagnostic performance differs between subgroups of patients.
Design Prospectively designed meta-analysis of individual patient data from prospective diagnostic accuracy studies.
Data sources Medline, Embase, and Web of Science for published studies. Unpublished studies were identified via direct contact with participating investigators.
Eligibility criteria for selecting studies Prospective diagnostic accuracy studies that compared coronary CTA with coronary angiography as the reference standard, using at least a 50% diameter reduction as a cutoff value for obstructive CAD. All patients needed to have a clinical indication for coronary angiography due to suspected CAD, and both tests had to be performed in all patients. Results had to be provided using 2×2 or 3×2 cross tabulations for the comparison of CTA with coronary angiography. Primary outcomes were the positive and negative predictive values of CTA as a function of clinical pretest probability of obstructive CAD, analysed by a generalised linear mixed model; calculations were performed including and excluding non-diagnostic CTA results. The no-treat/treat threshold model was used to determine the range of appropriate pretest probabilities for CTA. The threshold model was based on obtained post-test probabilities of less than 15% in case of negative CTA and above 50% in case of positive CTA. Sex, angina pectoris type, age, and number of computed tomography detector rows were used as clinical variables to analyse the diagnostic performance in relevant subgroups.
Results Individual patient data from 5332 patients from 65 prospective diagnostic accuracy studies were retrieved. For a pretest probability range of 7-67%, the treat threshold of more than 50% and the no-treat threshold of less than 15% post-test probability were obtained using CTA. At a pretest probability of 7%, the positive predictive value of CTA was 50.9% (95% confidence interval 43.3% to 57.7%) and the negative predictive value of CTA was 97.8% (96.4% to 98.7%); corresponding values at a pretest probability of 67% were 82.7% (78.3% to 86.2%) and 85.0% (80.2% to 88.9%), respectively. The overall sensitivity of CTA was 95.2% (92.6% to 96.9%) and the specificity was 79.2% (74.9% to 82.9%). CTA using more than 64 detector rows was associated with a higher empirical sensitivity than CTA using up to 64 rows (93.4% v 86.5%, P=0.002) and specificity (84.4% v 72.6%, P<0.001). The area under the receiver-operating-characteristic curve for CTA was 0.897 (0.889 to 0.906), and the diagnostic performance of CTA was slightly lower in women than in with men (area under the curve 0.874 (0.858 to 0.890) v 0.907 (0.897 to 0.916), P<0.001). The diagnostic performance of CTA was slightly lower in patients older than 75 (0.864 (0.834 to 0.894), P=0.018 v all other age groups) and was not significantly influenced by angina pectoris type (typical angina 0.895 (0.873 to 0.917), atypical angina 0.898 (0.884 to 0.913), non-anginal chest pain 0.884 (0.870 to 0.899), other chest discomfort 0.915 (0.897 to 0.934)).
Conclusions In a no-treat/treat threshold model, the diagnosis of obstructive CAD using coronary CTA in patients with stable chest pain was most accurate when the clinical pretest probability was between 7% and 67%. Performance of CTA was not influenced by the angina pectoris type and was slightly higher in men and lower in older patients.
Systematic review registration PROSPERO CRD42012002780.
It is currently unclear in which subgroups of patients with suspected coronary artery disease (CAD) computed tomography angiography (CTA) has highest diagnostic clinical performance. Current guidelines recommend choosing the type of first line imaging test by taking the pretest probability of CAD into account, because it substantially affects diagnostic accuracy.123 According to the most recent recommendation of the National Institute for Health and Care Excellence,4 coronary CTA should be the primary imaging test in patients with suspected CAD and possible angina, while the guidelines of the European Society of Cardiology on the management of CAD recommend considering CTA only in patients with a CAD pretest probability of 15-50%.34 Moreover, little is known about CTA’s diagnostic performance in clinically important patient subgroups such as sex, age, and angina pectoris type and its association with the estimated clinical pretest probability of CAD.
Optimising the use of diagnostic imaging tests in patients with suspected CAD is crucial, given that about two thirds of invasive coronary angiograms performed in Europe and the United States show no evidence of obstructive CAD and increasing use of cardiac imaging tests poses a burden on healthcare costs.56 CTA has the potential to reliably exclude obstructive CAD,78 while halving the events of coronary heart disease after five years of follow-up9 and improving the diagnostic yield of coronary angiography.810 Its implementation as a first line diagnostic imaging test in patients with suspected CAD remains controversial. Since available diagnostic accuracy studies of CTA are moderate in size, data pooling can provide a more accurate assessment of its diagnostic performance. Individual patient data allow researchers to evaluate clinically important subgroups and individually estimate the pretest probability and to determine its effect on the diagnostic test performance of CTA. With the rationale to define the clinical context and clinical probability in which CTA has highest discriminative ability to diagnose or rule out CAD, we formed the COME-CCT (Collaborative Meta-Analysis of Cardiac CT) Consortium to pool patient level data from diagnostic accuracy studies of CTA enrolling patients with a clinical indication for coronary angiography as the reference standard for angiographic CAD.11 This work will help clinicians identify those patients with stable chest pain for whom CTA is most suitable.
Study design and main objectives
COME-CCT is a collaborative meta-analysis using individual patient data (IPD) to summarise the published and unpublished evidence on the diagnostic performance of cardiac CTA, and the protocol has been published.11 The main objective was to assess the influence of the clinical pretest probability of CAD on the diagnostic accuracy of cardiac CTA in order to define CTA’s clinical discriminative ability for diagnosing or ruling out CAD depending on clinical risk. Therefore, we used a no-treat/treat threshold approach to define the pretest probability range for which CTA has highest diagnostic value and, vice versa, for which CTA is not appropriate, to better decide which patients to offer the test to.12 Positive and negative predictive values (PPVs and NPVs) were chosen as primary outcome measures as a function of pretest probability.13 Finally, the influence of sex, age, angina pectoris type, and number of CT detector rows on the diagnostic performance of coronary CTA was analysed in this primary outcomes publication. We used the PRISMA (preferred reporting items for systematic reviews and meta-analyses) statement for IPD systematic reviews for reporting this collaborative meta-analysis (checklist in web appendix 1).14 COME-CCT was designed in a multicentric and multicontinental fashion according to a prespecified study protocol,11 and the COME-CCT IPD meta-analysis was registered at PROSPERO.
Eligibility criteria and study selection
COME-CCT was designed as a prospective meta-analysis of IPD from prospective diagnostic accuracy studies comparing coronary CTA with invasive coronary angiography as the reference standard. Both tests used a diameter stenosis of at least 50% as the cutoff value to define angiographically obstructive CAD.
Eligible patients needed to have a clinical indication for coronary angiography due to suspected CAD because of stable chest pain, and both tests had to be performed in all patients to avoid verification bias.15 Results had to be provided using 2×2 or 3×2 cross tabulations for the comparison of CTA with coronary angiography.16 For the calculation of clinical pretest probability using an updated version of the Diamond and Forrester method,1718 the following information had to be provided: patient age, sex, and angina pectoris type.1920 Angina pectoris was classified as typical angina, atypical angina, non-anginal chest pain, or other chest discomfort according to Diamond and Forrester. The primary analysis included all patients irrespective of whether they had diagnostic or non-diagnostic (that is, unevaluable) CTA examinations.16
The search was performed in three databases (Medline, Embase, and Web of Science; sensitive search strategy described in web appendix 2).21 The search strategy was implemented for every database by two independent investigators of the central data management team, as described in the published protocol.11
Collection and harmonisation of individual patient level data
When the search was completed, we started IPD collection and subsequent data harmonisation. As defined in the study protocol,11 we emailed corresponding authors of eligible published studies identified by the search with a cover letter detailing the objectives of the collaborative meta-analysis and a CD containing a uniform IPD collection file (web appendix 3).11 Other authors were contacted if the corresponding author did not respond. Data from unpublished studies that met the inclusion criteria were retrieved from corresponding authors of published studies.22 The completed IPD collection files were sent back to the data management team.
Data harmonisation was performed by two independent readers at the site of the central data management team, who analysed data and searched for non-plausible data, including range checks, average and median checks (v published aggregated data results), minimum and maximum checks (v aggregated data results), wrong entries, non-logical values, and other data checks. Aggregate data of studies for which IPD were not provided were collected and compared with aggregate data of studies for which IPD were provided to rule out study selection bias. Aggregate data consisted of all data necessary for 2×2 tabulations to estimate sensitivity, specificity, and PPVs and NPVs, and to perform receiver operating characteristic curves analysis. Risk of bias and applicability concerns of the included studies were assessed by two independent readers of the central data management team, who were not involved in data harmonisation, using the QUADAS-2 tool.23
Primary and secondary outcomes
The primary outcomes of interest were the PPV and NPV of coronary CTA for the presence of obstructive CAD as measures of the post-test probability of disease needed for the no-treat/treat threshold model. PPV and NPV were evaluated as a function of the pretest probability of obstructive CAD and analysed by a generalised linear mixed model meta-regression including non-diagnostic CTA results.11 To define the range of appropriate pretest probabilities of obstructive CAD for CTA, we used the no-treat/treat threshold method.12 Following the European Society of Cardiology guidelines, the no-treat/treat thresholds for CTA were 15% and 50%, respectively, on the disease probability range. This means that for disease probabilities below 15%, other reasons for the chest pain should be considered, and for values above 50%, ischaemia testing should be recommended.3
Secondary outcomes were sensitivity and specificity analyses in women and men, in patients of different age groups, and with different angina pectoris types. Diagnostic performance of CTA was descriptively compared in computed tomography scanners with up to 64 detector rows versus those with more than 64 detector rows. A further post hoc analysis, which was not defined in the protocol of COME-CCT, were requested by reviewers: we analysed the use of core laboratories and quantitative coronary angiography in relation to diagnostic accuracy of CTA.
Using an intention-to-diagnose approach, we implemented a worst case scenario in which non-diagnostic CTA results were considered false positive if coronary angiography was negative, and considered false negative if coronary angiography was positive.16 We calculated clinical pretest probability using the validated CAD Consortium prediction tool, which is an updated version of the Diamond and Forrester model.1718 Specifically, probability was estimated using all elements of the prediction tool: patient age, sex, and clinical presentation (angina pectoris type). Based on this model, mean logit PPVs and NPVs with their standard errors and 95% confidence intervals were estimated. These quantities were back-transformed to the original scale to obtain summary PPVs and NPVs. For the statistical analysis, we applied a univariate logistic regression model24 extended by incorporating a random effect for study and a random slope for CTA or coronary angiography results, which is equivalent to a bivariate generalised linear mixed model.25 To maintain equivalence, interaction terms of CTA and covariates of interest are necessary and were thus included into the model. Using these data and model, we performed a statistical prediction for a new cohort following the ideas presented by Skrondal and Rabe-Hasketh.2627
To apply the no-treat/treat CAD probability thresholds for CTA, we chose two post-test probabilities to define the range when to offer CTA: below 15% when other reasons for the chest pain should then be considered (no treat threshold) and probabilities above 50% when ischaemia testing is then recommended (treat threshold). We then calculated the clinical prediction score pretest probabilities that yielded post-test probabilities after negative CTA of below 15% (that is, a NPV of at least 85%) and those after positive CTA of above 50% (that is, a PPV of at least 50%). In addition to the model in which non-diagnostic CTA results were considered false positive if coronary angiography was negative and false negative if coronary angiography was positive, we also calculated NPVs and PPVs depending on pretest probabilities in a model excluding non-diagnostic CTA.
Based on the generalised linear mixed model with the test result as a dependent variable, we estimated mean logit sensitivity and specificity, between-study variability in logit sensitivity and specificity, and the covariance between them and the effect of covariates. Areas under the receiver operating characteristic curves were constructed using the observed data and model based predictions. These also included random effects, which reflect variability between studies and unobserved influential variables. The clinical performance of CTA was compared including non-diagnostic CTA results between women and men, between four age groups, and between the four angina pectoris types. Furthermore, it was compared with quantitative coronary angiography as the reference standard and the presence of core laboratories in the case of multicentre studies, to determine if this affected the primary outcomes. The influence of these covariates was evaluated by the likelihood ratio test. We did not search for the most parsimonious statistical model because, for clinical reasons, the type of chest pain, for instance, is pivotal and should be included. To compare the area-under-the-curve results for inclusion versus exclusion of non-diagnostic examinations, we applied DeLongs’ test.28 Performance of CTA using up to 64 or more than 64 detector rows was compared empirically. We investigated publication bias using the rank based method for the arcsine difference of study specific sensitivities and specificities.29
As recommended by the PRISMA statement, we compared aggregate data of studies for which IPD was provided with those aggregate data of studies for which IPD was not provided to identify or exclude differences between these data based on a bivariate generalised linear mixed model with IPD available as a covariate.30 We calculated the likelihood ratio test for the models with and without this covariate. This model was estimated for aggregate 2×2 tables using the model of Chu and Cole.31 In our analyses, non-diagnostic CTA results were treated using an intention-to-diagnose approach (see above) as suggested by Schuetz and colleagues.16 Estimation was done with Stata 14, using the packages GLLAMM and gllapred for the predictions and MIDAS for the 2×2 diagnostic meta-analysis. Further statistical analyses were conducted by SAS version 9.4 (SAS Institute, Cary, NC, USA) and R 3.4,32 and packages lme4,33 meta,34 and pROC.35 Model based estimates of sensitivity and specificity were obtained by averaging over the random effects using the R package lsmeans.36
Patient and public involvement
We have insufficient evidence to comment on whether patients were actively involved in the design or management of the 76 studies included and for which IPD were provided. In this study, given the privacy concerns of IPD sharing, it was also not practical to involve patients in reviewing this data. The results of the study will be disseminated using press releases and the website of the study coordinator.
Included studies and study participants
We identified 154 eligible published studies, for which we sought IPD by contacting authors (fig 1). Overall, we identified 76 studies (74 published, 2 unpublished) for which IPD were provided for a total of 7813 participants. Of these 76 studies, 11 (including 2481 patients) had to be excluded from this main COME-CCT analysis because no information on angina pectoris presentation was recorded, chest pain was unstable, or patients had coronary artery stents or bypasses. Finally, a total of 5332 patients from the remaining 65 studies (63 published and two unpublished; supplementary table 1 in web appendix 2) were included in the main collaborative analysis, including 554 patients with non-diagnostic CTA examinations (fig 1). Risk of bias was low for all items in most studies, and applicability concerns were not present in any of the included studies assessed by the QUADAS-2 tool (supplementary tables 2-4 and supplementary figures 1-2 in web appendix 2). In most of the 78 studies for which IPD were not provided (for a total of 6684 patients included), the corresponding authors did not respond (56/78, 72%) and aggregate data were available for 76 studies, or 6077 patients (fig 1).
Patient characteristics of the 5332 patients from 64 studies available for IPD analysis and their assignment to pretest probability categories are presented in table 1. Patient characteristics for each dataset including chest pain symptoms and risk factor distribution are listed in supplementary table 5 in web appendix 2. Technical characteristics of imaging tests for each dataset are summarised in supplementary table 6 in web appendix 2. Table 1 shows empirical results for true positives and negatives, false positives and negatives, as well as non-diagnostic CTA results for different categories of clinical pretest probability. Up to a pretest probability of 40%, pretest probability predictions overestimated true CAD prevalence by about 10 percentage points using the updated version of the Diamond and Forrester method. Above a pretest probability of 50%, true CAD prevalence was underestimated by about 10 percentage points. Above a pretest probability of about 70%, empirical diagnostic accuracy including specificity decreased (table 1). Also, CTA using up to 64 versus more than 64 detector rows led to significantly lower empirical sensitivity (86.5% v 93.4%, P=0.002) and specificity (72.6% v 84.4%, P<0.001, table 1). Non-diagnostic examinations were rare for scanners with more than 64 detector rows (2.9%), but considerably more frequent on those with up to 64 detector rows (11.5%, table 1).
Diagnostic performance of CTA depending on pretest probability
Figure 2 shows the relation between PPVs and NPVs of CTA and clinical pretest probability in the generalised linear mixed model including non-diagnostic examinations. Based on the statistical model, we found that for a pretest probability range of 7-67%, the treat threshold of more than 50% post-test probability (ischaemia testing recommended) and the no-treat threshold of less than 15% (consider other reasons for the chest pain) were obtained using CTA. At a pretest probability of 7%, the PPV of CTA was 50.9% (43.3% to 57.7%) and NPV was 97.8% (96.4% to 98.7%). At a pretest probability of 67%, the PPV was 82.7% (78.3% to 86.2%) and NPV was 85.0% (80.2% to 88.9%).
When excluding non-diagnostic examinations, the PPV at a pretest probability of 7% was 68.0% (60.5% to 74.6%) and NPV was 98.3% (97.0% to 99.1%; table 2); at a pretest probability of 67%, the PPV was 88.9% (85.3% to 91.7%) and NPV was 91.4% (87.8% to 94.2%). The relation between PPVs and NPVs of CTA and clinical pretest probability in the generalised linear mixed model after excluding non-diagnostic CTA examinations is shown in supplementary figure 3 in web appendix 2. The model based predictive values for 7% and 67% pretest probabilities as well as for 15% and 50% pretest probabilities as recommended by the European Society of Cardiology guidelines are listed in table 2 for both, including and excluding non-diagnostic CTA results.
Clinically important subgroups
The sensitivity of CTA for all patients was 95.2% (92.6% to 96.9%) and the specificity was 79.2% (74.9% to 82.9%, table 3). The sensitivity of CTA for women and men was 93.5% (89.6% to 96.0%) and 95.8% (93.4% to 97.4%), respectively, while the specificity was 80.6% (75.9% to 84.6%) and 77.4% (72.4% to 81.8%, likelihood ratio test11.28, df: 2, P<0.001, table 3). Empirical data of women and men and their assignment to pretest probability categories are tabulated in supplementary tables 7 and 8 in web appendix 2. For patients older than 75, the sensitivity of CTA was 93.2% (88.6% to 96.0%) and the specificity was 73.6% (65.7% to 80.2%, table 3).
In the receiver operating characteristic analysis, CTA including non-diagnostic results had an area under the curve of 0.897 (95% confidence interval 0.889 to 0.906) versus a significantly larger area under the curve when non-diagnostic results were excluded (0.949 (0.943 to 0.954), P<0.001, fig 3). All further results are provided for the analyses with inclusion of non-diagnostic CTA examinations. The diagnostic performance of CTA was lower in women than in men (area under the curve 0.874 (0.858 to 0.890) v 0.907 (0.897 to 0.916), P<0.001, fig 3). A descriptive analysis revealed that the heart rate during CTA was higher in women than in men (supplementary figure 4 in web appendix 2). Heart rate was the only factor significantly associated with a non-diagnostic CTA examination (supplementary table 9 in web appendix 2). The lowest diagnostic performance of CTA was found in patients older than 75 (0.864, 0.834 to 0.894, P=0.018 v all other age groups, fig 3). Empirical data of patients of different age groups and their assignment to pretest probability categories are listed in supplementary tables 10-13 in web appendix 2. In a descriptive analysis, patients older than 75 had significantly higher coronary artery calcium scoresthan younger patients (supplementary figure 5 in web appendix 2).37 Accuracy of CTA was not significantly influenced by the angina pectoris type (area under the curve for typical angina 0.895 (0.873 to 0.917) v atypical angina 0.898 (0.884 to 0.913) v non-anginal chest pain 0.884 (0.870 to 0.899) v other chest discomfort: 0.915 (0.897 to 0.934), fig 3).
Further post hoc analyses
Empirical data of patients with different chest pain types and their assignment to pretest probability categories are tabulated in supplementary tables 14-17 in web appendix 2. The receiver operating characteristic analysis after excluding non-diagnostic CTA results showed that accuracy was significantly reduced only in patients older than 75, whereas sex was no longer a significant factor (supplementary figure 6 in web appendix 2). Overall, 3615 (69%) of 5266 patients were analysed with quantitative coronary angiography as the reference standard. We found no significant difference in diagnostic accuracy of CTA, irrespective of whether quantitative coronary angiography was used or not, while the use of core laboratories was associated with lower sensitivity and specificity (supplementary table 18).
Participation and publication bias
The comparison of the diagnostic accuracy studies for which IPD were provided with those studies for which only aggregate data were available (fig 1) showed no significant difference in diagnostic performance (P=0.73, fig 4 and table 4). We found no publication bias (supplementary figure 7 in web appendix 2), and heterogeneity analysis yielded variances of random effects of 0.673 for 1−specificity and 3.667 for sensitivity (table 5). We obtained similar values after adjusting age, sex, and type of chest pain (table 6), indicating that these covariates do not explain heterogeneity between studies. Table 6 also presents the model coefficients used for generating the receiver operating characteristic curves.
In this pooled analysis of patient level data, we show that coronary CTA is most appropriately implemented for clinical decision making in patients with suspected obstructive CAD and a pretest probability ranging from 7% to 67%. In this low-to-intermediate clinical probability range, coronary CTA was able to accurately stratify patients into those with a disease post-test probability of below 15%, in whom other reasons for the chest pain should be considered, and those with a probability above 50%, in whom further testing is recommended.3
Our study also showed that the diagnostic performance of CTA was not significantly influenced by the angina pectoris type, but it was higher in men and lower in older patients. After we excluded non-diagnostic examinations from the analysis, the accuracy of CTA improved and the difference in diagnostic performance between female and male patients became non-significant. Moreover, diagnostic examinations are now more commonly conducted by computed tomography scanners with more than 64 detector rows, which had lower rates of non-diagnostic examinations.
Clinical context and guidelines
Current European and US guidelines recommend calculating patients’ pretest probability of CAD to guide diagnostic decisions.338 The European Society of Cardiology specifically recommends considering CTA in patients with 15-50% pretest probability of obstructive CAD,3 whereas the NICE guideline recommends coronary CTA as the primary imaging test for all patients with possible angina and suspected obstructive CAD.4 Our results show that using the no-treat/treat threshold approach, CTA offers good to excellent results in pretest probability range of 7% to 67%. The procedure yields a post-test probability below 15%, where other reasons for the chest pain should be considered, in case of negative CTA (that is, NPV ≥85%); and above 50%, where ischaemia testing is recommended, in case of positive CTA (that is, PPV ≥50%). Since no IPD meta-analysis has so far investigated in which patients CTA has the highest diagnostic performance, the results presented here might have important implications for current guidelines. The results of the diagnostic performance model can also be used to define the appropriate pretest probability range depending on the NPV and PPV deemed to be acceptable for the specific diagnostic purpose.
The main clinical strength of coronary CTA is its high NPV, and this is supported by our findings, which show that CTA can also detect both obstructive and non-obstructive CAD and therefore is a suitable imaging modality to guide subsequent management.39 This may make patient management more efficient and can also lower costs, not least by reducing the high rate of negative coronary angiographies performed annually. Recently published randomised clinical trials support these assumptions. Although the PROMISE trial—which compared CTA with an initial functional testing strategy in the evaluation of chest pain—did not show a reduction in major adverse cardiovascular events (defined as death, myocardial infarction, and unstable angina needing hospital admissions, or a major procedural complication), subsequent invasive coronary angiography was more effective in the CTA group.40
The SCOT-HEART trial prospectively compared standard care with standard care plus CTA for the diagnosis of CAD in patients with recent onset chest pain.41 In the trial, CTA was found to increase diagnostic certainty, increase the identification of obstructive and non-obstructive CAD, and eliminate the need for further downstream stress imaging tests.41 Furthermore, the five year clinical outcome analysis of SCOT-HEART showed that standard care plus CTA resulted in a halving of fatal and non-fatal myocardial infarction without increasing the five year rate of coronary revascularisations but initiating more targeted preventive and anti-anginal treatments.9 However, some controversy remains about the use of coronary CTA as the first line diagnostic test in patients with stable chest pain and suspected CAD,42 and our IPD meta-analysis provides insights about in which patients CTA has highest predictive values.
Our IPD meta-analysis data can thus help physicians in better identifying the patients for whom coronary CTA is the most appropriate diagnostic test. Whether CTA can further improve clinical effectiveness in patients with a clinical indication for coronary angiography is an important question. The CAD-Man study showed that coronary CTA can reduce the need for invasive coronary angiography by up to 80% and can reduce procedural complications.8 A similar safety profile with non-inferiority of CTA versus invasive coronary angiography in terms of major cardiovascular events at one year was found in the CONSERVE trial.43 However, coronary CTA still has to be analysed in a multicentre study of patients with a clinical indication for invasive coronary angiography, and the randomised DISCHARGE trial will provide more data in this regard.44
Comparison with other studies
Meta-analyses using aggregated data from studies that mostly excluded patients with non-diagnostic CTA examinations or considered them positive have reported a mean sensitivity for CTA per patient of 97.2% to 100% and a specificity of 87.4% to 89%.2145 We found lower sensitivities and specificities when including non-diagnostic tests as false positives or negatives in our IPD analysis in a worst case scenario, confirming that the performance of diagnostic tests is lower when non-diagnostic test results are considered and not merely excluded from the analysis.16 Our data also confirm the findings of a study level meta-regression analysis suggesting a hyperbolic decrease and increase of the NPVs and PPVs with increasing pretest probability, respectively.7 We also showed that pretest probability overestimated true CAD prevalence by about 10 percentage points up to a pretest probability of 40%; while above a pretest probability of 50%, true CAD prevalence was underestimated by about 10 percentage points. Future trials should address how to improve the accuracy of pretest probability estimation in patients with suspected CAD. Also, CTA using more than 64 detector rows led to significantly higher empirical sensitivity and specificity, indicating that recent CTA technology with more than 64 rows should be used.
Criteria have been proposed to ensure a reasonable use of coronary CTA.4647 Our study can help refine these criteria by allowing to individually define the appropriateness of coronary CTA based on the patient’s clinical pretest probability. Moreover, according to our findings, one should be cautious to use CTA in patients with a clinical pretest probability exceeding 67% since the NPV drops below 85%. In addition, the odds to find obstructive CAD on CTA (and thus also the likelihood to require another invasive test after non-invasive CTA) increases with the pretest probability. On the other hand, the PPV of coronary CTA becomes rather low in patients with a pretest probability of less than 7%, so that, in this situation, about half of the positive CTA examinations would result in unnecessary further testing. For ease of understanding, we visualised the predictive values of coronary CTA depending on pretest probability in figure 2. The European Society of Cardiology guidelines suggest a pretest probability range of 15-50% for diagnostic testing with coronary CTA. In this narrower range of pretest probability, CTA had an NPV and PPV of at least 90.9% and 55.8%, respectively.
From a clinical perspective, the diagnostic performance of CTA was not influenced by the angina pectoris type and was equally effective in ruling out angiographic CAD in patients with different angina pectoris types. Even though the reductions in diagnostic performance of CTA were small, decision makers should be aware that CTA has a slightly lower accuracy in patients older than 75, and in women compared with men, if non-diagnostic CTA results are included in the analysis. As mentioned above, non-diagnostic examinations are rarely seen when using computed tomography scanners with more than 64 detector rows; and when excluding non-diagnostic examinations, performance of CTA was similar in women and men. Similarly, our results showed that women had higher heart rates than men when examined by CTA and higher rates were the only factor associated with non-diagnostic examinations. Similar diagnostic accuracy of coronary CTA in men and women was reported by a multicentre study including 291 patients48 and by two single centre studies including 570 and 1372 patients.4950 In our IPD analysis of 3473 men and 1859 women including non-diagnostic examinations, we showed a small reduction in the area under the curve of CTA in women by 0.023 compared with men (fig 3). This difference might be explained by women being more likely to have high heart rates during CTA, which was the only factor significantly associated with non-diagnostic CTA results.
Strength and limitations of study
Our study had strengths and limitations. IPD meta-analyses are considered the gold standard of systematic reviews. Even though the individual diagnostic accuracy studies were similar in terms of inclusion criteria and reference standard definitions, they varied in geographical origin and composition. Although this study was done in 22 countries and has a multicentric and multicontinental design, participation was not equally distributed across the globe, and ethnicity was not collected in data analysis. Moreover, obstructive CAD was defined by invasive coronary angiography as angiographically significant CAD in all patients, quantitative analysis of invasive angiography was used in 69% of patients, and functional definitions of CAD (eg, including invasive fractional flow reserve) were not used in the original studies. Thus, findings might not be generalisable to real world practice, although additional invasive fractional flow reserve is used in less than 10% of examinations worldwide,51 making the findings relevant for current clinical practice.
To define no-treat and treat thresholds, we estimated pretest probabilities by using the updated Diamond and Forester model (also recommended by the current the European Society of Cardiology guidelines). This calculator is validated for patients with suspected CAD referred for invasive coronary angiography, which is also the setting of this analysis. Other prediction models for pretest probabilities do not focus on this cohort but on patients referred for non-invasive assessment, as in the CONFIRM study.52 Furthermore, although results of exercise tests can also be included in pretest calculation, they are not included in currently validated probability calculators and could thus not be considered in our review.53
As shown in table 1, the most frequently used computed tomography scanners had 64 detector rows (2438 of 5332 patients); thus, CTA performance in clinical practice using state-of-the-art technology with more than 64 detector rows could have been even better. An important limitation of our IPD analysis of the clinical performance of coronary CTA was that not all 154 studies that were identified through our search strategy could be included because the responsible corresponding authors did not provide IPD. However, we sought to systematically retrieve all IPD from the studies identified by the systematic review and, despite several reminders, a relevant proportion of authors did not reply at all (56/154, 36%) or indicated that they could not participate in the COME-CCT Consortium because they had no access to original data (7/154, 5%). According to a systematic review of data retrieval in IPD meta-analyses, 68% of meta-analyses retrieved IPD from at least 80% of a median of only 14 eligible studies.54 With 154 eligible studies, our study was relevantly larger, which has been shown to complicate retrieval.54
Diagnostic performance results were similar in studies for which IPD were available versus those for which no IPD were provided. To include unpublished grey literature, we systematically asked all corresponding authors of the identified published studies about further unpublished analyses and systematically searched clinicaltrials.gov for unpublished diagnostic accuracy studies of coronary CTA and invasive coronary angiography registered in this database. With this approach, we found two unpublished studies that could be included in the COME-CCT database. Our findings did not show evidence of publication bias, but we found heterogeneity between studies, pointing to potentially unknown site specific factors that might have influenced diagnostic accuracy. All studies included patients who had suspected CAD and were clinically indicated to undergo coronary angiography. This gave us the opportunity to compare results from research CTA with clinically indicated coronary angiography in all patients to avoid verification bias. But the results are representative for patients clinically referred for coronary angiography, and there was likely to be bias particularly at the extremes of pretest probability. For instance, individuals with low pretest probability were likely to have other unmeasured risk factors that increased their clinical probability, which could have overestimated PPVs of CTA.
In a no-treat/treat threshold model, the diagnosis of obstructive CAD using coronary CTA in patients with stable chest pain was most accurate when the clinical pretest probability was between 7% and 67%. Performance of CTA was not influenced by the angina pectoris type, was slightly higher in men, and was lower in older patients.
What is already known on this topic
Coronary computed tomography angiography (CTA) is an accurate non-invasive alternative to invasive coronary angiography, and can rule out coronary artery disease (CAD) with high certainty
By contrast with recent guidelines from the National Institute for Health and Care Excellence, the European Society of Cardiology recommends not considering CTA in all patients with typical and atypical angina, but only in patients with a 15-50% pretest probability of CAD, estimated by clinical information such as sex, age, and chest pain type
What this study adds
According to a no-treat/treat threshold model, patients with a pretest probability of CAD ranging from 7% to 67% could benefit most from coronary CTA to rule out or confirm CAD
CTA using more than 64 detector rows was empirically more sensitive and specific than CTA using up to 64 detector rows
Performance of CTA was not influenced by the angina pectoris type and was slightly higher in men and lower in older patients
Contributors: The COME-CCT investigators shared data, approved the manuscript after reviewing the findings and interpretation, provided critical revisions of the manuscript for important intellectual content, and gave final approval of the version to be published; and are accountable for the work. RH, PS, and MD drafted the report. RH, RR, GMS, and VW undertook literature searches, and gathered and harmonised data. All members of the writing committee participated in the drafting of the original manuscript. All members of the coordinating centre participated in the data collection and analysis. The data management team harmonised all data. PS performed the statistical analysis. MD is the guarantor. The corresponding author affirms that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This study was funded by the joint programme of the German Research Foundation (DFG) and the German Federal Ministry of Education and Research (BMBF, 01KG1110) to PS and MD. All researchers are independent of the funding bodies. The funding bodies had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the manuscript for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: PS and MD had support from the joint programme of the German Research Foundation and the German Federal Ministry of Education and Research for the submitted work; PS has support from the German Research Foundation, grants from the European Union and grants from Bayer Pharma AG; GP reports grants from General Electric and is on the speakers bureau for Medtronic and Bracco; JH is on the speakers bureau for Abbott Vascular and Edwards Life Sciences; BG reports that the Cliniques St Luc UCL holds a master research agreement with Philips Medical Systems; UJS reports institutional grants, personal fees, and non-financial support from Astellas, Bayer, General Electric, Guerbet, HeartFlow, and Siemens; BLN reports grants from Siemens and HeartFlow, JKreports grants from CardiRad and personal fees from GE Healthcare; RRB reports that the University Hospital Zurich holds a research agreement with GE Healthcare; MYC reports an institutional research agreement with Canon Medical, formerly Toshiba Medical (no financial support/funding); DAH reports institutional support from Philips Healthcare during the conduct of the primary study; BH-M reports that the University Hospital Zurich holds a research agreement with GE Healthcare; BJC reports grants from CV Diagnostix and non-financial support from TeraRecon during the conduct of the study; PAK reports that the University Hospital Zurich holds a research agreement with GE Healthcare; KFK reports grants from Toshiba Medical Corporation, grants from the Danish Heart Foundation, grants from AP Møller og hustru Chastine McKinney Møllers Fond, and grants from the Danish Agency for Science, Technology and Innovation by the Danish Council for Strategic Research; AA-Z reports grants and non-financial support from Toshiba Medical Systems; JH reports grants from Toshiba Medical Systems during the conduct of the study; AS reports personal fees from General Electric and Toshiba; NP is on the speakers bureau for Toshiba Medical Systems and reports grants from Toshiba Medical Systems; GMS reports grants from the German Federal Ministry of Education and Research (BMBF), during the conduct of the study; DEN reports grants from Toshiba Medical Systems; MD is supported by the FP7 programme of the European Commission for the randomised multicentre DISCHARGE trial (603266-2, HEALTH-2012.2.4.-2) from the Heisenberg programme of the German Research Foundation (DE 1361/14-1), and the Digital Health Accelerator of the Berlin Institute of Health, has received lecture fees from Canon Medical Systems, Guerbet, Cardiac MR Academy Berlin, and Bayer, is the editor of Cardiac CT (Springer), and offers hands-on workshops on cardiac CT imaging (http://herz-kurs.de/); Charité institutional master research agreements exist with Siemens Medical Solutions, General Electric, Philips Medical Systems, and Canon Medical Systems, and the terms of these arrangements are managed by the legal department of Charité - Universitätsmedizin Berlin.
Ethical approval: All original studies operated under supervision of an appropriate human ethics committee. This meta-analysis is exempt from ethics approval because the study collected data from previous clinical studies in which informed consent was already obtained.
Data sharing: Requests for patient level data will be considered by the COME-CCT Consortium.
The lead author affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.