- Joël Coste, professor of medical statistics ()a,
- Béatrix Cochand-Priollet, assistant professor of pathologyb,
- Patricia de Cremoux, assistant professor of pharmacologyc,
- Catherine Le Galès, senior economistd,
- Cartier Isabelle, pathologiste,
- Molinié Vincent, pathologistf,
- Sylvain Labbé, pathologistg,
- Marie-Cécile Vacher-Lavenu, professor of pathologyh,
- a Département de Biostatistique, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Faculté de Médecine Cochin-Port Royal, Université Paris V, Paris, France
- b Service d'Anatomie et Cytologie Pathologiques, Hopital Lariboisière, Assistance Publique-Hôpitaux de Paris, Paris, France
- c Laboratoire de Physiopathologie, Département de Biologie des Tumeurs, Institut Curie, Paris, France
- d Centre for Health Economics and Administration Research (CREGAS), INSERM U537-CNRS UPRESA 8052, Le Kremlin-Bicêtre, France
- e Laboratoire Cartier, Paris, France
- f Service d'Anatomie et Cytologie Pathologiques, Hôpital Foch, Suresnes, France
- g Centre d'Anatomie Pathologique, Besançon, France
- h Service d'Anatomie et Cytologie Pathologiques, Hôpital Cochin
- i Service de Cytopathologie et Cytométrie clinique, Institut Curie, Paris, France
- Correspondence to:
- Accepted 29 January 2003
Objectives: To compare the sensitivity, specificity, and interobserver reliability of conventional cervical smear tests, monolayer cytology, and human papillomavirus testing for screening for cervical cancer.
Design: Cross sectional study in which the three techniques were performed simultaneously with a reference standard (colposcopy and histology).
Setting: Public university and private practices in France, with complete independence from the suppliers.
Participants: 828 women referred for colposcopy because of previously detected cytological abnormalities and 1757 women attending for routine smears.
Main outcome measures: Clinical readings and optimised interpretation (two blind readings followed, if necessary, by consensus). Sensitivity, specificity, and weighted κ computed for various thresholds of abnormalities.
Results: Conventional cervical smear tests were more often satisfactory (91% v 87%) according to the Bethesda system, more reliable (weighted κ 0.70 v 0.57), and had consistently better sensitivity and specificity than monolayer cytology. These findings applied to clinical readings and optimised interpretations, low and high grade lesions, and populations with low and high incidence of abnormalities. Human papillomavirus testing associated with monolayer cytology, whether systematic or for atypical cells of undetermined significance, performed no better than conventional smear tests.
Conclusions: Monolayer cytology is less reliable and more likely to give false positive and false negative results than conventional cervical smear tests for screening for cervical cancer.
What is already known on this topic
What is already known on this topic New technologies have been developed to improve the detection of cervical cancer and its precursors and reduce the rate of false negative results from conventional cervical smear tests
In several countries liquid based monolayer cytology is replacing conventional smear tests, despite controversy about whether these more expensive tests perform better
What this study adds
What this study adds Conventional cervical smear testing is superior in terms of low and high grade lesions and in populations with a low or a high incidence of abnormalities
Monolayer testing is less reliable and should not replace conventional cervical smear testing
Liquid based “monolayer” cytology, possibly combined with human papillomavirus testing, is replacing conventional smear tests for cervical cancer screening in several countries (including the United States and Switzerland). However, there is substantial controversy about whether the new and costly technologies perform better than conventional cervical smear tests. 1 2 We previously compared the cost of monolayer cytology (ThinPrep, CYTYC; MA, USA) and human papillomavirus testing (Hybrid Capture II test, Digene; Gaithersburg, MD, USA) with conventional smear tests.3 Here we assess the sensitivity and specificity of the three methods. We also examined the value of human papillomavirus testing in women with atypical squamous cells/glandular cells of undetermined significance (ASCUS/AGUS) and the interobserver reproducibility of the interpretation of conventional smears and monolayer cytology.
Full details of the study protocol have been published previously.4 To avoid spectrum (case mix) bias5 we considered two groups of consecutive women who were either referred for colposcopy because abnormalities had been detected on previous smears or were attending for routine smears at a French public university (n=2) and private practices (n=2). All procedures were carried out by skilled gynaecologists and experienced cytopathologists. Each woman underwent a standard conventional smear test. The remaining material was then used to prepare the monolayer slide and for human papillomavirus testing. To avoid work up bias, all women were evaluated by all three methods (conventional cervical smear tests, monolayer cytology, human papillomavirus testing) and by the reference method (colposcopy6 followed by biopsy if abnormalities were detected). To avoid review and context biases7 cytopathologists read the slides blind to the clinical context in addition to routine reading, separately and independently for the three methods. In cases of disagreement slides were read again to reach a consensus conclusion, given if necessary by an independent expert (optimised diagnosis). In a random sample of the women (30%) we assessed interobserver reproducibility of cytological diagnosis with readings blind to context.
Smear abnormalities were classified into five ordered categories (negative, ASCUS/AGUS, low grade (LSIL) or high grade (HSIL) squamous intraepithelial lesions, invasive cancer) and the reference standard into four ordered categories (normal colposcopy or negative biopsy result, cervical intraepithelial neoplasia (grades I, II, and III), invasive carcinoma8). We used optimised histological diagnoses for the reference standard. We carried out human papillomavirus testing using the cell suspension that remained after monolayer preparation9 using low risk (types 6/11/42/43/44) and high risk (types 16/18/31/33/35/39/45/51/52/56/58/59/68) human papillomavirus probes.
We compared the sensitivity, specificity, and proportions of unsatisfactory (according to the Bethesda system8) or limited slides using the two tailed MacNemar χ2 test. The interobserver reproducibility of the readings was assessed with weighted κ statistics.10
To assess potential “sampling bias” due to the split sample technique (monolayer being sampled after conventional cervical smears) we repeated statistical analyses in the subsample of women in whom the residual material after monolayer cytology was sufficient for human papillomavirus testing. The results were similar to those for the whole group and are not shown.
Between 1 September 1999 and 30 May 2000, 2585 women underwent investigation (table 1). Results of human papillomavirus testing were available from the 1785 women for whom there was enough residual material. The proportion of satisfactory slides was higher with conventional smear testing (91%) than with monolayer testing (87%), though the reasons for unsatisfactory or limited smears differed (table 2). Compared with conventional smear tests monolayer testing consistently showed more abnormalities (especially ASCUS/AGUS) (tables 3 and 4).
Conventional smear tests consistently had superior or equivalent sensitivity, specificity, and likelihood ratios than monolayer tests for the detection cervical intraepithelial neoplasia grade I or higher (table 5) and lesions ≥ grade II or higher (table 6). The sensitivity of systematic human papillomavirus DNA testing (high risk) was no higher than that of conventional smear testing and its specificity was much lower for both grades. For human papillomavirus testing only for ASCUS/AGUS, the sensitivity of the paired monolayer/human papillomavirus testing method was not significantly superior to cervical smear testing.
Interobserver agreement was good for conventional smears (weighted κ 0.69, 95% confidence interval 0.64 to 0.74) but only moderate for monolayers (0.57, 0.52 to 0.63) (table 7). The ASCUS/AGUS diagnosis with monolayer testing was especially unreliable.
Our results support the superiority of conventional cervical smear testing, whether considered clinical readings or optimised interpretations, low or high grade lesions, or populations with a low or a high incidence of abnormalities. Human papillomavirus testing, systematic or for a diagnosis of ASCUS/AGUS testing, carried out with monolayer cytology was no better than conventional cervical smear testing. The greater reliability of the interpretation of conventional smears rather than monolayer smears is consistent with their better diagnostic or screening performance. Our findings disagree with those of most previous studies.
We ensured that we obtained the reference standard of colposcopy/histology for all women in the study, unlike previous studies that compared monolayer testing with conventional smear testing and that considered concordant positive and concordant negative tests as true positives and true negatives with discrepancies resolved by consensus review. 1 11–20 In these studies the proportion of verified cases varied between 0.1% and 30%, and the work up bias was substantial, artificially inflating sensitivity and mathematically favouring the test with the higher rate of false positives: the monolayer technique (or human papillomavirus testing). Two other studies either did not find any difference between the methods21 or performed post hoc subgroup analyses.22
Limitations of study
In this study the cervical smear was prepared before the monolayer. However, a sampling bias favouring the conventional smear is unlikely as there were very few monolayer slides with only a few cells and the results were similar in the subgroup of women in whom human papillomavirus testing was still possible.23 The rates of unsatisfactory and limited slides were low, which may be due to our selection of skilled physicians. The cytopathologists were also selected according to their interest in reading smears: all had extensive experience in conventional smears and cervical biopsies, but their experience with monolayer cytology was initially limited. However, this bias was neutralised by the optimised interpretations in which the best assessment was obtained.
This study has implications for regulation of medical devices, clinical practice, and future research on screening for cervical cancer. Monolayer testing, which seems less reliable and less valid and is more expensive,3 should not replace conventional smear tests for cervical cancer screening. Human papillomavirus testing as complementary to conventional smear testing should be further evaluated in clinical research.24 Our results emphasise the need to improve the “hard evidence” in studies of new technologies for cervical screening by using adequate methodological standards.
Members of the French Society of Clinical Cytology Study Group: S Arkwright, A Biaggi, C Besançon-Roux, L Carbillon, I Cartier, B Cochand-Priollet (clinical coordinator), J Coste (methodological coordinator), J Darondel, P Dauvergne, P de Cremoux, A Dosda, E Foucher, I Gouget, D Grondard, B Karkouche, S Labbé, C Le Galès, I Le Guen, V Lepoutre, N Lestrat, H Magdelenat, A Malvy-Nickles, E Merea, V Molinié, A Odier, A Petitjean, S Peschard, P Piquet, L Pommaret, X Sastre-Garau, V Saha, N Seince, C Sigal-Hummel, M C Vacher-Lavenu, P Vielh, M Ziol.
Contributors: JC, M-CV-L, VM, BC-P, CLG, and PV initiated the study and developed the study protocol. JC and BCP coordinated the study. BCP, PdeC, IC, VM, M-CV-L, SL, and PV participated in coordinating the study, data collection, and editing. JC calculated the sample size and designed and performed the analysis. JC prepared the manuscript with input from all other authors, revised the manuscript for resubmission, and is the guarantor.
Funding Direction Générale de la Santé and Programme Hospitalier de Recherche Clinique, French Ministry of Health (AOM 98010); Association pour la Recherche sur le Cancer (9099). The guarantor accepts full responsibility for the conduct of the study, had access to the data, and controlled the decision to publish.
Competing interests None declared.
Ethical approval: The approval of the ethics committee (Hôpital Cochin, Paris) for the study was obtained in July 1998.