Papers

Cross sectional study of conventional cervical smear, monolayer cytology, and human papillomavirus DNA testing for cervical cancer screening

Joël Coste, professor of medical statistics ^a, Béatrix Cochand-Priollet, assistant professor of pathology ^b, Patricia de Cremoux, assistant professor of pharmacology ^c, Catherine Le Galès, senior economist ^d, Isabelle Cartier, pathologist ^e, Vincent Molinié, pathologist ^f, Sylvain Labbé, pathologist ^g, Marie-Cécile Vacher-Lavenu, professor of pathology ^h, Philippe Vielh, pathologist ⁱ,  for the French Society of Clinical Cytology Study Group

^a Département de Biostatistique, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Faculté de Médecine Cochin-Port Royal, Université Paris V, Paris, France, ^b Service d'Anatomie et Cytologie Pathologiques, Hopital Lariboisière, Assistance Publique-Hôpitaux de Paris, Paris, France, ^c Laboratoire de Physiopathologie, Département de Biologie des Tumeurs, Institut Curie, Paris, France, ^d Centre for Health Economics and Administration Research (CREGAS), INSERM U537-CNRS UPRESA 8052, Le Kremlin-Bicêtre, France, ^e Laboratoire Cartier, Paris, France, ^f Service d'Anatomie et Cytologie Pathologiques, Hôpital Foch, Suresnes, France, ^g Centre d'Anatomie Pathologique, Besançon, France, ^h Service d'Anatomie et Cytologie Pathologiques, Hôpital Cochin, ⁱ Service de Cytopathologie et Cytométrie clinique, Institut Curie, Paris, France

Correspondence to: coste{at}cochin.univ-paris5.fr

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objectives: To compare the sensitivity, specificity, and interobserver reliability of conventional cervical smear tests, monolayer cytology, and human papillomavirus testing for screening for cervical cancer.
Design: Cross sectional study in which the three techniques were performed simultaneously with a reference standard (colposcopy and histology).
Setting: Public university and private practices in France, with complete independence from the suppliers.
Participants: 828 women referred for colposcopy because of previously detected cytological abnormalities and 1757 women attending for routine smears.
Main outcome measures: Clinical readings and optimised interpretation (two blind readings followed, if necessary, by consensus). Sensitivity, specificity, and weighted  computed for various thresholds of abnormalities.
Results: Conventional cervical smear tests were more often satisfactory (91% v 87%) according to the Bethesda system, more reliable (weighted  0.70 v 0.57), and had consistently better sensitivity and specificity than monolayer cytology. These findings applied to clinical readings and optimised interpretations, low and high grade lesions, and populations with low and high incidence of abnormalities. Human papillomavirus testing associated with monolayer cytology, whether systematic or for atypical cells of undetermined significance, performed no better than conventional smear tests.
Conclusions: Monolayer cytology is less reliable and more likely to give false positive and false negative results than conventional cervical smear tests for screening for cervical cancer.

What is already known on this topic New technologies have been developed to improve the detection of cervical cancer and its precursors and reduce the rate of false negative results from conventional cervical smear tests In several countries liquid based monolayer cytology is replacing conventional smear tests, despite controversy about whether these more expensive tests perform better What this study adds Conventional cervical smear testing is superior in terms of low and high grade lesions and in populations with a low or a high incidence of abnormalities Monolayer testing is less reliable and should not replace conventional cervical smear testing

	Introduction

Top Abstract Introduction Methods Results Discussion References

Liquid based "monolayer" cytology, possibly combined with human papillomavirus testing, is replacing conventional smear tests for cervical cancer screening in several countries (including the United States and Switzerland). However, there is substantial controversy about whether the new and costly technologies perform better than conventional cervical smear tests. ^{1 2} We previously compared the cost of monolayer cytology (ThinPrep, CYTYC; MA, USA) and human papillomavirus testing (Hybrid Capture II test, Digene; Gaithersburg, MD, USA) with conventional smear tests.³ Here we assess the sensitivity and specificity of the three methods. We also examined the value of human papillomavirus testing in women with atypical squamous cells/glandular cells of undetermined significance (ASCUS/AGUS) and the interobserver reproducibility of the interpretation of conventional smears and monolayer cytology.

	Methods

Top Abstract Introduction Methods Results Discussion References

Full details of the study protocol have been published previously.⁴ To avoid spectrum (case mix) bias⁵ we considered two groups of consecutive women who were either referred for colposcopy because abnormalities had been detected on previous smears or were attending for routine smears at a French public university (n=2) and private practices (n=2). All procedures were carried out by skilled gynaecologists and experienced cytopathologists. Each woman underwent a standard conventional smear test. The remaining material was then used to prepare the monolayer slide and for human papillomavirus testing. To avoid work up bias, all women were evaluated by all three methods (conventional cervical smear tests, monolayer cytology, human papillomavirus testing) and by the reference method (colposcopy⁶ followed by biopsy if abnormalities were detected). To avoid review and context biases⁷ cytopathologists read the slides blind to the clinical context in addition to routine reading, separately and independently for the three methods. In cases of disagreement slides were read again to reach a consensus conclusion, given if necessary by an independent expert (optimised diagnosis). In a random sample of the women (30%) we assessed interobserver reproducibility of cytological diagnosis with readings blind to context.

Smear abnormalities were classified into five ordered categories (negative, ASCUS/AGUS, low grade (LSIL) or high grade (HSIL) squamous intraepithelial lesions, invasive cancer) and the reference standard into four ordered categories (normal colposcopy or negative biopsy result, cervical intraepithelial neoplasia (grades I, II, and III), invasive carcinoma⁸). We used optimised histological diagnoses for the reference standard. We carried out human papillomavirus testing using the cell suspension that remained after monolayer preparation⁹ using low risk (types 6/11/42/43/44) and high risk (types 16/18/31/33/35/39/45/51/52/56/58/59/68) human papillomavirus probes.

Analysis
We compared the sensitivity, specificity, and proportions of unsatisfactory (according to the Bethesda system⁸) or limited slides using the two tailed MacNemar ² test. The interobserver reproducibility of the readings was assessed with weighted  statistics.¹⁰

To assess potential "sampling bias" due to the split sample technique (monolayer being sampled after conventional cervical smears) we repeated statistical analyses in the subsample of women in whom the residual material after monolayer cytology was sufficient for human papillomavirus testing. The results were similar to those for the whole group and are not shown.

	Results

Top Abstract Introduction Methods Results Discussion References

Between 1 September 1999 and 30 May 2000, 2585 women underwent investigation (table 1). Results of human papillomavirus testing were available from the 1785 women for whom there was enough residual material. The proportion of satisfactory slides was higher with conventional smear testing (91%) than with monolayer testing (87%), though the reasons for unsatisfactory or limited smears differed (table 2). Compared with conventional smear tests monolayer testing consistently showed more abnormalities (especially ASCUS/AGUS) (tables 3 and 4).

Conventional smear tests consistently had superior or equivalent sensitivity, specificity, and likelihood ratios than monolayer tests for the detection cervical intraepithelial neoplasia grade I or higher (table 5) and lesions  grade II or higher (table 6). The sensitivity of systematic human papillomavirus DNA testing (high risk) was no higher than that of conventional smear testing and its specificity was much lower for both grades. For human papillomavirus testing only for ASCUS/AGUS, the sensitivity of the paired monolayer/human papillomavirus testing method was not significantly superior to cervical smear testing.

Interobserver agreement was good for conventional smears (weighted  0.69, 95% confidence interval 0.64 to 0.74) but only moderate for monolayers (0.57, 0.52 to 0.63) (table 7). The ASCUS/AGUS diagnosis with monolayer testing was especially unreliable.

	Discussion

Top Abstract Introduction Methods Results Discussion References

Our results support the superiority of conventional cervical smear testing, whether considered clinical readings or optimised interpretations, low or high grade lesions, or populations with a low or a high incidence of abnormalities. Human papillomavirus testing, systematic or for a diagnosis of ASCUS/AGUS testing, carried out with monolayer cytology was no better than conventional cervical smear testing. The greater reliability of the interpretation of conventional smears rather than monolayer smears is consistent with their better diagnostic or screening performance. Our findings disagree with those of most previous studies.

We ensured that we obtained the reference standard of colposcopy/histology for all women in the study, unlike previous studies that compared monolayer testing with conventional smear testing and that considered concordant positive and concordant negative tests as true positives and true negatives with discrepancies resolved by consensus review. ^{1 11-20} In these studies the proportion of verified cases varied between 0.1% and 30%, and the work up bias was substantial, artificially inflating sensitivity and mathematically favouring the test with the higher rate of false positives: the monolayer technique (or human papillomavirus testing). Two other studies either did not find any difference between the methods²¹ or performed post hoc subgroup analyses.²²

Limitations of study
In this study the cervical smear was prepared before the monolayer. However, a sampling bias favouring the conventional smear is unlikely as there were very few monolayer slides with only a few cells and the results were similar in the subgroup of women in whom human papillomavirus testing was still possible.²³ The rates of unsatisfactory and limited slides were low, which may be due to our selection of skilled physicians. The cytopathologists were also selected according to their interest in reading smears: all had extensive experience in conventional smears and cervical biopsies, but their experience with monolayer cytology was initially limited. However, this bias was neutralised by the optimised interpretations in which the best assessment was obtained.

Implications
This study has implications for regulation of medical devices, clinical practice, and future research on screening for cervical cancer. Monolayer testing, which seems less reliable and less valid and is more expensive,³ should not replace conventional smear tests for cervical cancer screening. Human papillomavirus testing as complementary to conventional smear testing should be further evaluated in clinical research.²⁴ Our results emphasise the need to improve the "hard evidence" in studies of new technologies for cervical screening by using adequate methodological standards.

	Acknowledgments

Members of the French Society of Clinical Cytology Study Group: S Arkwright, A Biaggi, C Besançon-Roux, L Carbillon, I Cartier, B Cochand-Priollet (clinical coordinator), J Coste (methodological coordinator), J Darondel, P Dauvergne, P de Cremoux, A Dosda, E Foucher, I Gouget, D Grondard, B Karkouche, S Labbé, C Le Galès, I Le Guen, V Lepoutre, N Lestrat, H Magdelenat, A Malvy-Nickles, E Merea, V Molinié, A Odier, A Petitjean, S Peschard, P Piquet, L Pommaret, X Sastre-Garau, V Saha, N Seince, C Sigal-Hummel, M C Vacher-Lavenu, P Vielh, M Ziol.

Contributors: JC, M-CV-L, VM, BC-P, CLG, and PV initiated the study and developed the study protocol. JC and BCP coordinated the study. BCP, PdeC, IC, VM, M-CV-L, SL, and PV participated in coordinating the study, data collection, and editing. JC calculated the sample size and designed and performed the analysis. JC prepared the manuscript with input from all other authors, revised the manuscript for resubmission, and is the guarantor.

	Footnotes

Funding: Direction Générale de la Santé and Programme Hospitalier de Recherche Clinique, French Ministry of Health (AOM 98010); Association pour la Recherche sur le Cancer (9099). The guarantor accepts full responsibility for the conduct of the study, had access to the data, and controlled the decision to publish.

Competing interests: None declared.

Ethical approval: The approval of the ethics committee (Hôpital Cochin, Paris) for the study was obtained in July 1998.

	References

Top Abstract Introduction Methods Results Discussion References

1.	Nanda K, McCrory DC, Myers ER, Bastian LA, Hasselblad V, Hickey JD, et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med 2000; 132: 810-819[Abstract/Free Full Text].
2.	Hartmann KE, Nanda K, Hall S, Myers E. Technologic advances for evaluation of cervical cytology: is newer better? Obstet Gynecol Surv 2001; 56: 765-774[CrossRef][Web of Science][Medline].
3.	Merea E, Le Galès C, Cochand-Priollet B, Cartier I, de Cremoux P, Vacher-Lavenu MC, et al. Cost of screening for cancerous and precancerous lesions of the cervix. Diagn Cytopathol 2002; 27: 251-257[CrossRef][Web of Science][Medline].
4.	Cochand-Priollet B, Le Galès C, de Cremoux P, Molinié V, Sastre-Garau X, Vacher-Lavenu MC, et al. Cost-effectiveness of monolayers and human papillomavirus testing compared to that of conventional Papanicolaou smears for cervical cancer screening: protocol of the study of the French Society of Clinical Cytology. Diagn Cytopathol 2001; 24: 412-420[CrossRef][Web of Science][Medline].
5.	Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research. Getting better but still not good. JAMA 1995; 274: 645-651[Abstract/Free Full Text].
6.	Stafl A, Wilbanks GD. An international terminology of colposcopy: report of the Nomenclature Committee of the International Federation of Cervical Pathology and Colposcopy. Obstet Gynecol 1991; 77: 313-314[Web of Science][Medline].
7.	Egglin TK, Feinstein AR. Context bias. A problem in diagnostic radiology. JAMA 1996; 276: 1752-1755[Abstract/Free Full Text].
8.	The Bethesda system for reporting cervical/vaginal cytologic diagnoses: revised after the second National Cancer Institute Workshop, April, 29-30, 1991. Acta Cytol 1993; 37: 115-124[Web of Science][Medline].
9.	Poljak M, Brencic A, Seme K, Vince A, Marin IJ. Comparative evaluation of first- and second-generation digene hybrid capture assays for detection of human papillomaviruses associated with high or intermediate risk for cervical cancer. J Clin Microbiol 1999; 37: 796-797[Abstract/Free Full Text].
10.	Fleiss J. Statistical methods for rates and proportions. New York: Wiley, 1981.
11.	Hutchinson ML, Zahniser DJ, Sherman ME, Herrero R, Alfaro M, Bratti MC, et al. Utility of liquid-based cytology for cervical carcinoma screening: results of a population-based study conducted in a region of Costa Rica with a high incidence of cervical carcinoma. Cancer 1999; 87: 48-55[CrossRef][Web of Science][Medline].
12.	Sheets EE, Constantine NM, Dimisco S, Dean B, Cibas ES. Colposcopically directed biopsy provide a basis for comparing the accuracy of ThinPrep and Papanicolaou smears. J Gynecol Tech 1995; 1: 27-33.
13.	Bishop JW, Bigner SH, Colgan TJ, Husain M, Howell LP, McIntosh KM, et al. Multicenter masked evaluation of AutoCyte PREP thin layers with matched conventional smears. Including initial biopsy results. Acta Cytol 1998; 42: 189-197[Web of Science][Medline].
14.	Roberts JM, Gurley AM, Thurloe JK, Bowditch R, Laverty CRA. Evaluation of the Thin Prep pap test as an adjunct to the conventional Pap smear. Med J Aust 1997; 167: 466-469[Web of Science][Medline].
15.	Bolick DR, Hellman DJ. Laboratory implementation and efficacy assessment of the ThinPrep cervical cancer screening system. Acta Cytol 1998; 42: 209-213[Web of Science][Medline].
16.	Park IA, Lee SN, Chae SW, Park KH, Kim JW, Lee HP. Comparing the accuracy of ThinPrep Pap tests and conventional Papanicolaou smears on the basis of the histologic diagnosis: a clinical study of women with cervical abnormalities. Acta Cytol 2001; 45: 525-531[Web of Science][Medline].
17.	Manos MM, Kinney WK, Hurley LB, Sherman ME, Shieh-Ngai J, Kurman RJ, et al. Identifying women with cervical neoplasia: using human papillomavirus DNA testing for equivocal Papanicolaou results. JAMA 1999; 281: 1605-1610[Abstract/Free Full Text].
18.	Ronnett BM, Manos MM, Ransley JE, Fetterman BJ, Kinney WK, Hurley LB, et al. Atypical glandular cells of undetermined significance (AGUS): cytopathologic features, histopathologic results, and human papillomavirus DNA detection. Hum Pathol 1999; 30: 816-825[CrossRef][Web of Science][Medline].
19.	Bergeron C, Jeannel D, Poveda J, Cassonnet P, Orth G. Human papillomavirus testing in women with mild cytologic atypia. Obstet Gynecol 2000; 95: 821-827[CrossRef][Web of Science][Medline].
20.	Schneider A, Hoyer H, Lotz B, Leistritza S, Kuhne-Heid R, Nindl I, et al. Screening for high-grade cervical intra-epithelial neoplasia and cancer by testing for high-risk HPV, routine cytology or colposcopy. Int J Cancer 2000; 89: 529-534[CrossRef][Web of Science][Medline].
21.	Ferenczy A, Robitaille J, Franco E, Arseneau J, Richart RM, Wright TC. Conventional cervical cytologic smears vs. Thin Prep smears. A paired comparaison study of cervical cytology. Acta Cytol 1996; 40: 1136-1142[Web of Science][Medline].
22.	Bergeron C, Bishop J, Lemarie A, Cas F, Ayivi J, Huynh B, et al. Accuracy of thin-layer cytology in patients undergoing cervical cone biopsy. Acta Cytol 2001; 45: 519-524[Web of Science][Medline].
23.	Bernstein SJ, Sanchez-Ramos L, Ndubisi B. Liquid-based cervical cytologic smear study and conventional Papanicolaou smears: a metaanalysis of prospective studies comparing cytologic diagnosis and sample adequacy. Am J Obstet Gynecol 2001; 185: 308-317[CrossRef][Web of Science][Medline].
24.	Kaufman RH. Is there a role for human papillomavirus testing in clinical practice? Obstet Gynecol 2001; 98: 724-725[CrossRef][Web of Science][Medline].

(Accepted 29 January 2003)