BMJ, doi: 10.1136/bmj.39210.582801.BE, (Published 1 June 2007)

research-article

Limitations of rapid HIV-1 tests during screening for trials in Uganda: diagnostic test accuracy study

¹ Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD 21205, USA, ² Johns Hopkins University, Bloomberg School of Public Health, Baltimore and Mailman School of Public Health, Columbia University, NY, USA, ³ Rakai Health Sciences Program, Uganda Virus Research Institute, Entebbe, Uganda, ⁴ Makerere University, Institute of Public Health, Kampala, Uganda, ⁵ National Institutes of Allergy and Infectious Diseases, National Institutes of Health, Entebbe, Uganda, ⁶ School of Medicine, Makerere University, Uganda

Abstract

Objective To evaluate the limitations of rapid tests for HIV-1.

Design Diagnostic test accuracy study.

Setting Rural Rakai, Uganda.

Participants 1517 males aged 15-49 screened for trials of circumcision for HIV prevention.

Main outcome measures Sensitivity, specificity, negative predictive values, and positive predictive values of an algorithm using three rapid tests for HIV, compared with the results of enzyme immunoassay and western blotting as the optimal methods.

Results Rapid test results were evaluated by enzyme immunoassay and western blotting. Sensitivity was 97.7%. Among 639 samples where the strength of positive bands was coded if the sample showed positivity for HIV, the algorithm had low specificity (94.1%) and a low positive predictive value (74.0%). Exclusion of 37 samples (5.8%) with a weak positive band improved the specificity (99.6%) and positive predictive value (97.7%).

Conclusion Weak positive bands on rapid tests for HIV should be confirmed by enzyme immunoassay and western blotting before disclosing the diagnosis. Programmes using rapid tests routinely should use standard serological assays for quality control.

Trial registration Clinical Trials NCT00425984 [ClinicalTrials.gov] .

Introduction

Rapid tests for the detection of antibodies to HIV-1 allow timely point of care provision of results and do not require the laboratory facilities needed for conventional enzyme immunoassays and western blot testing.¹ As a consequence rapid tests have been widely used for voluntary HIV counselling and testing, antenatal surveillance, and population screening.^{1 2} An FDA application is now pending for an over the counter home testing kit,³ and the expansion of requirements for HIV testing in the developing world will require use of these rapid tests.⁴ We have, however, encountered problems with the interpretation of positive results of rapid tests during the screening of populations in rural Uganda for two randomised trials of male circumcision for the prevention of HIV. We evaluated limitations of these tests in such settings.

Methods

The Rakai health sciences programme used rapid HIV tests to screen males for two randomised trials of circumcision for the prevention of HIV in a rural population in Rakai district of south western Uganda. One trial enrolled uncircumcised males aged 14-49 who were negative for HIV and the other uncircumcised males in the same age range who were HIV positive. The rapid tests were used to initially screen males for subsequent enrolment into these two trials.

During 2003-4 we used an algorithm incorporating three rapid HIV tests to screen potential participants for a randomised trial that enrolled males aged 15-49. The algorithm (figure) consisted of an initial screening with the rapid test Determine HIV-1/2/O (Abbott Laboratories, Abbott Park, IL). If the test result was negative the participant was given a diagnosis of HIV negative with no further rapid testing. If the test result was positive the sample was retested with the rapid test HIV 1/2 Stat-Pak Ultra Fast (Chembio Diagnostic Systems, Medford, NY). If both tests gave a positive result the participant was given a diagnosis of HIV positive with no further rapid testing. If the tests gave discordant results, the sample was further evaluated with the rapid test Uni-Gold Recombinant HIV-1/2 (Trinity Biotech, Bray, Ireland). For those samples assessed by all three tests, two positive test results were interpreted as a positive diagnosis. If two of the three tests gave negative results then the participant was diagnosed as being negative for HIV.

These rapid tests and the testing algorithm were approved by the Ugandan Ministry of Health and are widely used in Uganda and elsewhere. The tests were carried out in a field setting in small mobile laboratories set up in local buildings or tents. Tests were run on serum from blood collected and centrifuged immediately before testing. At the same time a separate serum aliquot was archived. To optimise test results the technicians were advised to ensure good lighting conditions and to use a flat table, according to the manufacturers' instructions. The reading of each result was timed with an electronic timer, using the duration specified by the manufacturer. Each test was read by two trained laboratory technicians, with the test card lying flat on the table. The tests were interpreted according to manufacturers' instructions, which recommend that any band in the positive region be considered as a positive result for HIV, irrespective of the strength of the band. This algorithm for rapid HIV-1 testing is widely used in Uganda. Subsequently all samples were batch retested for quality control, using two enzyme immunoassays (Vironostika HIV-1, Organon Teknika, Charlotte, NC and Cambridge Biotech, Worcester, MA), and discordant results were confirmed by western blotting (HIV-1 Western Blot; BioMerieux-Vitek, St Louis, MO).

View larger version (63K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Rapid testing algorithm and numbers of tests carried out for study sample

 
We tested one sample from each of 1517 participants. An initial assessment of 878 batched samples, however, suggested problems with false positive results, so after March 2004 the laboratory technicians recorded the intensity of positive bands in 639 samples. In this subgroup 125 samples tested positive for HIV, of which 37 were classified as weak positive bands (5.8%). Weak positive bands were defined as a sample with an apparent positive band that was lighter than the control positive band on the test card. We estimated the sensitivity, specificity, negative predictive values, and positive predictive values of this algorithm, compared with the results of the enzyme immunoassay and western blotting as the ideal methods. Analyses were carried out for the 1517 samples, for the 639 samples for which band intensity had been coded, and for the subgroup of tests in which we had excluded positive results coded as weak positive bands (n=602).

Results

The study population comprised 1517 males aged 15-49. The age distribution was 15-19 (21.3%), 20-24 (27.6%), 25-29 (20.4%), and 30-49 (28.8%). Fifty one per cent were married and most (65.2%) had achieved primary or secondary education (22.1%) or higher (7.3%). Only 5.4% reported no schooling. Most were sexually active (84.3%).

The table shows the results of the rapid tests for HIV. In the total sample of 1517 tests the three rapid test algorithm had reasonable sensitivity (97.7%, 95% confidence interval 94.1% to 99.4%) and negative predictive value (99.7%), but the specificity was low (90.4%, 95% confidence interval 88.7% to 91.9%) and the positive predictive value was unacceptably low (56.3%). Overall, 129 of 295 positive test results were false positives (43.7%) and four of 1222 negative results were false negatives (0.3%). Of the 129 false positives, 123 (95%) resulted from the Determine and Uni-Gold tests. Investigation of individual lots for each test did not show evidence of defective batches.

View this table: [in this window] [in a new window]   Sensitivity, specificity, positive predictive value, and negative predictive value of rapid tests for HIV before and after exclusion of results with weak positive bands

 
In the subsample of 639 tests with weak bands coded as HIV positive, the specificity (94.1%, 91.8% to 96.0%) and the positive predictive value (74.0%) were still unacceptably low and the false positive rate (26.0%) was high. Exclusion of the 37 samples with weak positive bands, however, noticeably improved the specificity (99.6%, 98.6% to 100.0%) and positive predictive value (97.7%) and reduced the rate of false positive results to 2.3% (2/86). Among the 37 samples coded as having weak positive bands 86.0% were HIV negative on enzyme linked immunoassay and western blotting and 8.1% had indeterminate results on western blotting. From experience indeterminate results on western blotting are almost always HIV negative on the basis of repeat enzyme immunoassay or western blotting or on polymerase chain reaction. Overall 94.1% of weak positive bands were not confirmed as positive by enzyme linked immunoassay and western blotting. Among the 37 samples with weak positive bands 70.3% had weak bands on the Determine test and 29.7% on the Uni-Gold test. This problem was not observed with Stat-Pak.

To assess the possibility that the problem with low specificity was primarily the result of observer error—that is, identification of "weak" bands when none existed—140 samples were sent for reassay at a Center for Disease Control and Prevention laboratory in Uganda. The laboratory reported 12 weak positive bands on at least one rapid test (8.3% of samples or 2.4% of all rapid tests carried out, R Downing, personal communication, 2006). These results are compatible with the findings that 5.8% of samples showed weak bands. Moreover, among the 10 samples with weak positive bands tested by the laboratory, which were then retested by enzyme immunoassay and western blotting, nine (90%) were negative on standard serology, a finding comparable to the 94.1% observed in the Rakai setting.

Discussion

An HIV testing algorithm consisting of three rapid tests in a Ugandan setting showed low specificity and low positive predictive values if weak bands were interpreted as positive, according to the manufacturers' recommendations. Exclusion of results with weak positive bands noticeably improved the performance of the algorithm and reduced the proportion of false positives to acceptably low levels.

The interpretation of positive bands on a rapid test is subjective. For example, the manufacturer's instructions for the Determine test state that any red in the patient's window should be interpreted as positive. It is possible that the laboratory technicians reading the test over-interpreted this instruction, and it is noteworthy that the specificity was higher during the period when the technicians were asked to code the band strength (table), suggesting that they became more cautious over time.

It is possible that the weak bands reflect cross reactions with other infections that might be endemic in these rural Ugandan populations, but we have no data in this regard. One other rural programme reported similar problems with false positive results from weak positive bands (H Grosskurth, personal communication, 2005), but two other evaluations of rapid tests in urban Uganda did not report these problems.^{5 6} We do not know whether the interpretation of weak positive rapid test results in rural Uganda affects other testing programmes in Africa. It is possible that this problem is caused by the dominant HIV-1 subtypes D, A, and AD recombinants found in Uganda, but high rates of false positive rapid test results have been reported in the United States, suggesting that this problem is not restricted to specific viral subtypes.³ Our findings may only pertain to the Determine and Uni-Gold tests and not to other rapid tests. Also, because these tests were only assessed in males we cannot determine whether the findings apply to females, although a gender specific difference in test performance seems unlikely. About 50% of males enrolled in the trials were identified from a parallel population cohort, and participants were fairly representative of uncircumcised males in the general population of Rakai. (Forty five per cent of eligible uncircumcised males in the cohort volunteered for trial enrolment.) Thus, although the external validity of our findings to other populations cannot be fully defined, it is likely that the problems we encountered may occur in other settings. These observations suggest that there is a need to assess the performance of rapid tests in a variety of settings, and it would be prudent to routinely retest a batch of samples by enzyme immunoassay and western blotting to maintain quality control in programmes using rapid tests.⁴ This is of particular importance given the proliferation of rapid tests, since the performance of various constructs may differ in divergent populations and settings. Furthermore, given the potential social and psychological consequences of providing a false positive HIV diagnosis, samples showing weak positive results on rapid tests may routinely need to be confirmed by enzyme immunoassay and western blotting before the disclosure of results. The proportion of samples yielding weak positive bands was relatively low in our study (5.8%), so retesting of weak positive results by enzyme immunoassay and western blotting would not impose a heavy burden on most programmes. It would, however, require laboratory backup which could lead to delay in disclosure of results to some participants.

In summary, using an algorithm of three rapid HIV tests, we found that 5.8% of results showed weak positive bands on one or more tests and that 94.1% of samples with weak positive results were negative or indeterminate on enzyme immunoassay or western blotting. We conclude that weak positive bands on rapid tests cannot be interpreted as positive in serum from Ugandan populations.

What is already known on this topic Rapid HIV tests provide timely, point of care methods for screening and diagnosis, but interpretation of positive bands is subjective What this study adds Weak positive bands on rapid HIV tests are mainly false positives and should be confirmed by enzyme immunoassay and western blotting before providing a diagnosis

---

Contributors: RHG prepared the manuscript. He is guarantor. FM carried out the analyses. DS and NKS were responsible for the overall management of work in Uganda. TL prepared the data files. FN oversaw the field work. PO was responsible for the HIV assays. GK was the medical officer in charge of the trials. SJR provided guidance on the interpretation of assay results and quality control for laboratory work. MJW was the principal investigator of the trial in HIV positive males and is guarantor for this component of the work.

Funding: The study was supported by the National Institutes of Allergy and Infectious Diseases, Division of AIDS, National Institutes of Health (grant No UO1 AI11171-01-02), the Fogarty International Center (grant No 2D 43 TW000010-19-AITRP), and the Gates Foundation.

Competing interests: None declared.

Ethical approval: This trial was approved by the scientific and ethics committee of the Uganda Virus Research Institute, Entebbe and the National Committee of Science and Technology, Kampala and the Committee on Human Research, Johns Hopkins University, Bloomberg School of Public Health, Baltimore.

References

1. Branson BM. Point-of-care rapid tests for HIV antibody. .www.cdc.gov/hiv/rapid_testing/materials/J_Lab_Med_20031.htm
2. McKenna SL, Muyinda GK, Roth D, Mwali M, Ng'andu N, Myrick A, et al. Rapid HIV testing and counseling for voluntary testing centers in Africa. AIDS 1997;(Suppl 1):S103-10.
3. Wright AA, Katz IT. Home testing for HIV. N Engl J Med 2006;354:437-40.[Free Full Text]
4. De Cock KM, Bunnell R, Mermin J. Unfinished business—expanding HIV testing in developing countries: N Engl J Med 2006;354:440-2.[Free Full Text]
5. Kassler WJ, Alwano-Edyengo MG, Marum E, Biryahwaho B, Kataaha P, Dillon B. Rapid HIV resting with same day results: a field trial in Uganda. Int J STD AIDS 1998;9:134.[Abstract/Free Full Text]
6. Downing RG, Otten RA, Marum E, Biryahwaho B, Alwano-Edyegu MG, Sempala SD, et al. Optimizing the delivery of HIV counseling and testing services: the Uganda experience using rapid HIV antibody test algorithms. J Acquir Immune Defic Syndr Hum Retrovirol 1998;18:384-8.[Web of Science][Medline]

(Accepted 11 April 2007)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Article

The last choice

Douglas Kamerow
BMJ 2007 335: 0. [Extract] [Full Text]

This article has been cited by other articles:

• Everett, D. B., Baisley, K., Changalucha, J., Vallely, A., Watson-Jones, D., Cook, C., Knight, L., Ross, D. A., Mugeye, K., McCormack, S., Lacey, C. J., Jentsch, U., Hayes, R. J. (2009). Suitability of Simple Human Immunodeficiency Virus Rapid Tests in Clinical Trials in Community-Based Clinic Settings. J. Clin. Microbiol. 47: 1058-1062 [Abstract] [Full text]  
• Frizelle, K.L. (2009). A Case for Intercultural Inquiry and Cognitive Justice in the Internationalisation of HIV/AIDS: A South African Example. Journal of Health Management 11: 79-91 [Abstract]  
• Pilcher, C. D., Hare, C. B. (2008). The Deadliest Catch: Fishing for HIV in New Waters. ANN INTERN MED 149: 204-205 [Full text]  
• Eller, L. A., Eller, M. A., Ouma, B. J., Kataaha, P., Bagaya, B. S., Olemukan, R. L., Erima, S., Kawala, L., de Souza, M. S., Kibuuka, H., Wabwire-Mangen, F., Peel, S. A., O'Connell, R. J., Robb, M. L., Michael, N. L. (2007). Large-Scale Human Immunodeficiency Virus Rapid Test Evaluation in a Low-Prevalence Ugandan Blood Bank Population. J. Clin. Microbiol. 45: 3281-3285 [Abstract] [Full text]  
• Madhivanan, P., Krupp, K. (2007). Technological challenges in diagnosis and management of HIV infection in resource limited settings. BMJ 335: 165-166 [Full text]  

Rapid Responses:

Read all Rapid Responses

Rapid assay tests for HIV1 or VCTC which plays important role in Primary health care settings in West Bengal for surveillance, diagnosis & treatment of HIV1?

Professor Pranab Kumar Bhattacharya, et al.
bmj.com, 13 Jun 2007 [Full text]

Re- discordant results from HIV rapid Testing

Olubunmi momoh-Negedu
bmj.com, 18 Jul 2007 [Full text]

Online poll

Find out more>>