Analysis

Monitoring and evaluation of programmes to prevent mother to child transmission of HIV in Africa

BMJ 2007; 334 doi: http://dx.doi.org/10.1136/bmj.39211.527488.94 (Published 31 May 2007) Cite this as: BMJ 2007;334:1143
  1. Richard Reithinger, epidemiologist1,
  2. Karen Megazzini, project director1,
  3. Stephen J Durako, vice president1,
  4. D Robert Harris, senior epidemiologist1,
  5. Sten H Vermund, director2
  1. 1Clinical Trials Area, Westat, Rockville, Maryland, USA
  2. 2Institute for Global Health, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
  1. Correspondence to: R Reithinger rreithinger{at}yahoo.co.uk
  • Accepted 6 May 2007

Many countries are expanding the coverage of programmes to prevent mother to child transmission of HIV. Although the need is unquestionable, Richard Reithinger and colleagues are concerned that without true measures of effectiveness we may not be making the best use of resources

In 2006, an estimated 2.3 million children under 15 years were living with HIV and about half a million babies became infected with HIV before birth, during delivery, or through breast feeding.1 Prevention of mother to child transmission of HIV is therefore a priority for agencies fighting the global HIV epidemic, but many questions remain about the effectiveness of the current programmes. We use the President's Emergency Plan for AIDS Relief2 as an example to examine how programmes to prevent mother to child transmission are monitored and evaluated and to highlight the problems.

Strategy to prevent transmission

Estimates of the efficacy of antiretroviral prophylaxis3 suggest that at least half of the world's children who are at risk of HIV infection might be protected if a mother receives antenatal care, is offered HIV counselling and testing, and, if infected, she and her baby receive prophylaxis. Prophylaxis is the mainstay of the strategy to prevent mother to child transmission.3 Several antiretroviral regimens are recommended in resource constrained settings, although nevirapine (either alone or with other drugs) is usually favoured because it is cheap, easy to administer, rapidly absorbed, and has a long half life.3 Depending on the regimen and the mother's choice of infant feeding, the risk of HIV transmission can be reduced to <2%.3 Whenever feasible, programmes should strive to provide highly active antiretroviral therapy to pregnant women.

Although formula feeding can reduce HIV infection rates among infants,4 it is often not acceptable, feasible, affordable, sustainable, or safe in resource limited settings.5 6 Programmes should provide counselling and support on various feeding options (including exclusive breastfeeding), highlighting the potential benefits and risks of each.7 Other activities include promotion of optimal obstetrical practices; improvement of antenatal, postnatal, and child health services; and treatment of maternal diseases. Community based activities are also often implemented to improve community knowledge of HIV and AIDS and counter negative attitudes to people with HIV.8

Routine monitoring and evaluation tools

Because most countries do not have the money or laboratory resources to determine HIV infection in infants using polymerase chain reaction tests, health facility and patient indicators are generally used to assess the effectiveness of programmes (table 1). The effectiveness of the programme is typically measured in terms of the estimated percentage (change) of HIV infected infants who are born to HIV infected mothers.9 10 However, table 1 shows that effective prevention relies on a cascade of steps. A pregnant woman must receive antenatal care at a centre offering HIV testing. If she consents to testing she must receive her HIV test result and, if infected, receive appropriate prophylaxis. She must then take the prophylaxis during labour and her infant must receive prophylaxis after delivery. Finally, the mother and infant must receive follow-up care to ensure that they have any necessary treatment and that the infant is tested for HIV, typically in the second year of life when HIV antibody testing is reliable.

Table 1

 Indicators for monitoring and evaluation of programme to prevent mother to child transmission of HIV used by the President's Emergency Plan for AIDS Relief

View this table:

The few studies that have attempted to evaluate such programmes show the logistical, managerial, and technical challenges in delivering effective preventive services (table 2).5 11 12 13 14 15 16 17 18 Coverage falls progressively the further women are along the prevention cascade. Thus, 12 months after delivery, only a fraction (19% in one study in Malawi5) of HIV positive mothers who received antiretroviral drugs will attend health services to have their infant tested for HIV. Clearly, this may have lethal consequences for those children who become HIV positive.

Table 2

 Studies monitoring and evaluating the effectiveness of programmes to prevent mother to child transmission of HIV

View this table:

The effectiveness of the programmes is compromised if there is a low coverage at any stage of the cascade. Monitoring and evaluation of each stage are thus critical to identify any weakness in the implementation and to develop strategies for improving coverage where necessary.

Challenges of measuring effectiveness

Relying exclusively on the indicators listed in table 1 has several limitations. Firstly, indicator data are commonly collected from several patient registers in the form of unlinked monthly summary reports.16 Consequently, for example, data on the number of pregnant women who tested HIV positive, received prophylaxis, and then delivered children who also received prophylaxis are not readily available. To obtain linked data, patient registers would need to be thoroughly cross-checked, which would be time consuming and cumbersome without computerisation and planning. The multitude of registers also adds a potential source of human error, with health workers sometimes forgetting to complete them. As a result, different registers may report different denominators.16

Secondly, the reported HIV prevalence rate in pregnant women is typically derived from those accepting voluntary counselling and testing. These women have a lower prevalence than those who refuse testing, probably because women at risk of HIV are less likely to volunteer for testing.11 16

Thirdly, in countries where same day HIV testing is unavailable, 15-67% of women who are tested do not obtain their test results and, hence, may not receive prophylaxis.17 18 19 Even in those countries where rapid testing is available, integration of testing services into antenatal care is essential to ensure that women receive prophylaxis.

Fourthly, and perhaps most importantly, none of the indicators listed in table 1 objectively measure adherence to the prophylactic drugs, and there can be substantial discordance between the proportion of women who agree to prophylaxis and the proportion of women who actually take the drugs. In an extensive surveillance study carried out in Zambia in 2003, only 68% of HIV infected women given a nevirapine tablet during antenatal care to take at the onset of labour had the drug in their umbilical cord blood at delivery.11 Moreover, both the woman and her infant received nevirapine in only 30% of cases despite preventive services being universally available in public antenatal clinics at the time of the evaluation.11 The unexpectedly low level of population coverage highlights the importance of using objective and definitive surrogates to measure the effectiveness of programmes. Once alerted, the Zambian team was able to make improvements at every level of the prevention cascade, increasing coverage as a consequence.

Fifthly, most indicators of programme effectiveness are quantitative rather than qualitative. The quality of services offered at health facilities affects supply and demand for voluntary testing, uptake of antiretroviral drugs, and other activities such as family planning.7 11 16 17 18 Also, current indicators do not capture possible adverse effects of the activities, whether related to the antiretroviral drugs or to the revelation of maternal or child infection status (such as, stigmatisation and domestic violence).

Finally, current indicators fail to evaluate effectiveness in terms of coverage and linkage to care for mothers, infants, and family members. As shown above, rates of postnatal follow-up can be very low.5 11 12 13 14 15 16 17 18 None of the indicators in table 1 capture postnatal follow-up rates so that, for example, the effect of breast feeding can be accurately assessed. Instead, estimates of infant coverage and infection prevented are modelled on nevirapine uptake data or from a limited number of cohort (research) studies.

Where to go from here?

Ideally programmes should carry out sequential prevalence studies or cohort studies in infants. These studies would reliably estimate infant HIV infections under operational conditions rather than extrapolate figures from clinical trial or “model programme” data. Although these studies could certainly be included in larger programmes, they are labour intensive and costly. For example, infants born to HIV infected mothers would have to be followed up for 15-18 months to rule out transmission. The care benefits of proper follow-up that accrue to mother and infant are substantial, so evaluation costs will be linked to service expenditures. Depending on venue and source of funding, ethical mandates to improve the standard of care for study participants would also reduce the generalisability of evaluations of enhanced prevention programmes and follow-up care. Thus, this type of evaluation would represent a best case scenario. Moreover, cohort studies will not yield data about the programme's coverage unless explicitly designed to do so.

One approach to measuring effectiveness is through anonymous screening of umbilical cord blood samples for HIV antibodies. Positive samples would then be tested for nevirapine (an indicator of maternal nevirapine coverage) using high performance liquid chromatography11 or the cheaper but less sensitive thin layer chromatography followed by high performance retesting of samples with negative results.20 Investigators in Zambia have used cord blood analysis combined with chart documentation of infants' receipt of nevirapine to measure true population coverage.11 When combined with patient level indicator data extracted from a woman's antenatal record at delivery, this approach provides an objective measure of effectiveness both for women who participated in the programme and in the broader population of women delivering in public labour wards.

Collection of patient data must be simplified and linked. In the absence of fully computerised, real time collection of patient health data,21 such linkage could be obtained by incorporating a small coded stamp into the medical record of women attending antenatal care (figure). A first generation of such a stamp was used when evaluating nevirapine coverage in pregnant women in Zambia22 and is being completed by health staff at each Zambian programme site during routine antenatal care and labour ward visits. The stamp provides a place to record important data for monitoring and evaluating prevention programmes. After delivery, the medical records are left in the health facilities' maternal and child health department, where the data can be readily entered on to computerised databases for analytical purposes or to aid clinic staff in postnatal outreach of infants born to HIV infected women.

Figure1

Example of data collection stamp for use on antenatal care and labour records. Spaces shown represent where the dates of the given activity are recorded; multiple lines are provided for documentation of counselling or testing during follow-up visits. CR=counselling refused; CA=counselling accepted; TR=testing refused; TA=testing accepted; R=reactive; NR=non-reactive; I=indeterminate; MGN=mother given nevirapine (during antenatal care); MRN=mother refused nevirapine; MIN=mother ingested nevirapine (at delivery); MRIN=mother refused to ingest nevirapine; IGA=infant given antiretroviral drugs; INGA=infant not given antiretrovirals; IFB=infant feeding breast; IFR=infant feeding replacement

Use of a data stamp, combined with the umbilical cord blood analysis, permits collection of anonymous linked maternal and infant data—for example, by combining data on maternal programme indicators, maternal population drug coverage, and data on infant population drug coverage. This is crucial when attempting to identify shortcomings in the implementation of programmes and to estimate the proportion of children who will become infected with HIV. Moreover, inclusion of a modified version of the stamp in an infant's immunisation card may remind health staff of the need to test the infant for HIV and thereby improve postpartum follow-up rates (at least among those who return for immunisations).

The data stamps must be coded to protect the confidentiality of the mother and infant. Confidentiality can be violated only if a healthcare provider reveals the meaning of the stamp to a third party. This is analogous to a medical provider revealing the contents of a patient's medical record. In fact, a coded stamp would be more secure than standard medical record notes because it could not be interpreted without an explanation, whereas a third party might read records successfully without help. Nevertheless, before stamps are widely implemented we need pilot research to assess their feasibility and acceptability in a given clinical and cultural context.

We recognise that the above approach may be unsuitable for areas where a small proportion of women deliver in public labour wards (such as Haiti), where there is insufficient laboratory capacity to process and store cord blood specimens, and where programmes lack funds for high performance liquid chromatography analysis. The cost of analysing each specimen is around $50 (£25; €37),20 although high volumes should reduce costs. Nevertheless, a coded stamp could still enhance monitoring and evaluation efforts by allowing for simplified, periodic extraction of data. These data could verify the data presented in monthly monitoring reports and identify any shortfalls in the programme.

For programmes that use more complex antiretroviral regimens (such as short course zidovudine plus single dose nevirapine or nevirapine-containing highly active antiretroviral therapy), the absence of nevirapine in the cord blood would not necessarily imply a woman was not at least partly covered, assuming she adhered to the zidovudine portion of the regimen. Although zidovudine can be detected in cord blood,23 it is a less perfect surrogate for infant infection than nevirapine because it has a shorter half life and detection in the cord blood does not prove full compliance with the longer zidovudine regimen.

Other methods of measuring the effectiveness of these programmes have been reported. In South Africa, for example, facility based case finding approaches have been used to identify infants born to women accessing programme services,24 and mathematical modelling has also been used.25 However, these approaches do not separate each element of the prevention cascade, making it hard to pinpoint a programme's failures and successes.

Conclusion

Currently, less than 10% of HIV infected pregnant women receive antiretroviral prophylaxis.26 Scaling-up of programmes to prevent mother to child transmission of HIV remains a huge challenge. Programmes supported by organisations with a high media profile, such as the President's Emergency Plan for AIDS Relief, are often subject to substantial political and public pressure to report successes rather than failures. Given the amount of public and private sector investments into these programmes and the programmes' potential effect on a country's societal and economic development, we believe that rigorous monitoring and evaluation of operational programmes is a moral and ethical imperative. Evaluation is essential to identify shortcomings in the programme and to conceptualise approaches to improve services. This, in turn, will improve a programme's cost effectiveness and long term sustainability and save the most infant lives.

Summary points

  • Programmes to prevent mother to child transmission of HIV are an important part of global AIDS initiatives

  • Data on the effectiveness of these programmes are scarce

  • Current indicators to monitor and evaluate the programmes are inadequate

  • Comprehensive indicators that monitor all stages of the prevention cascade are urgently needed

Footnotes

  • ARTICLE
  • We thank James Korelitz and the peer reviewers for insightful comments on the manuscript. We also thank Elizabeth Stringer, Moses Sinkala, and Jeffrey Stringer for allowing us to include a stamp similar to the one used in Zambia.

  • Contributors and sources: RR, KM, SJD, DRH, and SHV have all worked extensively in clinical and operational infectious disease research. The article arose following extensive discussions and planning for activities to monitor and evaluate programmes to prevent mother to child transmission of HIV. Part of this process included a subject review based on comprehensive literature search of medical databases as well as non-medical search engines. RR, KM, and SHV cowrote the manuscript; DRH and SJD contributed conceptually. All authors reviewed the manuscript. RR and SHV are the guarantors.

  • Competing interests: None declared.

  • Provenance and peer review: Not commissioned, externally peer reviewed.

References