Randomised controlled trial comparing effectiveness of touch screen system with leaflet for providing women with information on prenatal testsCommentary: Evaluating electronic consumer health material

BMJ 2000; 320 doi: http://dx.doi.org/10.1136/bmj.320.7228.155 (Published 15 January 2000)
Cite this as: BMJ 2000;320:155

Abstract

Objective: To compare the effectiveness of touch screen system with information leaflet for providing women with information on prenatal tests.

Design: Randomised controlled trial; participants allocated to intervention group (given access to touch screen and leaflet information) or control group (leaflet information only).

Setting: Antenatal clinic in university teaching hospital.

Subjects: 875 women booking antenatal care.

Interventions: All participants received a leaflet providing information on prenatal tests. Women in the intervention arm also had access to touch screen information system in antenatal clinic.

Main outcome measures: Women's informed decision making on prenatal testing as measured by their uptake of and understanding of the purpose of specific tests; their satisfaction with information provided; and their levels of anxiety.

Results: All women in the trial had a good baseline knowledge of prenatal tests. Women in the intervention group did not show any greater understanding of the purpose of the tests than control women. However, uptake of detailed anomaly scans was significantly higher in intervention group than the control group (94% (351/375) v 87% (310/358), P=0.0014). Levels of anxiety among nulliparous women in intervention group declined significantly over time (P<0.001).

Conclusions: The touch screen seemed to convey no benefit over well prepared leaflets in improving understanding of prenatal tests among the pregnant women. It did, however, seem to reduce levels of anxiety and may be most effective for providing information to selected women who have a relevant adverse history or abnormal results from tests in their current pregnancy.

Randomised controlled trial comparing effectiveness of touch screen system with leaflet for providing women with information on prenatal tests

  1. Wendy Graham (w.graham{at}abdn.ac.uk), directora,
  2. Pat Smith, honorary lecturerb,
  3. A Kamal, research fellowa,
  4. A Fitzmaurice, medical statisticiana,
  5. N Smith, consultant obstetricianb,
  6. N Hamilton, directorc
  1. a Dugald Baird Centre for Research on Women's Health, Department of Obstetrics and Gynaecology, Aberdeen Maternity Hospital, Aberdeen AB25 2ZL
  2. b Department of Obstetrics and Gynaecology, Aberdeen Maternity Hospital
  3. c Computer Assisted Learning Unit, Faculty of Medicine and Medical Sciences, University of Aberdeen, Aberdeen AB25 2ZD
  4. School of Public Policy, University College London, London WC1E 7HN
  1. Correspondence to: W Graham

    Key message

    • Throughout the NHS, efforts are being made to evaluate traditional methods of conveying information to patients, such as leaflets, and to develop and assess new approaches

    • This study compared the effectiveness of a touch screen system with a well designed leaflet at providing women with information on prenatal tests

    • The touch screen conferred no additional benefits over the leaflet when applied to an unselected population of pregnant women

    • Nulliparous women showed reduced anxiety levels after access to the touch screen, but further research is needed on the measurement of anxiety during pregnancy

    Introduction

    Informed choice has been an important component of health care in the United Kingdom for almost a decade. 1 2 One area in which this principle has long been applied is prenatal testing. Specific initiatives have been launched to promote women's awareness of best evidence on the effectiveness of specific tests and active participation in decisions about their care.3 The number of conditions for which screening is offered continues to grow rapidly, and women consequently face increasingly complex decisions.4 Studies have illuminated many dimensions to this complexity, including the professional and organisational barriers to informed choice,3 the huge variations in the scope and accuracy of information given,5 and the problem of receiving unsolicited and unanticipated information from screening.6 What is also clear is that informed choice depends on an effective partnership between the user, the provider and the communication medium.

    Throughout the NHS, efforts are being made to evaluate traditional methods of conveying information, such as leaflets, and to develop and assess new approaches. This paper reports the results of a recent trial to evaluate a touch screen information system for providing information on prenatal tests to women. The primary hypothesis was that access to the system would improve women's informed decision making regarding prenatal tests over and above that achieved by access to an information leaflet alone.

    Participants and methods

    Study population and setting

    Women attending a booking appointment at one of the five antenatal clinics at Aberdeen Maternity Hospital from April 1997 until January 1998 were invited to participate. This large teaching hospital had 4734 deliveries in 1997. The five clinics encompassed women with high and low risk pregnancies. We obtained consent for the trial from the Grampian Health Board and University of Aberdeen Joint Ethical Committee.

    Assignment

    Women were initially contacted by post and, along with their booking appointment, were given a baseline questionnaire to complete before coming to clinic. At their booking, the women were approached by a research midwife for their verbal consent to participate in the trial. Completed baseline questionnaires were collected, or, if necessary, a further copy was given for the woman to answer at the clinic. Once they had given consent, the women were randomised on a 1:1 ratio into the intervention (touch screen and information leaflet) or control (leaflet only) group. Allocation was made by the research midwife opening consecutive, sealed, opaque envelopes. The randomisation schedule was prepared by AK, who was not involved in recruiting.

    Interventions

    Both groups of women were given the information leaflet on prenatal tests developed specifically for the trial. Information leaflets already available in the antenatal clinic gave similar information to that provided by the touch screen information system, but none of these matched its scope and detail. The touch screen had been developed by three of the investigators (PS, NS, NH) over the previous two years. 7 8 It was a menu driven system with information organised into eight main topics and included video clips and voice overs. Patients accessed the information by means of a touch screen display (fig 1), which is operated by pressing the display with a finger, that was located in the antenatal clinic waiting area. Use of the touch screen was limited to women in the intervention group by means of a password. Privacy in using the system was enhanced by the availability of microphone headsets.

    Fig 1

    Menu display of options available on touch screen system. (Other views of system are available on the BMJ's website)

    Outcomes measured

    The primary outcome assessed in the trial was women's informed decision making on prenatal testing, as measured by their uptake and understanding of the purpose of five tests (ultrasound scan at booking, serum screening, detailed anomaly scan, amniocentesis, and chorionic villus sampling). Secondary outcomes included the women's satisfaction with the information they received and their anxiety levels.

    To assess the women's understanding of prenatal tests we gave them questionnaires for self completion at baseline, at around 16 weeks' gestation (after they had accepted or rejected serum screening), and at around 20 weeks' gestation (after they had accepted or rejected a detailed anomaly scan). The questionnaires asked a similar range of questions and were developed from validated schedules of previous studies 9 10 and two focus group discussions. The baseline questionnaire was returned by women at recruitment, whereas the other two were sent and returned by post. We sent one reminder after two weeks if we had not received a reply. The women's understanding of prenatal tests was assessed with multiple choice questions, which asked them to look at eight principal reasons for testing and to indicate which of six possible tests were conducted for each purpose (see fig 2). The responses were analysed as a dichotomous variable (correct or incorrect). We used the Spielberger state-trait anxiety inventory (STAI)11 to assess anxiety levels, with the A-state component administered in all three questionnaires and the A-trait included in the first and last questionnaires.

    Fig 2

    Extract from questionnaire used to assess women's understanding of prenatal tests

    Statistical analysis

    We estimated that we needed a sample size of 1000 women, 500 in each arm, to give 90% power to detect at the 5% significance level a difference of 10% in the proportion of women with an understanding of the reasons for serum screening indicative of informed decision making. This calculation assumed a baseline of 60% of the sample being informed9 and allowed for 5-10% to drop out.

    All data from the questionnaires were entered into the trial database. We conducted quality control checks on a random sample of 10% of the questionnaires. Statistical analysis was by SPSS12 on an intention to treat basis. We used independent and paired t tests to compare continuous variables after checking for normal distribution. We compared the outcome variables for the two groups using the 2 test and McNemar's test for paired data. We give significance levels of differences, and 95% confidence intervals. We used logistic regression to assess understanding of prenatal tests, after adjusting for important confounding factors such as parity and education.

    Results

    Recruitment—In total, 1477 women were identified as potential participants, of whom 1050 were found to be eligible and consented to take part (fig 3). Of the 427 who did not participate, 147 were ineligible and 280 did not consent. Of the 1050 participants, 670 (64%) returned all three questionnaires, 743 (71%) responded to only the first two questionnaires, and 710 (68%) responded to only the first and last questionnaires. Among the 875 women included in the baseline analysis, there were no significant differences between the characteristics of the intervention and control groups (see extra table on BMJ's website for details), and there were no major differences in the characteristics of the 175 women who did not return the baseline questionnaire and those who did. There were no significant differences between the two groups with regard to the characteristics of the women lost to follow up, nor the reasons for or rate of loss.

    Fig 3

    Progress of participants through trial

    Use of touch screen and information leaflet—Similar numbers of women in the intervention and control groups reported reading the information leaflet fully (218/380 (57%) and 234/381 (61%) respectively), and 12% in both groups indicated that they had only glanced at it. With regard to the touch screen, 32/374 (9%) women reported that they had never used it, and 342 women (91%) had used it at least once.

    Views on prenatal testing—In the baseline questionnaire similar proportions of women in both groups reported that they would accept most tests if offered. The highest level of acceptance was for detailed anomaly scans (98% (405/415) of intervention group, 97% (381/394) of control group), and the lowest acceptance for amniocentesis (40% (160/399) and 42% (163/386) respectively). The only difference in the second questionnaire (after the women had been given information on prenatal testing) was that the acceptability of amniocentesis had increased significantly in both groups (McNemar's test, P=0.030).

    Uptake of tests—Fewer than 1% of women did not receive any prenatal tests. Table 1 shows that the only significant difference between the two groups was that more women in the intervention group underwent detailed anomaly scanning (94% v 87%, P=0.0014).

    View this table:
    Table 1.

    Prenatal tests undergone by pregnant women randomised to control group (information leaflet on prenatal tests) or intervention group (leaflet plus touch screen information system). Values are numbers (percentages) unless stated otherwise

    Understanding of prenatal tests—In the baseline questionnaire women showed a high level of understanding of which prenatal tests were carried out for specific reasons, with the exception of chorionic villus sampling. Comparisons of baseline responses with those given by the same women in the second questionnaire showed significant improvements in knowledge for both groups (table 2). The logistic regression confirmed this, with no apparent greater gain in knowledge among women in the intervention arm.

    View this table:
    Table 2.

    Knowledge of pregnant women about which prenatal tests are undertaken “to check the development of the baby,” before and after they were given information leaflet on prenatal tests (control group) or leaflet plus access to touch screen information system (intervention group). Values are numbers (percentages) unless stated otherwise

    Satisfaction with information—Both groups reported high levels of satisfaction with the information leaflet, with over 95% indicating that they would recommend the leaflet to other pregnant women. A similar percentage of the women in the intervention arm reported that they would recommend the touch screen, and over a third (132/347) indicated a preference for the touch screen over the leaflet, while a quarter (91) indicated no preference, a fifth (72) preferred the leaflet, and the rest (52) were “not sure.”

    Anxiety levels—Table 3 shows the results of the Spielberger state-trait anxiety inventory. Compared with the results in the baseline questionnaire, both the A-state and A-trait components of the inventory measured in the third questionnaire had declined significantly in the intervention group, mainly among nulliparous women.

    View this table:
    Table 3.

    Anxiety levels of pregnant women before and after they were given information leaflet on prenatal tests (control group) or leaflet plus access to touch screen information system (intervention group). Values are mean scores for Spielberger state-trait anxiety inventory unless stated otherwise

    Discussion

    Antenatal screening is one of the most intensively researched subjects with regard to information for women and their informed choice.13 The principles of equity and quality, so well accepted in screening programmes, are now advocated for the process of giving information,4 but researchers have shown that one of the most serious obstacles to this is health professionals providing the information. 3 14 Touch screen information systems have the potential to reduce this barrier by providing consistent information and by being patient driven, thus enabling pregnant women to control information overload.15 Like all new technologies, however, they should be subject to rigorous evaluation.

    The touch screen evaluated in this trial conferred no additional benefit to that provided by the more traditional method of an information leaflet. It could be argued that only small effects could be expected in well educated pregnant women whose baseline level of knowledge and “compliance” with prenatal testing are already high. As found in other studies, 9 16 both groups of women in our study showed improvements in their knowledge, albeit from a high starting point, which highlights women's receptiveness to information given during pregnancy and, thus, the importance of making it appropriate and reliable.

    Interestingly, we observed a significant increase in the uptake of detailed anomaly scanning in the intervention group, and other studies comparing information provided by different methods have noted differential uptake of ultrasonography. 15 17 In the case of our touch screen, the use of video clips to show what can be gained from a detailed scan might have helped to reassure women and increase their desire for this investigation.

    Women's anxiety

    Our trial involved an unselected group of pregnant women, and in such a predominantly healthy population we found, as have other researchers,15 that the information provided did not raise anxiety. In fact, one apparent benefit of the touch screen was to reduce levels of anxiety. The mean score for the A-state component of the Spielberger state-trait anxiety inventory declined significantly, but we also found a significant fall in the mean score for the A-trait component, which is supposed to remain stable over time.11 Other studies have noted this instability in the A-trait when the inventory is applied during pregnancy, 18 19 and this effect warrants further investigation. In particular, we need to find the extent to which reduced anxiety could be replicated in a selected group of women with a previous adverse outcome or an abnormal finding from prenatal screening.

    Limitations of study

    Our findings should be interpreted in the light of two limitations: loss to follow up and the potential for contamination between groups. As with most longitudinal data collection, there was attrition of the number of participants from the point of recruitment to completion of the trial. Although this reduces the statistical power of the study, sub-group analysis showed no major differences in the characteristics of those women who did or did not complete all three questionnaires. As the participants were attending the same antenatal clinics it was not feasible to totally eliminate the risk of contamination between the intervention and control groups, with controls possibly observing the touch screen while it was being used. However, as we had introduced a password system for accessing the touch screen and provided microphone headsets, together with the need to stand right in front of the screen in order to see the images, contamination is likely to have been minimal.

    Future studies

    Further evaluations of this technology should also consider costs. The touch screen evaluated in this trial incurred initial development costs in 1994-5 of about £25 000, and additional costs can be envisaged in terms of maintenance of hardware and updating of information. A commitment to providing evidence based information must remain the major rationale for any future investment in computer technology.

    Acknowledgments

    We thank Ms Iciar Frade, Ms Jilly Ireland, Ms Gillian Payne, the medical and midwifery staff of Aberdeen Maternity Hospital, the secretarial staff of the Dugald Baird Centre, and, of course, the women who participated in the study.

    Contributors: WG identified the need for a trial and secured the funding; designed the trial; guided the collection, analysis, and interpretation of data; and was primarily responsible for writing the paper. PS had the original idea for a touch screen information system, organised support for its development, coordinated all aspects of the technical content of the system, advised throughout the conduct of the trial, and contributed to the interpretation of findings. AK coordinated all aspects of the conduct of the trial, including the design of data collection instruments, data coding, database design, and data entry, and undertook the preliminary analysis of results. AF advised on all statistical issues in the conduct of the trial, undertook the data analysis, and contributed to the writing of the paper. NS contributed to the design of the technical content of the touch screen, advised on the design and conduct of the trial, and assisted in the interpretation of findings. NH was responsible for the technical development of the touch screen and its maintenance throughout the trial, advised on the conduct of data collection, and contributed to the interpretation of the results. PS, AK, NS, and NH provided inputs to the revision of the paper. WG is guarantor for the study.

    Footnotes

    • Funding The trial was funded by the NHS Executive programme in evaluating methods to promote the implementation of research and development (project No 4-21).

    • Competing interests NH is a non-executive director of Cognetic Creations, which markets the touch screen technology used in the information system.

    References

    1. 1.
    2. 2.
    3. 3.
    4. 4.
    5. 5.
    6. 6.
    7. 7.
    8. 8.
    9. 9.
    10. 10.
    11. 11.
    12. 12.
    13. 13.
    14. 14.
    15. 15.
    16. 16.
    17. 17.
    18. 18.
    19. 19.

    Commentary: Evaluating electronic consumer health material

    1. Jeremy Wyatt (jeremy.wyatt{at}ucl.ac.uk), director, Knowledge Management Centre
    1. a Dugald Baird Centre for Research on Women's Health, Department of Obstetrics and Gynaecology, Aberdeen Maternity Hospital, Aberdeen AB25 2ZL
    2. b Department of Obstetrics and Gynaecology, Aberdeen Maternity Hospital
    3. c Computer Assisted Learning Unit, Faculty of Medicine and Medical Sciences, University of Aberdeen, Aberdeen AB25 2ZD
    4. School of Public Policy, University College London, London WC1E 7HN

      Consumer health material consists of specific content presented in a variety of formats and is of increasing importance to health services.1 Electronic material—including computer programs, web resources, and conventional or interactive video—allows a much wider range of formats for presenting content to consumers. Changing the format while retaining the same content may alter decisions,2 so such electronic systems need to be evaluated.

      The best research design depends on the question. If we are interested in format then a randomised trial is needed in which controls receive the same content presented in the usual format—paper or verbal. If the research question concerns the impact of improved content, controls should receive the same format but with normal content. To answer a pragmatic question (Which material is better?) controls should receive the best paper leaflet, but no inference can then be drawn about the relative contributions of improved content or electronic format.

      When information systems are evaluated it is often necessary to take a broader view. For example, in this trial, control patients could have been influenced by the touch screen system if they borrowed a password, looked over the shoulder of a woman using the system, or chatted to women in the intervention group in the antenatal clinic or class. The solution to such “contamination” is a cluster randomised trial3—randomising clinics, health centres, or districts rather than patients. This also avoids the need for passwords and unreliable randomisation with sealed envelopes.4

      In this trial, baseline knowledge was high (47% of participants had received higher education) so only minor improvements could ever be shown—a ceiling effect.5 Evaluators should seek out a group who are not so well informed but are able to use the novel information system. Although it may seem necessary to balance study groups for baseline knowledge, cluster designs make this difficult and large studies make it unnecessary.6

      In this trial an impressive 91% of women in the intervention group used the touch screen system, but in most other trials this figure will be lower. Investigators should avoid comparing outcomes in those who did use the system with those who did not. As in this trial, analysis should be by “intention to provide information.” However, only 70% of participants were followed up, leaving 148 who used the touch screen but whose knowledge or attitudes may conceivably have worsened. Investigators should vigorously pursue all participants and aim for minimum follow up of 80% by keeping questionnaires short and making only essential measures.

      The benefits of the touch screen system in this trial may have been underestimated by contamination and high baseline levels of knowledge. Graham and colleagues rightly state that: “Like all new technologies, these devices should be subject to rigorous evaluation.” With limited evidence of benefit for these expensive tools over well designed leaflets, they seem to fit best into the National Institute for Clinical Excellence (NICE) category C: for NHS use only in the context of rigorous research studies.7

      Footnotes

      • website extra: An extra table and figures appear on the BMJ's website www.bmj.com

      References

      1. 1.
      2. 2.
      3. 3.
      4. 4.
      5. 5.
      6. 6.
      7. 7.

      THIS WEEK'S POLL