Measuring later health status of high risk infants: randomised comparison of two simple methods of data collection
BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7324.1276 (Published 01 December 2001) Cite this as: BMJ 2001;323:1276- David Field (dfield{at}uhl.trent.nhs.uk), professor of neonatal medicinea,
- Elizabeth S Draper, senior research fellowb,
- Melanie J Gompels, research midwifec,
- Colin Green, health economistd,
- Ann Johnson, developmental paediatriciane,
- David Shortland, consultant paediatricianf,
- Mitch Blair, senior lecturer in child healthg,
- Bradley Manktelow, statisticianb,
- Caroline R Lamming, research health visitorb,
- Catherine Law, senior lecturerh
- a Department of Child Health, University of Leicester Medical School, Leicester LE1 6TP
- b Department of Epidemiology and Public Health, University of Leicester Medical School
- c Wessex Institute for Health Research and Development, University of Southampton, Southampton SO16 7PX
- d Medtap International, London W1Y 1RL
- e National Perinatal Epidemiology Unit, Institute of Health Sciences, Oxford 0X3 9DU
- f Poole General Hospital, Dorset BH15 2JB
- g Imperial College School of Medicine, London SW7 2AZ
- h Medical Research Council Environmental Epidemiology Unit, University of Southampton, Southampton General Hospital, Southampton SO16 6YD
- Correspondence to: D Field
- Accepted 24 August 2001
Abstract
Objective: To test two methods of providing low cost information on the later health status of survivors of neonatal intensive care.
Design: Cluster randomised comparison.
Setting: Nine hospitals distributed across two UK health regions. Each hospital was randomised to use one of two methods of follow up.
Participants: All infants born ≤32 weeks' gestation during 1997 in the study hospitals.
Method: Families were recruited at the time of discharge. In one method of follow up families were asked to complete a questionnaire about their child's health at the age of 2 years (corrected for gestation). In the other method the children's progress was followed by clerks in the local community child health department by using sources of routine information.
Results: 236 infants were recruited to each method of follow up. Questionnaires were returned by 214 parents (91%; 95% confidence interval 84% to 97%) and 223 clerks (95%; 86% to 100%). Completed questionnaires were returned by 201 parents (85%; 76% to 94%) and 158 clerks (67%; 43% to 91%). Most parents found the forms easy to complete, but some had trouble understanding the concept of “corrected age” and hence when to return the form. Community clerks often had to rely on information that was out of date and difficult to interpret.
Conclusion: Neither questionnaires from parents nor routinely collected health data are adequate methods of providing complete follow up data on children who were born preterm and required neonatal intensive care, though both methods show potential.
What is already known on this topic
What is already known on this topic Outcome of neonatal intensive care should include later health status not just early mortality
Although these data are commonly sought, for various reasons no existing routine system currently delivers the information for ≥95% of the population (95% representing the minimum acceptable standard)
Running one-off studies to gain later follow up data is difficult and costly
What this study adds
What this study adds Potentially these data could come from parents but to reach 95% ascertainment perhaps 5-10% of parents would require help and support to provide information
Existing data flows may be able to provide the required information if the timing of routine reviews and methods of data recording were harmonised across the United Kingdom
The costs attached to introducing such a system seem to be low
Introduction
The speciality of neonatal intensive care has developed over the past 30 years. This period has seen a rapid growth in provision of care and dramatic improvements in the survival of premature infants.1 However, these developments have been accompanied by concern that the falling mortality may have been achieved at the cost of high rates of disability in survivors.
Information on late morbidity in survivors of neonatal intensive care is needed by several groups of people: by parents, so that they can understand the possible consequences of survival in their baby and take informed decisions about their child's care; by the clinical team, for sharing with parents, for evaluating their service, and for research; by commissioners of neonatal care and other services for children, so that they can make informed decisions and plans; and by the general public, so that they can take part in an informed debate on priorities in health care.
Although official reports since 1992 have highlighted the need for neonatal units and health authorities to collect information on later morbidity,2–5 it remains generally unavailable on a population basis or outside specialist centres. However, there is broad agreement on the data that should be collected and on the features of systems to collect such data.6 Such systems should be simple, standardised between units, based on existing data collection systems (and hence be capable of being implemented at little additional expenditure), and capable of achieving high levels of ascertainment.6 We carried out a pragmatic cluster randomised controlled trial to compare two approaches to the collection and collation of information on preterm infants who had required neonatal intensive care and had survived to 2 years of age. Both systems had the potential to fulfil these criteria. One method was based on a parental assessment of the child's health at 2 years. The other method relied on collation and review of clinical information collected as part of routine service delivery.
Methods
We collected information on a group of babies at high risk who had been admitted to the neonatal units of the nine collaborating hospitals in the former Trent and Wessex NHS regions over one calendar year (1997). All units provided intensive care, but this was a bigger component of the workload of the large units. Babies were eligible for inclusion in the study if they were born at ≤32 completed weeks of gestation (that is, up to and including 32 weeks and 6 days) and if their mothers lived in the health authority in which the hospital was located. A hierarchical dating algorithm on the basis of last menstrual period, ultrasound scanning, and other clinical information was used to assess gestation.
Before the start of the study we randomised neonatal units to one of the two intervention arms. We used minimisation7 to ensure that the two arms of the trial were about equal by region, city (Leicester and Nottingham, which each had two participating neonatal units), and size of unit (Wessex had three small and two large units; all units in Trent were large). The assignment of units within each region was performed by a statistician who was not part of the study team using tables of random sampling numbers. The identity of neonatal units was concealed in sealed envelopes.
Information collected on all babies, regardless of intervention arm, was based on the Oxford minimum dataset.6 The Oxford dataset provides a framework for describing health status at 2 years of age (corrected for gestation at delivery) focusing particularly on major health problems—for instance, if the child is blind or has severe developmental delay. It also includes basic demographic and clinical information about the child's early clinical management. All babies in the study were “flagged” with the NHS central registry, which allowed us to be aware of children who had moved or died and hence enabled us to avoid contacting parents whose child had died after discharge from the neonatal unit.
Interventions
The essence of both interventions was that a small amount of clerical time was funded to enable follow up of eligible babies by using a core protocol and standardised data collection materials but with considerable flexibility to tailor activity to local circumstances. The study was coordinated on a regional basis, with one coordinator in Trent and one in Wessex. To prevent the coordinators becoming part of the intervention, their role with clerks was restricted to training, developing the local system, and, thereafter, triggering requests for information.
Parental assessment intervention—Before the infant was discharged from the neonatal unit, the clerk based in the unit asked parents if they would take part in the study. When agreement was obtained, the clerk noted basic demographic and clinical details about the baby6 and provided parents with supplementary pages to insert into their child health record. These pages contained forms for parents to complete with their assessment of the child's health at 2 years (corrected for gestation). The parents kept the child health record, which was marked with a study sticker, and the baby's general practitioner and health visitor were notified of the family's participation in the study. After discharge the unit clerk kept occasional informal contact with parents through birthday and Christmas cards to the child. On each contact, a reply paid card was enclosed for the parents to inform the clerk of receipt and any changes of address. At 2 years corrected age, the clerk wrote to the parents, inviting them to complete and return the supplementary pages in the child health record. Parents were free to ask for help in doing this, including help from their general practitioner or health visitor.
Community review intervention—Before the infant was discharged from the neonatal unit the clerk obtained consent and basic descriptive information, as described above. This information was then passed to a designated clerk in the community child health department. Over the next two years, the community based clerk developed and maintained a file of information on each child, using whatever sources of clinical information were routinely available. These might include hospital discharge summaries, information from routine child health surveillance, and outpatient letters. If the family moved out of the area, the clerk tried to obtain appropriate information from services in the new area. When the child reached 2 years corrected age, the clerk was asked to collate and review the information and complete a customised form to record equivalent items of information to those obtained by parental assessment.
Procedure
Although our primary aim was to test two different methods of data collection, we thought it was necessary to show that the items of information we collected were valid. To do this, we selected a 10% sample of children in each method by taking every child recorded as having a major health problem at 2 years and every fifth child who was reported as not having a major health problem, starting with births from 15 April 1997, until a quota of 47 children had been achieved. These children were visited at home within six weeks of their 2nd birthday (corrected age) by one of the study coordinators (either a health visitor or a neonatal nurse). The coordinator, who had not seen the information already recorded for the child at 2 years, reviewed the same areas of development by interview of the parents and assessment of the child. She then checked for any discrepancies between her information and that obtained in either intervention arm and attempted to obtain further information from the parents to resolve discrepancies.
We chose a sample size of 233 children in each arm of the trial to give 90% power, at the 5% significance level, to detect a 15% difference between the two groups in the proportion of children having outcome data at 2 years corrected age. Previous work with routine systems has shown that outcome data are available for about 80% of 2 year old children.8 We hypothesised that in this investigation one method might be similar to this level and the other might show an increase to 95%. We considered 95% as the target in terms of achieving a representative sample of children at 2 years.9 The calculation of sample size was based on the conservative assumption of four clusters within each arm and a maximum intraclass correlation of 0.025.10
To ensure the accuracy of the database we used a double data entry method. Data were stored on an Access database and analysed with SAS, version 8.0. We carried out a pragmatic analysis for the trial with a primary outcome of “an ability to ascertain the overall outcome of a child (at 2 years corrected age) into one of two categories: normal or severe functional loss.” To allow for the cluster randomised design we calculated the proportion of positive responses for each outcome of interest at each hospital and used the results as summary statistics for each hospital. We then compared the two groups using these summary statistics by weighted two sample t tests.11
We also carried out a health economic appraisal. Resources used within the data collection process consisted primarily of staff costs but additional variable resource items, such as services of the Office for National Statistics and consumables, were also included. To assess the additional staff input required in the data collection process we carried out an exercise on workload for data collection in each of the three years that the study was running. Clerks were asked to keep a diary of the time they spent working on the study. Study coordinators completed a questionnaire on their time input to the study, separating out tasks specific to the study (research) and ongoing tasks that formed part of the data collection process.
We assessed the value of staff time using standard pay scales applied to ward clerks, community clerks, and health visitors. 12 13 An additional overhead of 24% was applied to the ward/community clerk pay scale to cover employment costs (employers' national insurance contributions and superannuation) and standard overheads (for example, office space and equipment).
We obtained ethical approval from the eight local research ethics committees relating to the participating neonatal and community child health services.
Results
We recruited 236 infants to each arm of the study (table 1). There were no significant differences between the two arms in terms of clinical characteristics. Of those infants eligible for recruitment, two in each arm were not approached because of a failure of procedures. Three families refused to join the parent questionnaire arm of the study, and in two of these cases this seemed to be because of language difficulties. Seventeen families declined to take part in the community follow up arm. In most cases no reason was stated, but some families expressed concern that the study might provide information to social services.
Four children in the community follow up arm died between recruitment and a corrected age of 2 years. None of the children in the parental questionnaire arm of the study died. As outcome in these children was known we included them in the appropriate numerators and denominators.
Some information was obtained from parents of 214 (90.7%) children, while community clerks reported on 223 (94.5%). The 214 responses from parents were largely complete. In each of the nine fields of development covered by the Oxford minimum dataset, health status was described in no less than 210 cases. In contrast, community returns showed more variation, particularly in relation to motor development (193 complete) and communication (197 complete). As a result of these differences complete data were available for a far higher proportion of children in the parental arm (85.2%, 95% confidence interval 76.0% to 94.3% v 67.0%, 42.9% to 91.0%), although in 60 of the 65 cases in the community arm where data were missing this was limited to just one or two fields of development. Most of these infants were from just one centre.
Table 2 shows a summary comparison of the two methods. Percentages relate to ascertainment in relation to the 236 infants recruited to each arm. The proportion of returned questionnaires, with details of their completeness, is provided for each method with 95% confidence intervals, which are adjusted for the cluster design.
Accuracy
More of the parents' questionnaires had all fields completed compared with the community questionnaires (difference 18%, P=0.048). Of the 214 questionnaires returned by parents, only 51.4% of were returned, as requested, within six weeks of their child reaching a corrected age of 2 years. This compared with 71.2% in the community arm. Data from the community arm were often not current, with clerks having access only to information about the child from the last time he or she was seen. Comments about the child's abilities were based on information recorded before 18 months of corrected age in 28% of cases. Because of the timing of routine screening this was a particular problem in relation to vision. However, even after we excluded this item the proportion of variables reported on the basis of data before 18 months of corrected age was still 22%. Information was retrieved from various sources available to the clerks. However, about two thirds was obtained either by contacting the health visitor or by reviewing the records of earlier assessments held on the community database.
From data returned by parents we identified 26 children (11.0%) with severe functional loss in one or more fields, while for 105 (44.5%) there were no concerns at all. Information obtained by community surveillance produced figures of 20 children (8.6%) with severe functional loss in one or more fields and 98 (42.2%) with apparently normal health.
Validation
Study coordinators visited and assessed 47 children to determine whether health status had been correctly assigned. All 23 for whom information had been provided by parents were found to have been correctly allocated by parental report (eight had severe functional loss, 15 were normal). Of the 24 children assessed from their community records (10 with reported severe functional loss, 14 normal), five were found to have been incorrectly allocated by the community clerks. All five were children recorded as having severe functional loss when in fact they were normal. In three cases this occurred because the clerk completed the form with data about development that had been obtained when the child was much younger and the record had not been updated in relation to later progress. In two cases the records were ambiguous in terms of the child's health.
Financial analysis
The costs of implementing the two approaches were an average of £2271 per centre (range £1520-£3170) with the parental questionnaire and £3709 (£2430-£4047) with the community review. The variation by centre largely reflected the working practices of the clerks involved, with marked differences in the time spent on each case. The equivalent figures for average cost per case recruited were £37 (£25-£52) for the parental questionnaire and £61 (£40-£67) for the community review.
Discussion
Several reviews and inquiries have commented on the importance of information about long term health outcomes for children who receive neonatal intensive care. 3 4 14 While this is self evident in relation to the children concerned, the comments have generally been based on the wider importance of such information. “Quality care” in relation to neonatal care should equate with high rates of survival among children who need this type of support and who then go on to function normally in society. This type of information would clearly be of interest to parents, who want to know not just if their premature infant will survive but the chances that he or she will be normal. Community services and specialist education provision could be delivered much more efficiently if planning was informed by accurate and ongoing data about the health status of the target population. Despite this clear need for information, apart from isolated cohort studies funded as research, it is simply not available in the United Kingdom or indeed in much of the developed world.
Past problems
The failure to make progress in this area seems to result from several factors. The initial focus of neonatal intensive care was on improving survival. Therefore, although outcome was seen as important, procedures to gain these data were not established in a systematic fashion. Where cohorts of infants have been reviewed in later childhood the approach has generally been to use a detailed series of tests to look for minor as well as major variations from normal. These types of assessment require the use of trained professionals and hence are costly.15 The community services in the United Kingdom perform health surveillance as part of their core activity and collect information about the health status of all children.16 Currently there is no routine system for identifying subgroups, such as infants who have required intensive care, to allow their data to be abstracted. More importantly there is no national protocol with regard to how and when children are reviewed by community services. Similarly the approach to documentation and recording varies widely.8
New opportunities
We wanted to identify a simple method of obtaining data about the later health status of this group of children, who are at high risk of long term neurodevelopmental problems (although the methods should apply to any other group—for instance, those below a certain birth weight). We chose a corrected age of 2 years as we thought this would provide feedback to clinicians in a time frame that was still relevant to their practice. The level of detail was selected to be informative for all interested groups while being cheap to collect.
Although we aimed to compare the two methods, we were aware that even if modest numbers of children were lost to follow up serious under-reporting of impairments could occur.9 Because of this, our target for either method to succeed was to achieve outcome data for 95% of eligible children. This did not occur, and hence overall outcomes (rates of normality and serious impairment) must be interpreted with caution, particularly in the community arm where ascertainment was based on only 67% of the infants recruited. While this is disappointing, there are positive elements to emerge from both methods.
Most parents clearly found the questionnaire easy to use and were able to provide accurate information, although we thought some misunderstood the concept of corrected age. Further refinement, particularly for when English is not the family's first language, may improve the response rate. While we suggested to families at recruitment that they could get advice from a health visitor or general practitioner when they completed the form, we did not try actively to involve these professional groups. If the 35 families who could not complete all questions or did not return the form at all had asked for help, the level of ascertainment could have risen to above our threshold for success of 95% but would have added considerably to the costs.
In relation to the community review some information was available for 94.5% of children. This high return was achieved because existing systems allowed the clerks to track children even when they moved districts. However, the systems for collecting and recording information were variable, and hence there were areas of development for which either there was no information or the information recorded could not be interpreted. Electronic patient records will provide an opportunity to deal with the issues of definition and recording of outcome. The timing of reviews and decisions about who should be reviewed need to be considered by the relevant professional groups as a matter of urgency. It is hard to identify value for money within the current system. Similarly it is hard to justify the extent of local variation given previous recommendations. 17 18
Financial context
Given the financial constraints of the NHS it is often difficult to convince commissioners of health care that expenditure on obtaining follow up data is justified. The cost of neonatal intensive care is around £1000 a day and about 1.5% of births in the United Kingdom are at ≤32 weeks' gestation. The additional costs associated with the methods used in this study (highest estimate £67 per case) are trivial by comparison and might lead to information on the success or otherwise of neonatal care, which at present can be measured only by early mortality.
With further refinement, both of the approaches tested in this study seem to have the potential to provide this information, but at present neither is adequate.
Acknowledgments
We thank the perinatal teams in the participating hospitals and, in particular, Hazel Lacy and David Curnock, Neonatal Unit, Nottingham City Hospital NHS Trust; Val Walster and Judith Grant, Neonatal Unit, Queen's Medical Centre, University Hospital NHS Trust; Christine Clark, Neonatal Unit, Leicester Royal Infirmary NHS Trust; Joy Tibbles and Wren Hoskyns, Neonatal Intensive Care Unit, Leicester General Hospital NHS Trust; Helen Holden and Rashmin Tamhne, Directorate of Children's Services, Leicestershire and Rutland Healthcare NHS Trust; Sue Curtis, Nottingham Community Health NHS Trust; R O Walters and Lynne Field, Neonatal Unit, North Hampshire Hospital; P Rowlandson, C Burtwell, and J Turner, Neonatal Unit, St Mary's Hospital NHS Trust, Isle of Wight; C Holme, Royal Devon and Exeter Healthcare NHS Trust; J Barnes and I Webb, Salisbury District Hospital; S Reveley, R Coppen, and S French, Poole Hospital NHS Trust; M A Hall and A Holloway-Moger, Princess Anne Hospital, Southampton; M Sutton, Winchester; and Helen Moody, Southampton Community Health. Alistair Shiell provided helped with the random allocation of units.
Contributors: DF, ED, CL, MB, DS, and AJ were responsible for the initial study design. CG developed the economic assessment. CRL and MG provided day to day oversight and supervised data collection. ED and BM carried out the analysis. All of the authors contributed to the final report, which was edited by DF. DF and CL are guarantors for the paper.
Footnotes
-
Funding NHS Research and Development Programme (Maternal and Child Health). ESD is funded by Leicestershire Health.
-
Competing interests None declared.