Pragmatic randomised controlled trial to evaluate guidelines for the management of infertility across the primary care-secondary care interfaceBMJ 2001; 322 doi: https://doi.org/10.1136/bmj.322.7297.1282 (Published 26 May 2001) Cite this as: BMJ 2001;322:1282
- Jillian Morrison, senior lecturer ()a,
- Louise Carroll, research assistanta,
- Sara Twaddle, headb,
- Iain Cameron, professorc,
- Jeremy Grimshaw, professord,
- Alastair Leyland, research scientiste,
- Heather Baillie, statisticianf,
- Graham Watt, professora
- a Department of General Practice, University of Glasgow, Glasgow G12 0RR
- b Department of Research and Development, Stobhill NHS Trust, Glasgow G21 3UW
- c Department of Obstetrics and Gynaecology, University of Glasgow, Queen Mother's Hospital, Yorkhill, Glasgow G3 8SJ
- d Health Services Research Unit, University of Aberdeen, Aberdeen AB25 2ZD
- e Medical Research Council Social and Public Health Sciences Unit, University of Glasgow, Glasgow G12 8RZ
- f Department of Public Health, University of Glasgow
- Correspondence to: J Morrison
- Accepted 28 February 2001
Objective: To investigate the effect of clinical guidelines on the management of infertility across the primary care-secondary care interface.
Design: Cluster randomised controlled trial.
Setting: General practices and NHS hospitals accepting referrals for infertility in the Greater Glasgow Health Board area.
Participants: All 221 general practices in Glasgow; 214 completed the trial.
Intervention: General practices in the intervention arm received clinical guidelines developed locally. Control practices received them one year later. Dissemination of the guidelines included educational meetings.
Main outcome measures: The time from presentation to referral, investigations completed in general practice, the number and content of visits as a hospital outpatient, the time to reach a management plan, and costs for referrals from the two groups.
Results: Data on 689 referrals were collected. No significant difference was found in referral rates for infertility. Fewer than 1% of couples were referred inappropriately early. Referrals from intervention practices were significantly more likely to have all relevant investigations carried out (odds ratio 1.32, 95% confidence interval 1.00 to 1.75, P=0.025). 70% of measurements of serum progesterone concentrations during the midluteal phase and 34% of semen analyses were repeated at least once in hospital, despite having been recorded as normal when checked in general practice. No difference was found in the proportion of referrals in which a management plan was reached within one year or in the mean duration between first appointment and date of management plan. NHS costs were not significantly affected.
Conclusions: Dissemination of infertility guidelines by commonly used methods results in a modest increase in referrals having recommended investigations completed in general practice, but there are no detectable differences in outcome for patients or reduction in costs. Clinicians in secondary care tended to fail to respond to changes in referral practice by doctors. Guidelines that aim to improve the referral process need to be disseminated and implemented so as to lead to changes in both primary care and secondary care.
What is already known on this topic
What is already known on this topic Most previous research into clinical guidelines has focused on their development and implementation
Evidence is lacking about the outcomes and costs associated with the use of clinical guidelines
What this study adds
What this study adds Clinical guidelines that may alter the balance of care between general practice and hospital settings require more intensive implementation than guidelines aimed at either setting on its own
The cost effectiveness of clinical guidelines should not be assumed
Clinical guidelines aim to promote evidence based practice, improve patient outcome, and allow more efficient use of resources, but there is still uncertainty about whether they achieve these aims.1 About 15% of couples experience difficulty and delay in conceiving.2 The Effective Health Care Bulletin on subfertility suggested that the clinical management of these couples could be improved by guidelines covering aspects of investigation and referral by general practices.3 A previous study in Aberdeen showed that such guidelines increased the numbers of couples with appropriate investigations completed before referral but did not provide data on outcomes or costs.4
Our main hypotheses were that clinical guidelines for the management and referral of patients with infertility by general practices would reduce the proportion of patients referred too soon after attempting to conceive and increase the number of appropriate investigations carried out in general practice. We thought this would lead to a reduction in hospital based investigations and outpatient visits, resulting in a shorter time to reach a management plan and reduced overall costs to the health service. As Glasgow is a complex referral setting with five secondary and tertiary referral sites and noticeable contrasts in socioeconomic deprivation, we also examined the effects of the hospital site and deprivation score on outcomes.5
Participants and methods
A multidisciplinary group with professional or personal experience of subfertility developed the consensus guidelines. Research evidence was incorporated where it existed—that is, if there was an area of uncertainty about best practice, one or more members of the group searched the literature and reviewed the evidence. A management pack for infertility was produced containing the detailed guidelines, quick reference guidelines (in box), a structured form for use by doctors at referral, and two information leaflets for patients. The guidelines were introduced as part of a pragmatic, randomised controlled trial. Approval for our study was given by local ethics committees.
Key recommendations in the guidelines for when a couple presents
Ensure appropriate knowledge about how to conceive, then:
Confirm rubella immunity
Update cervical smear
Advise to take folic acid
Before referral, arrange:
One semen analysis (if first result is abnormal, repeat)
One measurement of serum progesterone concentration during the midluteal phase (if no evidence of ovulation from first result, repeat)
Only perform additional endocrine tests (follicle stimulating hormone, luteinising hormone, thyroid stimulating hormone, testosterone, prolactin, oestradiol) if cycle is less than 21 days or greater than 35 days
Refer without delay if:
The woman is aged 35 years or more
The woman's cycle length is less than 21 days or greater than 35 days
The woman's progesterone concentration is less than 20 nmol/l
The woman has a history of ectopic pregnancy, pelvic infection, endometriosis, or anatomical abnormalities
The man has two abnormal results for semen analyses
Otherwise delay referral until couple have been trying to conceive for 12 months
After referral continue to provide support and information as needed
The study population comprised patients resident and registered with doctors in the Greater Glasgow Health Board area. All 621 doctors and 47 registrars in 221 general practices in Glasgow were informed by letter about the study and invited to participate.
We stratified practices into small, medium, and large on the basis of number of partners (one or two, three or four, and five or more, respectively) and by the location of the catchment areas of the five large hospitals that manage infertility in the health board area. We randomised these into intervention and control practices. To reduce the risk of contamination, practices sharing the same premises were allocated to the same group.
Doctors in intervention practices were sent the management pack and invited to attend one of four meetings to discuss the implementation of the guidelines. Overall, 57 doctors (17%) attended a meeting. Individual visits to practices to discuss implementation were also offered, and two practices were visited. Doctors in the control practices were informed that they would be sent a copy of the guidelines after 12 months. Relevant professionals in the participating hospitals were informed about the project during individual meetings with members of the study team (JM and LC), and infertility specialists from three of the participating hospitals were members of the guideline development group.
In each of the five hospitals, referral letters for infertility were screened by the research assistant for one year from June 1996 to May 1997, using a system of key words. Information about the date of birth, postcode, and whether the referral was for a woman only, a man only, or a couple was recorded from the referral letter.
Management survey before referral
Shortly after we had identified referrals, the referring doctor was sent a questionnaire requesting information about the management of the couple before referral, including questions about the dates of presentation and referral and investigations carried out. A reminder was sent after three weeks if necessary.
Review of hospital management from case notes
Twelve months after we had identified referrals, we obtained data from hospital records about hospital care, including the investigations carried out and the time taken to reach a management plan. A management plan was defined as a definite decision about management and included: specific treatment such as ovulation induction with or without intrauterine insemination, tubal surgery, male surgery, or referral for assisted conception techniques; one partner only continuing with specific investigations; review appointment only; or discharged from further hospital management. To comply with the Human Fertilisation and Embryology Act, we collected no further data on couples referred for consideration of assisted conception techniques. This decision was recorded as “a management plan in place,” and these couples were included in the analysis.
We originally estimated that a sample size of 600 referrals would have 80% power to detect at the 5% significance level a 15% difference in the number of measurements of serum progesterone concentrations during the midluteal phase and a difference between groups greater than 0.275 of the standard deviation of continuous variables, such as time intervals.
For presentation in the tables, we aggregated data about referrals from each practice and proportions determined for items under investigation. We tested the effect of the intervention by comparing the mean proportions for items under investigation in the two groups of practices. We used multilevel modelling to account for the clustering effect of practices and to assess the effect of patient deprivation and site of hospital referral.6 All analyses were on an intention to treat basis.
For tests and investigations before referral we extracted information on the use of resources from the prereferral questionnaire. For use of hospital resources in the 12 months after referral we extracted information relating to tests, investigations, and clinic attendances from case notes.
To reduce bias we included in the analysis only cases where full information was available. We then applied unit costs to each unit of resource use. To reduce bias as a result of using different costs from each hospital, we used a single set of unit costs. All costs are in 1998 prices. We compared mean costs with t tests.7 The costs of developing and disseminating the guidelines are not presented as part of this cost analysis as we wanted to focus on the effect of the guidelines on the cost of referral.
Seven of the 221 practices withdrew from the study, leaving 214 (96%) in the trial (figure). We collected data on 689 referrals. Median referral rates were 3.25 per 1000 from 88 454 women aged 20–44 years in the intervention practices and 3.27 per 1000 from 95 141 women aged 20–44 in the control practices (P>0.05). The two groups of referred patients were similar for age of the women at the time of referral, the deprivation category of their postal address, and whether the referral was made for a woman only, a man only, or a couple (table 1).
Management by doctors before referral
The referring doctor could not be identified from 30 referral letters (17 intervention, 13 control). Overall, 553 of 659 (84%) prereferral questionnaires were returned: 249 of 305 from the 85 intervention practices (82%) and 304 of 354 from the 79 control practices (86%).
Overall, 71 referrals were made after less than 12 months' reported duration of infertility (30 intervention, 41 control). Reasons for early referral included the woman being aged over 35 years (n=12), a relevant medical history according to the guidelines (n=32), an abnormal result for serum progesterone concentration in the midluteal phase or semen analysis (n=15), a history of infertility (n=1), and a history of miscarriage or stillbirth (n=5). More than one reason was given in 11 referrals. On checking the referral letter for the 17 cases with no apparent reason for early referral, six had a positive history according to the guidelines, and five had a history of infertility. Only six referrals after less than 12 month's infertility did not have an appropriate reason for early referral given either in the questionnaire or in the referral letter (1 intervention, 5 control).
Each investigation was carried out more frequently in the intervention practices, but these differences were not significant (adjusted odds ratios in table 2). A difference was found, however, in the total number of investigations, with referrals from intervention practices having a significantly higher mean. Deprivation category and referral hospital did not influence these results.
Review of case notes
Case notes were available for inspection for 617 of 689 (90%) referrals (285 intervention, 332 control).
Outcome of referral
A management plan was in place within one year for 292 referrals (47%). No difference was found between intervention or control practices in the proportion of couples for whom a management plan was made (145 intervention practices, 147 control practices: odds ratio 1.239, 95% confidence interval 0.869 to 1.765: P=0.236).
The mean duration between first appointment and date of the management plan was 3.34 (SD 2.93) months for referrals from intervention practices and 2.98 (2.59) months for referrals from control practices (95% confidence interval for difference −6.9 to 47.9; P=0.236).
No difference was found in the mean number of outpatient visits before a management plan was reached (1.92 (0.87) intervention, 1.91 (0.90) control (95% confidence interval for difference −13.9 to 20.3; P=0.836)). The results relating to hospital management were not influenced by correcting for the patient's deprivation category or for hospital site.
Measurement of serum progesterone concentration during the midluteal phase was repeated in 159 of the 287 referrals (55%), including all 111 referrals with a normal test result in primary care before referral. Semen analysis was repeated in 95 of the 168 cases (57%), including 20 of 58 referrals with a normal test result in primary care before referral.
Full information on which costs could be calculated was available for 159 referrals from intervention practices and 183 referrals from control practices. No significant differences were found between the two groups in the total costs to the NHS, although the median cost was lower in referrals from control than intervention practices (table 3).
Following a comprehensive evaluation of the effect of clinical guidelines on the management of all cases of presumed infertility referred for hospital investigation in Glasgow over one year, few patients were referred inappropriately early for investigation of infertility. The guidelines resulted in a modest increase in the number of recommended investigations carried out before referral. There was no evidence that subsequent hospital management was influenced as a result of better investigation by doctors before referral. Many investigations were repeated, possibly representing an important opportunity cost to the NHS. There was no evidence that patient outcomes were improved or that the intervention led to cost savings for the NHS (in fact it may have led to increased costs owing to the unnecessary repetition of investigations).
Strengths and weaknesses
Our study is the largest one of referral guidelines to date. We involved 96% of general practices in the Greater Glasgow Health Board area. Response rates for surveys of care before and after referral were also high (84% general practices, 90% hospitals), and information on outcomes and costs to the NHS was obtained. A recent systematic review of interventions including guidelines to improve outpatient referrals from primary care to secondary care found only one other study in which most eligible providers participated, and a cost analysis was included (J Grimshaw et al, unpublished data 2000). The study was similar to the previous study in Grampian. This has enabled replication of an apparently successful intervention in a more complex referral setting, allowing the external validity of the intervention to be assessed.
Interpretation within context of setting and intervention
Our implementation strategy followed current practice at the time of the study, including simple distribution of the guidelines, educational meetings, and visits to practices, but uptake of educational meetings and visits to practices was low. Use of more intensive methods to encourage the guidelines to be used might have increased uptake but would have required additional resources.8
The increases in recommended investigations before referral are smaller than those observed in the evaluation of similar guidelines in Grampian, where the proportions of cases who had progesterone concentrations measured during the midluteal phase and semen analyses checked increased from 40% to 71% and from 41% to 51%, respectively.4 The reasons for the lower baseline levels and smaller guideline effects in Greater Glasgow are not known but could include the general effects of severe socioeconomic deprivation on general practice in Glasgow and specific problems in implementation of the guidelines.5 For example, some aspects of the guidelines, such as the recommendations for semen analysis, may have been more difficult to implement in an urban setting, where fewer couples are registered with the same doctor. In addition, Grampian is a geographically discrete referral setting where couples are referred to a single teaching hospital, and there is a strong history of research collaboration across the primary care-secondary care interface. Arguably, Glasgow provides a more typical setting for research of this nature.
Possible mechanisms and implications for clinicians or policymakers
Referral behaviour is potentially difficult to change. For most clinical problems the decision to refer is taken infrequently in general practice and is influenced by a range of complex factors, including psychosocial considerations, which are poorly recognised in referral guidelines. Guidelines continue to be produced, but the results of our study show that their simple dissemination is unlikely to be sufficient in complex referral settings. Furthermore, interventions will be undermined if management in secondary care is not coordinated with that of primary care. It is possible that the duplication of tests in hospital reflects an unnecessarily rigid application of local clinical protocols or a distrust of results obtained from unfamiliar laboratories, but clearly there is scope for better communication and coordination of care between hospitals and general practices. It is likely that referral guidelines such as those being prepared by the National Institute for Clinical Effectiveness will not succeed without important activities for local implementation across the interface. For example, in contrast to our findings Thomas et al found that patients referred for urological investigation by doctors who used guidelines were seen and had a management decision more rapidly than control patients.9 These guidelines, however, were part of a complex intervention that included reorganisation of the secondary care system to streamline the referral process.
Relatively few studies of interventions to improve referrals exist. Further studies are needed, especially to explore methods of engaging both primary care and secondary care. Our study shows the need to replicate implementation studies to test the generalisability of effects in other settings.
We thank Professor Allan Templeton and Dr Robin Yates for their help in designing this study; members of the guideline development group (H Barlow, R Brogan, I Cameron, E Forrester, J Grimshaw, C Jagger, J McAllister, D Mack, J Morrison, J Norman, M Reid, R Roberts, R Robertson, D Shapiro, Y Taylor, S Twaddle, G Watt, A Wilding, R Yates); and the doctors, clinical and clerical staff in Glasgow Royal Infirmary University NHS Trust, Southern General Hospital, Stobhill NHS Trust, Victoria Infirmary NHS trust, and the Western Infirmary.
Contributors: JM prepared the grant application, supervised the development of the guidelines, data collection and preliminary analysis, and wrote the paper. She will act as guarantor for the paper. LC organised data collection, collected data, and carried out preliminary analysis. IC, JG, and GW were involved in the conception, design, and ongoing management of the study and interpretation of data analysis and contributed to writing the paper. ST was involved in the conception, design, and ongoing management of the study and interpretation of data analysis and designed and organised data collection and analysis on costs, interpreted data analysis on costs, and contributed to writing the paper. AL and HB were responsible for statistical analysis and contributed to writing the paper.
Funding The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Executive Department of Health. However the views expressed are those of the authors. This work was funded by the NHS research and development programme on the primary-secondary care interface (grant No 2-09).
Competing interests None declared.