Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation
BMJ 2009; 338 doi: https://doi.org/10.1136/bmj.b1440 (Published 27 April 2009) Cite this as: BMJ 2009;338:b1440
This article has a correction. Please see:
- Rick Goud, research fellow1,
- Nicolette F de Keizer, assistant professor1,
- Gerben ter Riet, associate professor12,
- Jeremy C Wyatt, professor3,
- Arie Hasman, professor1,
- Irene M Hellemans, cardiologist1,
- Niels Peek, assistant professor1
- 1Department of Medical Informatics, Academic Medical Centre, University of Amsterdam, Meibergdreef 15, 1100 DD, Amsterdam, Netherlands
- 2Horten Centre, University of Zurich, Zurich, Switzerland
- 3Health Informatics Centre, University of Dundee, Dundee DD2 4BF
- Correspondence to: N Peek n.b.peek{at}amc.uva.nl
- Accepted 18 December 2008
Abstract
Objective To determine the extent to which computerised decision support can improve concordance of multidisciplinary teams with therapeutic decisions recommended by guidelines.
Design Multicentre cluster randomised trial.
Participants Multidisciplinary cardiac rehabilitation teams in Dutch centres and their cardiac rehabilitation patients.
Interventions Teams received an electronic patient record system with or without additional guideline based decision support.
Main outcome measures Concordance with guideline recommendations assessed for two standard rehabilitation treatments—exercise and education therapy—and for two new but evidence based rehabilitation treatments—relaxation and lifestyle change therapy; generalised estimating equations were used to account for intra-cluster correlation and were adjusted for patient’s age, sex, and indication for cardiac rehabilitation and for type and volume of centre.
Results Data from 21 centres, including 2787 patients, were analysed. Computerised decision support increased concordance with guideline recommended therapeutic decisions for exercise therapy by 7.9% (control 84.7%; adjusted difference 3.5%, 95% confidence 0.1% to 5.2%), for education therapy by 25.7% (control 63.9%; adjusted difference 23.7%, 15.5% to 29.4%), and for relaxation therapy by 25.5% (control 34.1%; adjusted difference 41.6%, 25.2% to 51.3%). The concordance for lifestyle change therapy increased by 3.2% (control 54.1%; adjusted difference 7.1%, −2.9% to 18.3%). Computerised decision support reduced cases of both overtreatment and undertreatment.
Conclusions In a multidisciplinary team motivated to adopt a computerised decision support aid that assists in formulating guideline based care plans, computerised decision support can be effective in improving the team’s concordance with guidelines. Therefore, computerised decision support may also be considered to improve implementation of guidelines in such settings.
Trial registration Current Controlled Trials ISRCTN36656997.
Introduction
One of the main challenges in contemporary health care is to increase the application of sound clinical evidence to routine care.1 Although clinical practice guidelines are designed to promote effectiveness and discourage the use of ineffective treatments, adherence to guidelines in practice is often poor.1 2 3 Dissemination of practice guidelines on paper alone has generally proved to be insufficient. Instead, a carefully designed strategy for change usually needs to be used for effective implementation of guidelines.2 4 5 6
Patient tailored computerised decision support to individual professionals at the point of care is one of the most effective methods of improving decision making.1 4 5 7 8 It has been shown to improve the decisions of individual professionals in screening for cancer,9 10 in vaccination,11 in management of diabetes,12 13 for ordering (laboratory) tests,14 15 for dosing and prescribing of drugs,16 17 and in other settings.7 However, computerised decision support has also failed to improve practitioners’ performance,7 8 18 19 and in which circumstances and settings it is optimally effective is unclear.7 20 21
Specialist medical care is nowadays often provided not by individual professionals but by multidisciplinary teams.22 23 24 25 26 Working in multidisciplinary teams integrates the professional knowledge and skills of different disciplines and is generally considered to improve the coordination, quality, and continuity of care for patients.22 24 25 Whereas individual decision making is mainly a cognitive process, decision making in teams is additionally influenced by the social context, such as the interpersonal relationships within the team.26 27 Whether computerised decision support can improve this process is unknown, as all previous trials have evaluated its effect on the decisions of individual practitioners.
This paper reports on a cluster randomised trial evaluating the effect of computerised decision support on concordance of multidisciplinary teams with guideline recommended therapeutic decisions in outpatient cardiac rehabilitation. Cardiac rehabilitation is a multidisciplinary secondary prevention strategy for patients who have had a cardiac incident (such as a myocardial infarction) or a cardiac intervention (such as heart surgery). Cardiac rehabilitation is critical to ensure that patients are in the best possible physical and psychosocial condition to return to and maintain their normal place in society and to reduce their future cardiovascular risk.28 29 30 31 32 Cardiac rehabilitation has been shown to be cost effective in economic evaluations in North America and Europe.30 However, in many Western countries cardiac rehabilitation practice is poorly standardised and does not follow the available scientific evidence.30 33 34
The objective of this study was to determine the extent to which computerised decision support can improve concordance of multidisciplinary teams with guideline recommended therapeutic decisions. To avoid contamination across patient groups resulting from teams learning from the system, we chose a cluster randomised design.35 As outpatient centres have only one multidisciplinary cardiac rehabilitation team, entire outpatient centres were the units of randomisation.
Methods
Guidelines for cardiac rehabilitation
To stimulate evidence based cardiac rehabilitation services, the Netherlands Heart Foundation (a patients’ interest organisation) and the Netherlands Society for Cardiology (a professional organisation) published national guidelines for cardiac rehabilitation in 2004.36 Consistent with international standards,29 30 37 the national guidelines state that patients should be offered an individualised rehabilitation programme during which each of four treatments should be provided: two “standard” treatments—exercise training and education therapy (education about the consequences of the patient’s disease)—and two “new” but evidence based treatments—lifestyle change therapy (risk related behavioural adjustment) and relaxation and stress management training. To develop an individualised rehabilitation programme, the guidelines describe a needs assessment procedure that requires 15 to 40 data items concerning the patient’s medical, physical, psychological, and social condition and lifestyle to be gathered. This procedure generally takes place two weeks after discharge from the hospital, after which, during weekly meetings, the multidisciplinary cardiac rehabilitation team decides on the content of the patient’s rehabilitation programme on the basis of the information collected during the needs assessment procedure. The team, which usually includes physical therapists, nurses, psychologists, dietitians, social workers, and rehabilitation specialists or cardiologists, is jointly responsible for the execution of this programme during the next six to 12 weeks. All outpatient cardiac rehabilitation services act under the responsibility of cardiologists.
CARDSS guideline implementation system
To stimulate the implementation of these guidelines, we developed an electronic patient record system with computerised decision support functionalities called CARDSS (cardiac rehabilitation decision support system).38 CARDSS actively guides users through the needs assessment procedure by way of a structured dialogue, prompting them to record the necessary information. In addition, CARDSS assists in formulating a patient specific rehabilitation programme by providing computerised decision support: it automatically shows whether each of the four treatments is recommended by the guidelines, on the basis of the patient’s needs assessment data. On request, CARDSS provides the rationale behind its recommendations and links to relevant research evidence.
During the trial, one or more members of the multidisciplinary cardiac rehabilitation team, usually a specialised nurse or therapist, collected needs assessment data during a 30-60 minute meeting with the patient. These team members recorded the data directly into CARDSS during the visit or entered them into CARDSS shortly afterwards. The data were subsequently used as input for the weekly multidisciplinary team meeting, where all decisions about the patient’s rehabilitation programme were made. The needs assessment data recorded in CARDSS, including guideline based therapeutic recommendations from the system, were always available during this meeting, either directly through CARDSS (for example, projected on a screen) or on a paper report printed from CARDSS. The team recorded the final therapeutic decisions in CARDSS at the end of the meeting.
To facilitate acceptance by users, CARDSS provided additional information management services, including registration, documentation, and calculation of summary statistics across patients.38 39 In a pilot study in four cardiac rehabilitation centres, CARDSS was readily accepted and integrated into existing working procedures.40
Participants
The national guidelines and CARDSS were jointly presented at a national conference on cardiac rehabilitation in January 2004. Six months later, the Netherlands Heart Foundation surveyed all 101 Dutch cardiac rehabilitation centres for their interest in purchasing the system for €100 (£89; $130). Each interested centre was eligible to participate in the trial, except for the four centres that participated in the pilot study.40 To participate, the centre had to agree to document in CARDSS the needs assessment of each cardiac rehabilitation patient seen during the trial. We offered participants several incentives, including reimbursement of the purchasing costs of CARDSS, free training, and free helpdesk services.
Intervention
Participating centres worked with either of two versions of CARDSS: an intervention version or a control version. The intervention version had full functionality, whereas the control version comprised all the information management services but did not provide therapeutic recommendations. We thereby controlled for the potential positive effect of the information management services and dialogue structure provided by CARDSS on the decision making of rehabilitation professionals, a phenomenon known as the “checklist effect.”41 42 In the control arm, multidisciplinary teams selected rehabilitation treatments by using their own judgment; the written guidelines could always be consulted on paper or electronically within CARDSS. Control arm teams could explain their decisions but were not obliged to do so. Intervention arm teams could base their decisions on the system’s therapeutic recommendations. Non-concordance with the guidelines’ recommendations required recording of the reason, as “patient refusal,” “lack of facilities,” “disagreement with guideline,” and “other.”
At the start of the centre’s inclusion in the study, all multidisciplinary cardiac rehabilitation teams received a standardised training course, designed by the investigators, during which both the control and intervention versions of CARDSS were demonstrated to all teams. Teams participated in the study for at least six months, after which they all received the full version of CARDSS.
Outcome measures
For all four cardiac rehabilitation treatments individually, we used concordance of the multidisciplinary teams’ therapeutic decisions with the guidelines as the outcome measure. We defined concordance at the level of the patient; it implied treating patients who should have been treated and not treating patients who should have been untreated, according to the guideline. To evaluate the effect of computerised decision support on multidisciplinary teams’ decision making, we compared the proportions of concordant cases between the intervention and control arms. In addition, we evaluated the effect of the computerised decision support on undertreatment (withholding treatment from patients who should receive it) and overtreatment (treatment of patients who should be left untreated) of patients.
Sample size
On the basis of data from the pilot study,40 we used a mean of 22 eligible patients per month per centre, an average intra-cluster correlation coefficient of 0.04, and an average baseline concordance rate of 60% for all four cardiac rehabilitation treatments as estimators in the sample size calculation. Calculations showed that with a six month follow-up we would need 36 participating centres to detect a 10% absolute difference in concordance rate with 80% power at a type I error risk (α) of 5%. We used the Stata statistical software package to calculate sample size.43
Randomisation and allocation
We used dedicated software to do concealed randomisation using variable block sizes, stratified by type of centre (university hospital, autonomous rehabilitation centre/non-university hospital). We also stratified non-university hospitals by the mean number of new patients seen each month in the year before randomisation (<20, 20-30, >30).
After the project team had given the standardised training course, the centre received an email message with a key code that activated CARDSS and determined allocation of the team to the control or intervention arm. Allocation could not be influenced by, and was unknown to, the investigator giving the course. Centres could not be blinded to allocation because of the character of the intervention.
Data validation
During the trial, we asked all centres to retain their original administration system (mostly the paper based patient’s record) to record information on patients’ rehabilitation programmes.
To assess the quality and completeness of record keeping in CARDSS, we did an audit of data in each participating centre during or at the end of the trial. During the data audit we randomly selected the records of 10 patients receiving cardiac rehabilitation that were created during the trial period from the centre’s original administration system to serve as a reference standard. Firstly, to verify that all cardiac rehabilitation patients seen at participating centres had been entered into CARDSS, we checked if each of the selected patients had a record in CARDSS. Secondly, we verified the quality of patients’ data stored in CARDSS by comparing each of the 10 patients’ demographic information and therapeutic decisions recorded in the original administration system with the data in CARDSS. If we did not find two or more selected patients in CARDSS or if discrepancies in demographic information or therapeutic decisions existed in more than two records, we considered all the data of the centre in question to be unreliable and excluded them from the analyses. If a centre passed the data audit but data analysis showed that 20% or more of a centre’s patients’ records missed any data necessary to determine concordance with the guideline, we excluded that centre from the analyses.
To reduce potential dilution of the contrast in treatment by centres’ potential suboptimal performance in the initial phase (learning curve before reaching a plateau), we excluded the data of patients enrolled in the first two weeks of CARDSS being used in each participating centre, with a minimum of 10 patients, from the analyses. We did the outcome assessment unblinded, but it could not be influenced by the assessors as concordance data were not subject to judgment.
Statistical analysis
We estimated the effects of computerised decision support on concordance with the guideline at the level of the patient by fitting logistic regression models to the concordance data for the four types of treatment. We used three patient level variables (age, sex, and indication for cardiac rehabilitation) and two centre level variables (weekly volume of new patients, and whether the centre is either a specialised rehabilitation centre or part of an academic hospital) as covariates to adjust for differences in case mix between the intervention and control groups. We used natural splines to model non-linear effects of continuous variables (age of patient and volume of centre). Furthermore, to account for potential correlation of outcomes within centres, we used generalised estimation equations with exchangeable correlation.44 45 Because of the small number of clusters, we repeated the analyses with jack-knife estimators of variance.46 47 In addition, we repeated the analysis described above to estimate the effect of computerised decision support on undertreatment and overtreatment of patients. We used SPLUS version 6.2 (Insightful Corp, Seattle, WA, USA) for statistical analyses.
Results
In October 2004 we invited 40 centres that were interested in purchasing CARDSS to participate in the trial. Thirty five (88%) centres accepted this invitation. Of these, four centres were unable to implement the system before the end of the recruitment period because of inadequate information technology infrastructure. Figure 1⇓ shows that the remaining 31 centres were assigned to the intervention arm (16 centres) or the control arm (15 centres). Centres were enrolled in the trial between January and December 2005. The last centre completed participation in the trial in July 2006.
During the trial, five control arm centres discontinued their participation. Three control centres were reluctant to continue participation as they believed that the benefits of CARDSS without computerised decision support did not compensate for the increased workload of learning to work with the system. One control centre had to stop participation because of a temporary lack of personnel, and another centre accidentally deleted its CARDSS database during an update of the server’s operating system in the last month of the trial.
The data audit revealed poor data quality in three intervention centres, which we therefore excluded from the analysis; in these centres, participants reported that they were unaware that they had to record their therapeutic decisions in CARDSS after consulting its recommendations. We excluded data from a further one intervention centre and one control centre, as in both centres more than 70% of patients’ records missed one or more data items as a result of incorrect use of the system. We analysed data from 21 centres including 2787 patients. Table 1⇓ shows the baseline characteristics of both trial arms at the level of centres and patients.
Baseline characteristics of clusters and patients. Values are numbers (percentages) unless stated otherwise)
Table 2⇓ shows the primary results of the trial. Computerised decision support increased concordance with the guideline’s recommendations for exercise, education, and relaxation therapy. The increased concordance with decisions for the lifestyle change therapy was not statistically significant. Table 2⇓ reports only the confidence intervals according to the standard generalised estimation equations, as we found similar results with the jack-knife estimator. The jack-knife estimator did result in borderline significance for the exercise therapy (95% confidence interval 0% to 5.4%).
Primary results of trial: concordance rates and difference between intervention and control group in concordance with guideline recommendations for four measured rehabilitation therapies. Values are percentages (numbers) unless stated otherwise
For all four treatments the actual intra-cluster correlation coefficient was higher than anticipated, especially for education and relaxation therapy. Concordance with the guideline for exercise therapy was higher in the control group than had been estimated in the sample size calculation, but it was much lower than estimated for the relaxation and lifestyle change therapy. Figure 2⇓ shows the variation in concordance with guideline recommendations for each treatment.
Fig 2 Concordance of control and intervention centres with guideline recommended treatment decisions. Each box shows median value (line inside box), quartiles (box edges), extreme values (whiskers), and outliers (horizontal lines) within category
Figure 3⇓ shows the crude data on guideline recommendations and multidisciplinary teams’ therapeutic decisions for both study arms and for each of the four treatments. In both the intervention and control arms undertreatment was more common than overtreatment, particularly for the relaxation, education, and lifestyle change therapies; differences between the two study arms were significant for the relaxation and education therapies. These treatments should have been given to 2310 (85.4% of all patients) and 2419 (88.9%) patients, of whom 1310 (56.7%) and 490 (20.3%) did not receive the recommended treatment. The adjusted difference between the control and intervention arm in undertreatment was 42.8% (95% confidence interval 1.1% to 68.0%) for relaxation therapy and 25.8% (14.9% to 33.6%) for education therapy, in favour of the intervention arm. We also found a significant difference for overtreatment with exercise therapy. Of the 343 patients (12.2% of all patients) who should not have been given exercise therapy, 111 (32.4%) patients incorrectly received this treatment. The adjusted difference in overtreatment with exercise therapy between the two study arms was 25.7% (4.3% to 54.1%), with more overtreatment in the control arm. Other differences between the study arms were either not statistically significant or very small.
Fig 3 Number and percentage of patients who were correctly treated (true positives), correctly untreated (true negatives), overtreated (false positives), and undertreated (false negatives) for each cardiac rehabilitation treatment, in intervention and control centres
In the intervention arm, patients’ refusal was reported as the main reason for non-concordance with recommendations for exercise (77/121), education (127/199), relaxation (407/651), and lifestyle change (381/686). Lack of sufficient facilities was another important reason for non-concordance with recommendations about lifestyle change (160/686) and relaxation therapy (68/651). Recording of non-concordance in the control arm was voluntary; reasons were recorded for only 152 (8%) of the 1821 non-concordant decisions. For those decisions, patients’ refusal was also the main reason for non-concordance with recommendations for all four treatments.
Discussion
We found that computerised decision support improved the concordance of multidisciplinary cardiac rehabilitation teams’ decisions for three out of four rehabilitation treatments for which we measured concordance with guideline recommendations. Computerised decision support reduced cases of both overtreatment and undertreatment.
Strengths and weaknesses of study
Although recruiting participants for trials is always difficult, the recruitment for a trial of computerised decision support faces some additional challenges. In this study, we had to recruit entire multidisciplinary teams instead of individual professionals or patients, which meant that not just one professional but the entire team had to be motivated to participate. In addition, motivated teams could participate only if their centre had an adequate information technology infrastructure and both the team and the centre’s information technology department were willing and able to allocate resources for implementation of CARDSS within a limited time frame. For this reason, we enrolled only 31 of the 101 eligible centres in the study, which is nevertheless among the largest numbers of participants to date in studies evaluating computerised decision support.7 It does restrict the generalisability of our results to settings where teams are motivated to work with a computerised decision support aid and where sufficient information technology support and facilities are available to implement the system. However, as information technology support and facilities are rapidly improving in most hospitals, we believe that this requirement will be of less concern for such trials in the future.
Another potential source of bias in our results is the attrition rate. In the control arm, three centres discontinued participation as they found that it was not worth the effort of implementing CARDSS in their daily practice without receiving decision support. In the intervention arm, we excluded three clinics from the analyses as they failed to properly record therapeutic decisions in CARDSS. Such dropouts make it impossible to do a genuine intention to treat analysis. However, attrition did not seem to be related to concordance with the guideline but rather to the fact that teams faced the additional barrier of implementing a new electronic patients’ record system, as well as learning to use the computerised decision support aid. Therefore, we believe that if computerised decision support can be provided through an electronic patients’ record system that is already used on a routine basis, its additional benefit will be more easily realised.
In the intervention group, multidisciplinary teams were prompted to record the reason for non-concordance when they did not follow a recommendation of the system. This necessity to record the reason for non-concordance may have pressured the teams to follow the recommendations, thus increasing concordance with the guideline. Although we believe it to be unlikely that entire teams let their decisions be affected by the necessity to record reasons for non-concordance, the effect size of this feature is unknown.
In our study, the research team also led the development of the computerised decision support system. Garg et al found that this can lead to a potential bias in outcome assessment.7 However, several factors should have reduced the chances of such biases, including blinding of the investigators during the allocation procedure, use of objective outcome measures, and the involvement of an external evaluator (JCW) and a statistician (GtR) from another department outside the project team.
In this study, we measured the impact of computerised decision support on concordance with guideline recommended therapeutic decisions by teams, which is a measure of quality of care processes. When evaluating the effect of quality improvement interventions, such process measures are commonly used instead of patient related outcomes.7 Process measures are even preferable over patient related outcomes if the process measures are based on evidence or on accepted standards of care,7 48 as is the case in cardiac rehabilitation.36
Strengths and weaknesses in relation to other studies
Many studies have previously studied the effect of computerised decision support on individual professionals’ decision making. Two systematic reviews found that active computerised decision support systems (systems that automatically provide individual professionals with advice) are more effective than passive systems, which require professionals to request advice.7 20 Therapeutic recommendations rather than diagnostic advice and computerised decision support at the time and location of decision making were also found to increase the chances of success. We took these known success factors for computerised decision support systems into account during the development of CARDSS,38 and its users judged them favourably in a usability study.49 The positive opinion of professionals towards CARDSS is also reflected by the fact that it is still used in at least 35 of the 101 outpatient centres in the Netherlands.
A cluster randomised trial is usually the most rigorous method of evaluating interventions intended to affect professionals’ behaviour.50 51 52 So far, however, only a few studies evaluating computerised decision support have applied the cluster randomised design, and even fewer studies accounted for the clustering of patients in the statistical analysis.53 Our study design ensured that the estimated effects of computerised decision support could not be biased by the “checklist,”41 42 “Hawthorne,” “feedback,” or “carryover” effects described by Friedman and Wyatt,35 and our statistical analysis accounted for correlation of measurements of concordance within centres.54
Meaning of study
Although multidisciplinary settings are common in contemporary health care, no studies have yet evaluated the effect of computerised decision support in such a setting.7 In contrast to decision making by individual professionals, multidisciplinary decision making depends on social factors, such as the experience, profession, interpersonal relationships, and characters of team members.24 25 26 27 Our results show that computerised decision support can also be an effective instrument in multidisciplinary settings, where such social factors play an important role in decision making.
This study shows that, in a multidisciplinary team motivated to adopt a computerised decision support aid that assists in formulating guideline based care plans, such support can be effective in improving multidisciplinary teams’ concordance with guidelines. On the basis of our findings, we encourage the use of computerised decision support aids in settings where multidisciplinary teams are motivated to use them.
Unanswered questions and future research
Computerised decision support did not improve concordance with guidelines’ recommendations for the lifestyle change therapy, and although concordance with recommendation for the relaxation therapy increased, considerable undertreatment still existed. Although centres started to participate in the trial between one and two years after the release of the guideline that officially introduced these treatments, a considerable number of outpatient clinics still had insufficient facilities to offer these “new” treatments to all eligible patients during the trial. The literature emphasises that many different types of barriers to implementation of guidelines exist and may require different strategies for change,2 4 5 but little is known about the types of barriers that different strategies, including computerised decision support, can tackle.4 5 Our results suggest that a computerised decision support system alone is insufficient to improve concordance with guidelines when this requires additional resources—for example, in increasing uptake of lifestyle change therapy. More research is therefore needed to understand how computerised decision support can improve concordance with guidelines and which additional strategies for change need to be considered to overcome the remaining barriers.
What is already known on this topic
Many studies have found that computerised decision support is an effective instrument to improve the quality of individual health professionals’ decisions
Although multidisciplinary settings are common in contemporary health care, no studies have evaluated the effect of computerised decision support on decision making in teams
What this study adds
Concordance with guidelines of multidisciplinary teams’ decisions can be improved by assisting these teams with electronic, guideline based therapeutic recommendations
Notes
Cite this as: BMJ 2009;338:b1440
Footnotes
We thank Niels Agtereek (formerly of the Department of Medical Informatics, Academic Medical Centre, University of Amsterdam; now at McKesson Nederland BV, Nieuwegein, the Netherlands) for his assistance with the enrolment and training of participants, the data collection, and the data audit.
Contributors: RG, NP, and JCW were involved in the study concept and design. RG was responsible for acquisition of data. RG, NP, GtR, NFdK, AH, and IMH were involved in analysis and interpretation of data. RG, NP, NFdK, and JCW drafted the article. All authors critically revised the manuscript for important intellectual content and approved the final version. RG is the guarantor.
Funding: This project was funded by ZonMW, the Netherlands Organisation for Health Research and Development, Health Care Efficiency Research Program 2004, subprogram Implementation, under project no. 945-14-205. The researchers were independent from the funder. ZonMW had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.
Competing interests: The authors are academic originators of the software but have no financial or personal relationships with external people or external organisations that could inappropriately influence this work. The technology transfer office of the Academic Medical Centre in Amsterdam is searching for a professional software company to maintain and update the CARDSS software. Financial compensations for working at the Academic Medical Centre are according to the Dutch law.
Ethics approval: Not needed (according to the medical ethics committee of the Academic Medical Centre in Amsterdam).
Study sites (study coordinators): Antonius Hospital, Sneek (M Stolp, S Sprietsema); Bethesda Hospital, Hoogeveen (J Hein); Canisius-Wilhelmina Hospital, Nijmegen (H Peters, M Renema, F Bayat); Deventer Hospital, Deventer (R Reinderink); Elkerliek Hospital, Helmond; Groene Hart Hospital, Gouda (H Koers); Stichting Zorgsaam Hospital, Terneuzen (L Van Haecke ); Hospital De Tjongerschans, Heerenveen (D Lubberts); Hospital Gelderse Vallei, Ede (M Steenbergen); Hospital Lievensberg, Bergen op Zoom (N de Wijs, G in ‘t Veld); Hospital Nij Smellinghe, Drachten (G Nijholt, M van Aalsum); Isala Klinieken, Zwolle (A B Nieuwveld, E C J Schuck, Y M Schulte Nordholt); Jeroen Bosch Hospital, ‘s Hertogenbosch (M Groenendaal); Laurentius Hospital, Roermond (A Beurskens); Maasland Hospital, Sittard (N E Janssen, P Reijnen); Mesos Medical Centre, Utrecht (F A Verhoeff); Regional Hospital Queen Beatrix, Winterswijk (L van den Nouweland); Rehabilitation Centre Breda, Breda (H van Boxel); Rijnland Hospital, Leiderdorp (C Fransz); Rijnlands Rehabilitation Centre, Leiden (H J van Exel); Rivierenland Hospital, Tiel (P van Arkelen, H Junte); Röpcke-Zweers Hospital, Hardenberg; Slotervaart Hospital, Amsterdam (L Rombout, M Zurburg); Sophia Rehabilitation, Delft (J Wiegel); St Franciscus Gasthuis, Rotterdam; St Lucas Andreas Hospital, Amsterdam (J van Dam); Sionsberg Hospital, Dokkum (S Boersma, M Smid); University Medical Centre Groningen, Groningen (L J A Redelaar); University Medical Centre Utrecht, Utrecht (I K Geut, J Zweers); Waterland Hospital, Purmerend (E van Koningsveld); Zaans Medical Centre, Zaandam (J Blonk).
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.