Improving quality in resource poor settings: observational study from rural RwandaBMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3488 (Published 30 October 2009) Cite this as: BMJ 2009;339:b3488
- Meera Kotagal, volunteer, student12,
- Patrick Lee, clinical mentor, hospitalist physician, clinical instructor in medicine123,
- Caste Habiyakare, nurse4,
- Raymond Dusabe, doctor4,
- Philibert Kanama, nursing director4,
- Henry M Epino, medical director, attending physician, instructor in surgery125,
- Michael L Rich, director6,
- Paul E Farmer, co-founder, Presley professor, chief178
- 1Partners In Health, 888 Commonwealth Avenue, Third Floor, Boston, MA 02215, USA
- 2Harvard Medical School, 260 Longwood Avenue, Boston, MA 02115
- 3Newton-Welleseley Hospital, 2014 Washington Street, Newtown, MA 02462
- 4Kirehe District Hospital, Rwanda Ministry of Health/Partners In Health, Kigali, Rwanda
- 5Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114
- 6Partners In Health, PO Box 3432, Kigali, Rwanda
- 7Department of Global Health and Social Medicine, Harvard Medical School, 641 Huntington Avenue, Boston MA 02115
- 8Division of Global Health Equity, Brigham and Women’s Hospital 651 Huntington Avenue 7th Floor, Boston, MA 02115
- Correspondence to: M Kotagal, Department of Surgery, University of Washington, Box 356410, Seattle, WA 98195, USA
- Accepted 23 June 2009
Problem Hospitals in rural Africa, such as in Rwanda, often lack electricity, supplies, and staff. In our setting, basic care processes, such monitoring vital signs, giving drugs, and laboratory testing, were performed unreliably, resulting in delays in treatment owing to lack of information needed for clinical decision making.
Design Simple quality improvement tools, including plan-do-study-act cycles and process maps, were used to improve system level processes in a stepwise fashion; resources were augmented where necessary.
Setting 50 bed district hospital in rural Rwanda.
Measurement of improvement Three key indicators (percentage of vital signs taken by 9 am, drugs given as prescribed, and laboratory tests performed and documented) were tracked daily. Data were collected from a random sample of 25 charts from six inpatient wards.
Strategy for change Our intervention had two components: staff education on quality improvement and routine care processes, and stepwise implementation of system level interventions. Real time performance data were reported to staff daily, with a goal of 95% performance for each indicator within two weeks. A Rwandan quality improvement team was trained to run the hospital’s quality improvement initiatives.
Effects of changes Within two weeks, all indicators achieved the 95% goal. The data for the three objectives were analysed by using time series analysis. Progress was compared against time by using run chart rules for statistical significance of improvement, showing significant improvement for all indicators. Doctors and nurses subjectively reported improved patient care and higher staff morale.
Lessons learnt Four lessons are highlighted: making data visible and using them to inform subsequent interventions can promote change in resource poor settings; improvements can be made in advance of resource inputs, but sustained change in resource poor settings requires additional resources; local leadership is essential for success; and early successes were crucial for encouraging staff and motivating buy-in.
Routine care processes critical to hospital care are difficult to execute in overworked, understaffed, and under-resourced settings. The financial and material needs of developing nations have been well documented.1 2More recently, shortages of human resources have been widely reported.3 4These material and human resource gaps have a substantial impact on routine care.
Limited data have been published on quality improvement efforts in resource poor settings. Projects in Russia, Colombia, South Africa, Rwanda, and the Democratic Republic of Congo have achieved moderate successes with simple quality improvement tools.5 6 7 8 9 10 11 12Additionally, the University Research Consortium has extensively used quality improvement methods in more than 20 developing nations through its Quality Improvement Project. However, scepticism remains about quality improvement approaches in settings of severe resource constraints.13 Such scepticism may be warranted.
We determined whether using quality improvement methods could improve the quality of care provided at one district hospital. We hypothesised that these methods, with the simultaneous addition of resources, would improve overall quality of care.
Kirehe District Hospital is one site in a collaboration among the non-profit Partners In Health, the Clinton Foundation, and the Rwandan Ministry of Health. The 50 bed hospital in eastern Rwanda has six wards (one each for women, men, paediatrics, malnutrition, and maternity, and an isolation ward for tuberculosis patients) and serves a catchment area of 292 000 people, predominantly subsistence farmers. At the beginning of this project, the hospital had 37 nurses and four doctors, with a patient:nurse ratio of about 10:1. The first of two previous quality initiatives in the hospital involved top-down instructions from the hospital leadership about how to improve. The second focused on personal report cards linked to a pay-for-performance system; it was applied inconsistently, and short term gains in quality were not sustained. Both initiatives were eventually discontinued.
Outline of problem
At Kirehe Hospital we were unable to monitor vital signs consistently, give drugs in a timely fashion, and carry out requested laboratory tests promptly. These gaps in basic care led to delays in diagnosis and treatment. Without vital signs and laboratory results, doctors reported difficulty assessing patients; incomplete treatment plans often resulted. Patients did not consistently receive drugs as prescribed, resulting in delayed improvement and prolonged hospital stays and occasionally in worse clinical outcomes.
Key improvement measures
The three processes targeted in this initiative—monitoring vital signs, giving drugs, and executing and documenting laboratory tests—were selected through an informal discussion with the doctors, based on the ease with which nurses could understand these processes and their critical role in patient care.
For each objective, one indicator was tracked daily and reported during the morning staff meeting. Our aim was to reach 95% (in keeping with level II reliability14) for each goal—vital signs taken by 9 am, drugs given as prescribed, and laboratory tests completed and recorded in the chart by 9 am the morning after they were ordered—for five continuous days within two weeks of the start of the project. Performance in keeping with level II reliability on process measures corresponds to a defect level less than 1 in 100 and “indicates processes intentionally designed with tools and concepts based on the principles of human factor engineering.”14 We chose this performance level to focus on system changes, not just on increased individual effort, in order to reach our goals. For this study, we concentrated on process measures alone because they had an important clinical rationale and were fundamental providing care. Outcome measures were not selected for evaluation in this study because of the additional cost of measuring them in a resource poor setting.
Baseline data were collected on each of the three selected indicators for two weeks before the start of the initiative. Every weekday morning at 9 am, the nurse in charge of quality improvement reviewed a sample of 25 charts equally distributed across all six wards. This method was continued throughout the initiative and data were collected between October 2007 and March 2008. The results were validated by a second team member who accompanied the quality improvement nurse during chart rounds on average once a month. Additionally, a convenience sample of patients was briefly interviewed at least weekly to confirm that vital signs had been performed properly and drugs given properly, and not simply recorded in the chart.
Analysis and interpretation
The baseline data highlighted important gaps in performance for these three care processes. The daily reporting of data throughout the initiative prompted routine discussions of failures with the staff, allowing for identification of obstacles.
Strategy for change
The intervention was based on Nolan et al’s Model for Improvement.15 This model emphasises planning the intervention in precise detail, implementing it on a small scale, studying the results, and then expanding, revising, or abandoning the intervention accordingly through plan-do-study-act cycles. This initiative aimed to evaluate a series of interventions and resource inputs collectively, rather than a single intervention, in an attempt to assess their value as a comprehensive set.
The hospital leadership designated one Rwandan nurse as the quality improvement nurse; as the project progressed, a doctor was added to the leadership team. Both were trained by using a quality improvement protocol developed by foreign staff with input from quality improvement experts (fig 1)⇓. The quality improvement team consisted of this nurse and doctor, the hospital nursing director, the hospital medical director, and two foreign staff members.
The initiative had two primary components. The first focused on educating hospital staff about quality improvement and the importance of the three selected care processes. This training used the quality improvement protocol in figure 1⇑ to explain plan-do-study-act cycles, alongside a presentation of baseline data highlighting gaps in current performance. The second component was a series of plan-do-study-act cycles, implementing interventions in a stepwise fashion while tracking selected indicators. Data were made visible to the staff on large run charts posted in the staff meeting room. The previous day’s data were reviewed in the morning staff meeting, followed by a discussion of failures led by the quality improvement nurse. One key element of the interventions was adding both extra staff and equipment as necessary.
Staff discussions of performance data identified two resource gaps that limited our ability to maintain rate of monitoring of vital signs above 90%. The first was a lack of equipment necessary for monitoring vital signs—the hospital had only one functioning blood pressure cuff and three thermometers. One month into the project, new equipment (three blood pressure cuffs, six thermometers, and an oxygen saturation monitor with adult and paediatric probes) was purchased.
The staff discussions also alerted the hospital management to the high patient:nurse ratios, and that performance gains depended on considerable, probably unsustainable, staff effort. At the start of the project, there were 37 nurses on staff—one nurse per ward per shift. During the second month of this project, eight additional nurses were hired, allowing for two nurses per ward on the day shift and an additional nurse in the hospital at night.
Daily publication of data to the staff and one education session on the importance of timely administration of drugs were the primary interventions. After the eight nurses were hired (about 20 days after the start of the project) improved staffing ratios made it possible for the nurses to give drugs at scheduled times.
The third objective—laboratory tests performed and recorded by 9 am the morning after they were ordered—started about a month after the initial two objectives. After initially making the data visible to all staff, we introduced a new improvement tool: process mapping. We created a simple process map, and a facilitated discussion determined the reasons for failure at each step in the process. For each failure, the team identified an idea for improvement.
The process map for laboratory testing identified two major obstacles. Firstly, results were not being retrieved from the laboratory in a timely way, largely because it was difficult to locate the results. This problem was addressed by placing a box in the laboratory for results sheets for each ward. Nurses depositing samples at the lab noted the ward name on each sample.
Secondly, process mapping identified the human resource gap as a major obstacle to improvement. With only one nurse per ward, there was a substantial delay in executing routine processes as drugs were given and laboratory tests were drawn only after morning rounds had finished. This delay made it difficult for the laboratory team to process and return the results before 9 am the following morning. Having more nurses allowed for one nurse to go on ward rounds with the doctor while the other nurse gave drugs and collected samples for laboratory testing.
Effects of change
All three improvement projects resulted in rapid, substantial improvements, reaching the goal of 95% for five continuous days within two weeks. At baseline the median performance for vital signs was 57%, giving drugs 63%, and laboratory testing and documentation 46%. Making the data visible resulted in early improvement in each project, but the results were not consistently at 100% until shortfalls of equipment and staff were remedied. Shortly after these additions, data were routinely at 100% for each of the three improvement indicators. These improvements, and corresponding interventions, are further described on annotated run charts (figs 2-4)⇓ ⇓ ⇓.
The data for each of the three interventions were analysed by using time series analysis. Progress was compared against time by using run chart rules for statistical significance of improvement.16 Medians were calculated for baseline data. A new median was calculated when the run chart rule reached a significant change. At this point we considered that a new system was now in place.
We have shown that the use of simple quality improvement tools, in the hands of frontline caregivers in a resource poor setting, as an adjunct to the addition of resources, can improve care processes and build capacity for improvement. This initiative evaluated the impact of a series of stepwise interventions implemented through consecutive plan-do-study-act cycles, rather than focusing on a single individual intervention. Four important lessons emerge from this initiative that may be relevant to improvement work in resource poor settings.
Shed light on performance
Making data visible, and using it as a tool to make decisions about subsequent interventions and resource inputs, contributed greatly to the initiative’s success. This simple intervention helped build team spirit by eliminating the blaming and competitive aspect of previous quality initiatives and encouraging staff to work together towards a collective goal.
Increase available resources
We hypothesised that utilisation of quality improvement methods, with the simultaneous addition of resources, would improve the overall quality of care. Although substantial improvements occurred before material and human resources were increased, we believe that the sustainability of improvements is linked to these additional resources, including blood pressure cuffs and additional nursing staff. Quality improvement in resource rich settings has largely focused on using abundant resources efficiently. In resource poor settings, though, quality is often limited by gaps in resources that cannot be overcome by hard work and system improvements alone. Resource inputs must also be augmented. In this instance, a quality improvement project could not have been executed in 2005 at the start of the Partners In Health/Ministry of Health collaboration: decrepit physical infrastructure, few healthcare professionals, and profound shortages of essential medicines meant that there was little care to improve. Before improving the quality of care could be a primary objective, substantial resources had to be added. These resources included human resources (nurses and doctors, in addition to auxiliary staff), infrastructure (inpatient wards, laboratory facilities, electricity), supplies (drugs and equipment), and knowledge (training on disease management, basic patient care, and how to improve the system.)
However, additional resources cannot be added haphazardly. Basic quality improvement tools allow us to identify existing gaps so that additional resources can be most efficiently incorporated into the system. In this project, human resources were widely identified as the greatest need. The consistent use of data and the efforts made to improve quality of care encouraged the hospital leadership to support for resource investments.
The real costs of this work are considerable. Eight new nurses were hired, and new materials were purchased, costing about $41 000 (£25 200; €28 700) per year. However, the improvement effort itself did not require additional resources as the Rwandan nurse and doctor directing the improvement effort added their quality improvement responsibilities to their daily work at no additional cost. The additional resources outlined here were important for initiating improvement in these routine care processes and the continued availability of material resources and attention to basic resource needs will be critical for sustaining the improvement.
Train local leaders
Many quality improvement projects have identified the need for local leadership; this is even more essential when improvement work is undertaken in a crosscultural setting.17 18The leadership of a Rwandan nurse and doctor was essential for this project’s success. They were able to encourage and motivate their colleagues more effectively, and their experience was invaluable as the team planned the interventions. As this project was started under their leadership, they are better positioned to guide subsequent quality initiatives.
Show early success
Early successes in quality improvement are crucial for encouraging staff and motivating buy-in.19 This principle is no less true in resource poor settings. Two efforts initiated by outsiders had preceded our work, resulting in initial scepticism. But early successes in our project, showing what was possible, encouraged staff and empowered local ownership of the project. Staff became enthusiastic early on in the process when the daily focus on data and a discussion of failures provided an avenue for the nursing staff to voice to the medical director and nurse manager what was making it difficult for them to do their jobs, and open conversations ensued about obstacles to providing care. The management’s response helped the staff realise that attention was being paid to what had previously been thought of as unnoted concerns. Before this initiative, staff had attended staff meetings daily, but rarely discussed issues or concerns.
One limitation of this study is the verifiability of data on drugs given to patients. Chart review did not allow for easy verification of whether drugs had been given on time or had simply been documented. Every effort was made to validate data by asking patients if and when they had received their drugs, but this limited verifiability was a limitation for this objective.
The second limitation is the limited timeframe of the study. As in all improvement projects, holding the gains is a major challenge. Since the intervention included the addition of staff and equipment, one important focus to ensure sustainability has been the process changes instituted to ensure that the necessary resources have continued to be available. For example, the unit nurses now do a weekly ward round with the nurse in charge and take an inventory of the equipment and discuss any system problems that may have arisen in the past week. This huddle, and the resulting improvement in dialogue between staff nurses and the charge nurse, may have a beneficial effect on sustainability by allowing early identification of limited resources. One important lesson from this study is that in resource poor countries, quality improvement methods can be used to ensure that the initial improvement is sustained by focusing attention on the need for resources as well as on data collection and reporting.
Since the completion of this study, performance has been monitored while real time data reporting to staff has been scaled back in an effort to evaluate whether resource inputs and system changes have resulted in sustainable improvements. Review of these subsequent performance data will be essential for determining whether the stepwise interventions used here can promote sustainable change.
Use of process measures
The third major limitation is the emphasis on process measures instead of outcome measures. The three process measures selected as the focus of this study were viewed as fundamental processes critical to the provision of care. Outcome measures were not evaluated, in part because of the additional cost of measurement required in a resource constrained setting. These projects have subjectively improved care throughout the hospital, but data on patient outcomes are lacking. Since the completion of this project, doctors have uniformly commented that earlier perceived difficulties with giving drugs and obtaining laboratory results and vital signs have vastly improved. Further research is needed to quantify the impact on outcome measures, such as length of stay, morbidity, and mortality.
Quality improvement methodology has been widely used in resource rich settings to improve the quality of health care at a system level by improving efficiency of existing systems and maximising current resource inputs. The improvement projects described here used the same methodology in a rural district hospital in a resource poor setting and showed similar improvements.
A key difference in this initiative was the systematic addition of resources to fill existing gaps. Using quality improvement methods in resource poor settings not only leads to improved efficiency but also enables identification of resource gaps that must be ameliorated to improve quality. When plan-do-study-act cycles and process maps are used to identify gaps, new resources can be added most efficiently.
The quality improvement literature has noted that three components are necessary for success in improvement—will, ideas, and execution.20 Though these are necessary for quality improvement in any setting, they are not sufficient in many resource poor settings. These settings have a fourth requirement—resources. To improve the quality of care in resource poor settings, system development and increased resources must go hand in hand; both are necessary for continued success.
Cite this as: BMJ 2009;339:b3488
We thank Harry Atherton, Terri Byczkowski, Uma Kotagal, Jane Roessner, and Pamela Schoettker for help in preparing this article.
Contributors: MK, PEF, PL, and HME conceived and designed the project; MK analysed data and drafted and finalised the manuscript. PL, CH, RD, PK, HME, MLR, and PEF revised the manuscript. MK is guarantor.
Funding: Harvard University Committee on General Scholarships; MK received a Sinclair Kennedy travelling fellowship.
Competing interests: None declared.
Ethical approval: The study was reviewed and deemed exempt by the Institutional Review Board of Harvard Medical School and The Partners Human Research Committee (institutional review board for Brigham and Women’s Hospital and Massachusetts General Hospital), as well as by the Rwandan National Ethics Committee, all of which waived informed consent for patients.
Provenance and peer review: Not commissioned; externally peer reviewed.