- Trisha Greenhalgh, director1,
- Katja Stramer, senior research fellow2,
- Tanja Bratan, research fellow2,
- Emma Byrne, research fellow3,
- Jill Russell, senior lecturer2,
- Henry W W Potts, lecturer3
- 1Healthcare Innovation and Policy Unit, Centre for Health Sciences, Barts and The London School of Medicine and Dentistry, London E1 2AD
- 2Division of Medical Education, University College London
- 3Centre for Health Informatics and Multiprofessional Education, University College London
- Correspondence to: T Greenhalgh
- Accepted 27 May 2010
Objective To evaluate a national programme to develop and implement centrally stored electronic summaries of patients’ medical records.
Design Mixed-method, multilevel case study.
Setting English National Health Service 2007-10. The summary care record (SCR) was introduced as part of the National Programme for Information Technology. This evaluation of the SCR considered it in the context of national policy and its frontline implementation and use in three districts.
Participants and methods Quantitative data (cumulative records created nationally plus a dataset of 416 325 encounters in participating primary care out-of-hours and walk-in centres) were analysed statistically. Qualitative data (140 interviews including policy makers, managers, clinicians, and software suppliers; 2000 pages of ethnographic field notes including observation of 214 clinical consultations; and 3000 pages of documents) were analysed thematically and interpretively.
Results Creating individual SCRs and supporting their adoption and use was a complex, technically challenging, and labour intensive process that occurred more slowly than planned. By early 2010, 1.5 million such records had been created. In participating primary care out-of-hours and walk-in centres, an SCR was accessed in 4% of all encounters and in 21% of encounters where one was available; these figures were rising in some but not all sites. The main determinant of SCR access was the identity of the clinician: individual clinicians accessed available SCRs between 0 and 84% of the time. When accessed, an SCR seemed to support better quality care and increase clinician confidence in some encounters. There was no direct evidence of improved safety, but findings were consistent with a rare but important positive impact on preventing medication errors. SCRs sometimes contained incomplete or inaccurate data, but clinicians drew judiciously on these data along with other sources. SCR use was not associated with shorter consultations or reduction in onward referral. Successful introduction of SCRs depended on interaction between multiple stakeholders from different worlds (clinical, political, technical, commercial) with different values, priorities, and ways of working. The programme’s fortunes seemed to turn on the ability of change agents to bridge these different institutional worlds, align their conflicting logics, and mobilise implementation effort.
Conclusions Benefits of centrally stored electronic summary records seem more subtle and contingent than many stakeholders anticipated, and clinicians may not access them. Complex interdependencies, inherent tensions, and high implementation workload should be expected when they are introduced on a national scale.
Shared electronic records are being introduced in some countries for exchange of various types of data including medication, allergies, medical history, laboratory reports, referral letters, and discharge summaries (see Discussion). This paper considers the summary care record (SCR), a structured summary held on a national database and accessible to authorised staff over a secure internet connection, in the English National Health Service.
The National Programme for Information Technology in England was established in 2005 and led by Connecting for Health, the informatics arm of the Department of Health.1 It aimed to provide secure, nationally integrated electronic records as part of a wider political vision. In its 13 year administration (1997-2010), the UK Labour government pursued policies aimed at modernising public services (increasing efficiency, accountability, transparency, and orientation to the needs of the service user).2 3 Large scale information systems, which would integrate services and support choice by empowered citizens, were viewed as a key vehicle for achieving this goal.4 5 6 A series of policy documents in 1998-2008 sought to centralise control over specification, procurement, resource management, and delivery of information systems.1 7 8 9 10 11 12 The original vision included a central technical infrastructure known as the “Spine” and a nationally shared electronic record7; in early strategy documents the latter became a nationally stored summary record.13
Implementation of the SCR began in 2007 in two early adopter sites; a national roll-out began in mid 2008. By May 2010, 113 of 152 primary care trusts in England had committed to participating in the programme; 16 had begun to create SCRs; 29.8 million people (73% of the population aged >16 ) had been sent a letter informing them that an SCR would be created for them if they did not opt out; and 1.5 million such records had been created.
An SCR is currently drawn from the electronic record held by a person’s general practitioner; it contains three data fields—medication, allergies, and adverse reactions.14 It is intended for use in emergency and unscheduled care.15 Individuals may register to view their record via an internet-accessible personal health organiser (HealthSpace). Some general practices have begun to “enrich” their patients’ records with additional data fields such as important diagnoses and end-of-life care preferences. A forthcoming upgrade will allow secondary and community care staff to add data.
We reported previously on initial efforts to introduce SCRs in early adopter sites16 and on patients’ attitudes to SCRs.17 This paper, based on a longer report,14 addresses efforts to extend the programme nationally from 2008 to 2010. Research questions were
What is the usability, use, functionality, and impact of the SCR, and what explains variation in its adoption and use?
How have the fortunes of the SCR programme been shaped and constrained by influences at the macro, meso, and micro level?
What are the transferable lessons for practice and policy?
Management and governance
The evaluation was commissioned by the Department of Health through a competitive tendering process run by the Connecting for Health Evaluation Programme at the University of Birmingham. It was overseen by an external advisory group chaired by a layperson with representation from key stakeholders including patients, professional bodies, academic institutions, and Connecting for Health. Ethical approval was obtained from two multicentre research ethics committees—Thames Valley in January 2007 (06/MRE12/81 and subsequent amendments) and North West 8 in September 2009 (09/H1013/36 and subsequent amendments)—although these bodies classified some aspects of the study as “audit” and hence outside their remit.
We drew on Patton’s “utilisation-focused evaluation,” which views complex programmes as having multiple stakeholders, each with different expectations of the programme and the evaluation.18 We followed Klein and Myers’ method for interpretive field studies in large scale information systems, which emphasises continuous, iterative comparison of findings in one part of the project with an emerging overarching story of the whole.19 We refined these generic approaches into a specific method described in detail elsewhere.20 In short, we made four key assumptions:
Electronic records are not passive containers for information but actively shape and constrain care
People and technologies are linked in complex, dynamic socio-technical networks which offer possibilities and limit what is possible (materially and socially) in particular situations and contexts. Hence it is more useful to study “the process of socio-technical change” than “deployment of technology X”
People and technologies in a network both “act” but not in the same way (for example, people have values and feelings whereas technologies do not)
A technology programme should be studied at macro level (such as national policy, wider social norms and expectations), meso level (such as organisational processes and routines), and micro level (such as particular experiences of patients and professionals) and both qualitatively and quantitatively to build a rich, contextualised picture of complex change (fig 1⇓).
Qualitative data were collected with a view to answering the following questions:
How was the SCR framed (that is, what did its development and introduction seem to mean) in policies, strategies, and business plans; by service users and health professionals; and by critical voices such as the press and civil liberties activists? What hopes and concerns did different stakeholders have about it?
What were the social and technical challenges associated with efforts to create SCRs? How and to what extent were these overcome?
Once SCRs had been created, how were they accessed and used at the clinical front line? If they were not accessed and used, why not?
Qualitative data sources and their contribution to different components of the case study are shown in web extra table A. As is recommended practice in large scale organisational case studies,21 22 we drew on multiple data sources of different types, including over 140 interviews; 2000 pages of field notes reflecting approximately 1500 hours of ethnographic observation (incorporating detailed field notes on 214 clinical encounters); 3000 pages of documents such as policies, business plans, minutes, internal reports, communications, and media coverage; and feedback from stakeholders to drafts of our findings. Details of how this large and complex dataset was analysed are given in our full report.14 In sum, to analyse interviews and field notes, we read and annotated texts, discussed among team members, developed provisional explanations, and iteratively refined these using the constant comparative method.23 Rigour was defined in terms of authenticity (immersion in the case through extended fieldwork), plausibility (developing explanations of local phenomena which made sense to participants and drawing these together into a coherent overall narrative), and criticality (systematically questioning assumptions that are taken for granted).24 25 To analyse documents, we drew on the principles of critical discourse analysis.26
Quantitative data were collected with a view to answering the following questions:
What proportion of patients seen in emergency and unscheduled care had an SCR; in what proportion of these was it accessed; and what was the trend over time?
How did SCR availability and access vary in different sites and settings?
How did SCR access vary with patient characteristics (such as age, sex, nature of complaint) and clinician characteristics (such as doctor or nurse, level of experience)?
Was the SCR associated with a change in consultation length (for example, was there any evidence that its use made the consultation shorter)?
Quantitative data were collected by stakeholders in the programme and passed to us in either summary or raw form. Connecting for Health gave us cumulative summary statistics, updated weekly, on numbers of SCRs created, which were collated from weekly reports sent to them by participating regional centres. These indicated the national rate of record creation but were not amenable to further analysis. Adastra (a software company which supplies medical record software to unscheduled care organisations) passed us Excel files containing raw data on 416 325 consultations (on 325 321 episodes) in primary care out-of-hours and walk-in centres in the three sites in our sample between August 2008 and January 2010. Patient identifiers had been removed and clinician identifiers pseudonymised. Sites A and B were early adopters and had begun to create SCRs in late 2007. Site C joined the programme as part of the national roll-out and began to create SCRs in August 2009. Each entry gave a clinician identifier, diagnostic code(s) entered by the clinician, length of consultation to the nearest minute, whether an SCR was available, and whether it was accessed. Around a quarter of patients had had consultations with more than one clinician for the same episode (such as telephone consultation with a nurse followed by on site consultation with a doctor).
Analysis of this dataset is described in detail in the web extra appendix. Most analyses were done on a subset of 223 029 episodes (120 659 in site A, 102 370 in site B, and 65 588 in site C), representing times when SCRs were available in the three sites. Briefly, we conducted descriptive statistical tests (such as percentages); χ2 test for categorical variables; Pearson’s correlation coefficients for relationships over time; Mann-Whitney tests for skewed continuous variables; and logistic regression to explore the contribution of different variables to an overall model of factors that influenced SCR access. Analyses were undertaken for each site separately and all sites combined, and also for doctors and nurses separately and all clinicians combined. We used narrative synthesis to integrate qualitative and quantitative data.
What stakeholders expected of the summary care record
The SCR programme had multiple stakeholders (government and civil service, healthcare organisations, information technology (IT) suppliers, professional bodies, frontline clinicians and managers, service users, civil liberties groups). Policy makers anticipated six main benefits—better quality (such as more informed) care, safer care (such as reduction in medication errors), more efficient care (such as shorter consultations), more equitable care (such as in low-literacy patients or limited English speakers), reduction in onward referral (such as fewer hospital admissions), and improved patient satisfaction (because more needs would be met).27 28 29 30 In contrast, some critics framed the programme as a monolithic, inefficient, and delayed government IT project, and civil liberties campaigners saw it as a manifestation of the “Database State.”31 Patient organisations generally viewed the SCR positively as offering better and safer care, though they recognised a trade-off against the risk to privacy.
Implementing the programme
Creating SCRs and supporting clinicians to access them in unscheduled care involved numerous interdependent tasks (box 1) and occurred more slowly than originally planned. Local healthcare organisations were encouraged to maximise creation of SCRs, promote their use in a range of settings, document locally relevant benefits, and report back regularly to Connecting for Health on structured templates. National implementation managers and clinical leads were appointed to support this process. But in the face of finite budgets, competing policy priorities, and ethical concerns being voiced by local and national professional bodies, one NHS organisation (such as a primary care trust) had limited power to control and monitor the performance of another (such as a general practice) in “deploying” the SCR.
Box 1: Tasks in implementing the summary care record programme
Strategy and business planning (needed at national, regional, and local level)
Clarifying vision and setting goals
Estimating costs and timescales
Negotiating and allocating a budget
Aligning with other strategic priorities
Formalising relationships (contracts, service level agreements, memoranda of understanding)
Project management (needed at national, regional, and local level)
Establishing workstreams with targets and milestones
Developing business processes
Selecting, coordinating, and supporting participant organisations
Human resource issues (recruiting, training, supervising, performance managing)
Monitoring and reporting (including risk management)
Technology development (IT suppliers plus technical departments in participant organisations)
Understanding use cases and specifying what is needed from the IT
Testing (including security)
Post-release support (such as fixing bugs)
Rhetorical and communicative tasks
Framing and making sense of the programme
Justifying the programme (ethically, financially, politically) to key interest groups
Informing the public and managing queries and opt outs
Dealing with ongoing tensions
Redefining scope and content to reflect ideas for new uses
Operationalising “informed consent” when staff were busy and patients disinterested or confused
Making security and information governance processes workable (technical and human elements)
Balancing the need for high quality data with the goal of maximising number of records available
Balancing children’s autonomy and protection with parents’ wish to opt out on a child’s behalf
The date on which a general practice first uploaded patient data to create SCRs was known as the “go-live.” The challenges of engaging clinicians, finding resources, aligning business processes, training staff, and informing patients, along with delays in provision of technical solutions, combined to produce slippage in go-live dates and a concomitant loss of local morale and motivation. “Bugs” sometimes emerged when go-lives were attempted, necessitating manual workarounds for what was intended to be an automated upload.
Suppliers of electronic patient record software to general practices worked to develop upgrades that were compliant with the SCR and fix bugs that emerged during uploads. However, most of their customers were not asking for functionality to create SCRs, so this work was seen by some as diverting resources from the company’s core business of improving “in hours” general practice record systems. In contrast, Adastra, the main provider of software to primary care out-of-hours organisations, perceived strong demand from its customers for viewing shared records, so a one-click route to the SCR was a high priority development. Small size, competing business priorities, changes in specification, and the inherently unpredictable nature of software development limited the ability of some suppliers to deliver SCR functionality within the tight timeframes set out in the business plans of Connecting for Health and NHS organisations.
SCRs were hosted on the NHS Spine, provided under contract by a large commercial supplier, British Telecommunications (BT). Some staff employed by BT felt that the original specification had been set in board-level meetings with insufficient attention paid to the detail of frontline clinical work and perceived the contract as inflexible, though this was not the view of top management. Amendments to the contract were needed to incorporate changes to the specification that had not been fully anticipated at the outset of the programme.
Before patients’ SCRs were created, they were sent a letter explaining the programme and telling them how to opt out if they wished. Fewer than 1% of people opted out, but for various reasons (alleged lack of balance in information provided, letters discarded unread or not understood, confusing opt-out process) the extent to which “informed consent to upload” had been obtained was contested.32 In late 2008, the consent model changed to require clinicians to seek consent at the point of care. Support from some professional bodies for the programme increased after this change. However, concerns were by no means fully allayed, and the British Medical Association wrote to the minister of health in March 2010 expressing concern that a mass public mailout before the May 2010 general election was giving patients insufficient time to consider their options.33
Accurate and reliable SCRs depended on data quality of the local records from which they were created. In 2007-9, a national incentive scheme funded work in general practices to achieve data quality targets (which were initially a requirement for joining the SCR programme), but funding for this was not renewed in 2009. Some but not all primary care trusts allocated a budget for a local data quality scheme.
Unauthorised access through technical (“hacking”) or human (malice or error) breaches is an inherent risk of any nationally accessible record. For the SCR, this risk came with political sensitivities.31 Connecting for Health required the highest technical security standards and lengthy penetration testing before approving a product. Staff wishing to access a patient’s SCR were required to confirm a “legitimate relationship.” Detailed information governance procedures were introduced to monitor performance and generate “alerts” on possible unauthorised accesses. While some staff sympathised with these measures, others found them bureaucratic, intrusive, and unworkable. Fear of surveillance stopped some staff attempting to access SCRs altogether.
Use and non-use of summary care records: quantitative findings
Key quantitative findings are listed below; these raise qualitative questions which are addressed in the next section. Unless stated otherwise, the figures given below are derived from the Adastra dataset, detailed statistical analysis of which is presented in the appendix (see bmj.com). Some analyses focused on site B, chosen as showing the longest consistent use of the SCR.
What proportion of patients had a summary care record?
The proportion of patients seen in unscheduled primary care settings (out of hours centres and walk-in centres) who had an SCR increased from 19% to 31% in site A, 18% to 35% in site B between August 2008 and January 2010, and from 0% to 24% in site C between August 2009 and January 2010. This figure was known as the “hit rate” (that is, the chance of a clinician finding an SCR if he or she chose to look for one). Given that fewer than 1% of people in each site had opted out, the main determinant of the hit rate was whether the patient’s general practitioner was participating in the programme. Despite primary care trusts in sites A and B signing up as “early adopters” of the programme in 2007, 56% of general practices in site A and 24% in site B had not gone live with SCRs by early 2010. Hit rate was reduced by patients attending unscheduled care “out of area.”
In what proportion of patients was the summary care record accessed?
SCR accesses in secondary care settings were very low (fewer than 30 per week across the three sites, Connecting for Health data). Overall, in participating primary care out-of-hours and walk-in centres, an SCR was accessed in 4% of all encounters and in 21% of encounters where one was available. Trends in SCR accesses in primary care settings over time, expressed as a proportion of all patients seen, are shown in fig 2⇓. The three sites show different patterns: early adoption followed by a period of partial abandonment (site A); slow incremental adoption (site B); and more rapid adoption after a late start (site C). Statistical analysis of these trends is presented in the web appendix.
How did summary care record access vary with clinician characteristics?
The most significant single factor predicting SCR access was the identity of the clinician seeing the patient, which accounted for approximately 15% of total variance (data from site B, logistic regression, χ2(103)=3591, P<0.0001). Individual clinicians accessed available SCRs between 0 and 84% of the time (fig 3⇓). Clinicians who saw more patients in unscheduled care accessed SCRs significantly more often than average (data from site B, χ2(1)=470.9, P<0.0001). Overall, doctors were no more or less likely than nurses to access SCRs (χ2(1)=0.2, P=0.7; χ2(1)=2.3, P=0.13; χ2(1)=1.8, P=0.19 respectively in sites A, B, and C).
How did summary care record access vary with patient characteristics?
We found small absolute differences in access rates by the sex and age of patients, which reached statistical significance because the dataset was large. A multiple regression model incorporating site (A, B, or C), age, sex, date, day of week, and time of day (see web appendix) was statistically highly significant but accounted for only a small fraction of the variance in SCR access (pseudo R2=3.6%).
Analysis by diagnostic code was limited by marked inconsistency in coding practices between sites and between clinicians within sites. With that caveat, we found some evidence that the SCR was more likely to be accessed in conditions such as urinary tract infection for which information on current or recent drugs and allergies was likely to be useful (logistic regression χ2(1)=45.1, P<0.0001) and when the patient had more than one coded diagnosis (χ2(1)=78.7, P<0.0001). However, variation by diagnostic code was much less marked than variation between clinicians. In short, as fig 3 shows, many clinicians did not access SCRs regardless of the presenting problem.
Was summary care record use associated with change in consultation length?
Overall, consultations in which the SCR was accessed were significantly longer than consultations in which it was not accessed (median 10 minutes v 9 minutes, Mann-Whitney test, z=−6.4, P<0.0001). This difference was more marked for nurse consultations (15 minutes v 12 minutes, z=−7.5, P<0.0001). However, subanalyses by site and type of clinician (see web appendix) showed that this relationship was not consistent either between sites or by clinician type. Differences may be attributable to confounding variables (see below).
Use and non-use of summary care records: qualitative findings
Explaining low hit rate
Some clinicians working in unscheduled care settings said they rarely looked for SCRs because there was only a small chance of finding one given that many local general practices had not yet gone live. Reasons why a practice had not gone live varied considerably between sites. A number of general practice record systems were not compliant with SCRs; practices using these systems (covering 20% of the population nationally at the time of the study) would be unable to participate in the programme without replacing in-house software. Other reasons included inadequate data quality, inadequate technical infrastructure (such as the need to install a more powerful server), opposition to the programme, and competing priorities.
Different potential barriers had a greater or lesser impact in different sites. In site A, for example, the local medical committee included opinion leaders who opposed the SCR programme and a number of practices had yet to meet locally set data quality standards. The rapid rate of SCR creation in site C appeared to be attributable to a combination of factors predicted previously16: national and local policy goals were aligned, improving out-of-hours services was an explicit strategic priority, top management strongly supported the programme, middle managers were enthusiastic, local clinical engagement was strong (including commitment to data quality work), opponents to the programme were few and did not occupy powerful positions, and participating organisations had favourable technical capacity (for example, almost all general practice record systems were SCR compliant).
Explaining non-access of summary care records
Analysis of ethnographic data on 214 clinical consultations showed that non-access of the SCR had many different reasons, including not being clinically indicated; not technically possible (for example, the organisation was temporarily unable to access the Spine); clinician not trained or motivated (including “forgotten” smart card or password); delays in supplying working smart cards (such as to short term staff); and the organisation “not pushing” SCR use, usually because of a perception, correct or otherwise, that the hit rate would be low (table 1⇓). In secondary care settings, non-adoption was further explained by lack of an integrated technical solution (that is, the clinician had to exit one piece of software and go into another to access an SCR) and by the temporal and spatial challenges of accessing computer terminals in real time (see examples in box 2).
Box 2: Examples of how small details made a big difference to actions and outcomes
Case FN22/#01—A hospital pharmacist is doing her rounds on an intermediate ward of newly admitted patients. This patient is a limited English speaker who seems confused. He has a plastic carrier bag by his bed containing some bottles of medication. The pharmacist asks if she may access his SCR, and the patient consents. However, the pharmacist decides not to do so, since the patient may not have understood the question and in this hospital an internal rule has been introduced that only doctors may use the emergency override option (“best interests of patient”).
Although the pharmacist undoubtedly had a legitimate relationship with this patient, a number of key nuances combined to produce “non-access” of the SCR: both the need for further information and the patient’s consent were equivocal, and the pharmacist was bound by local as well as national access constraints.
Case FN04/#42—A senior doctor in a busy accident and emergency department seeks to access the SCR of a patient who is bleeding. He finds one of three terminals (shared by about 20 clinicians) and asks the junior doctor who is using it to hurry up. Within seconds he has inserted his own smart card and is logging on. When he finds the patient’s record, he remembers he has not asked the patient’s consent but quickly selects “best interests of patient” on the grounds that the information he seeks (about anticoagulant medication) could be life saving.
Had this doctor not been confident to use the emergency override, he would have had to make a long journey back to the patient to ask consent (and lose his place in the queue for the terminal). The combination of seniority, confidence, technical skill, and a clear clinical need for the information combined to produce “access” rather than “non-access” in this case—although disappointingly the patient’s record did not contain the information he sought.
Case FN06/#68—A nurse in a walk-in centre sees a patient with a headache. The nurse has her own consulting room. She has already logged on to the computer terminal with her smart card at the start of the session. The patient is not sure of her medication, and the nurse asks, “Do you mind if I look up your GP record?” The patient replies, “Please yourself,” and the medication list is quickly found.
In this case, the physical and technical infrastructure of the walk-in centre (every clinician has a terminal) along with key characteristics of the clinician (experienced, trained, in possession of a working smart card) and the patient (fully conscious, not seriously unwell, confident in the clinician) combined to produce rapid, one-click access to the patient’s SCR.
The partial abandonment of SCRs in site A (see fig 2⇑) may have been partly due to a change of premises, during which some staff temporarily lost access privileges so that their smart cards ceased to work. Temporary reduction in access rate was also seen during periods when the proportion of locums and inexperienced staff was higher (such as Christmas and August).
Benefits associated with summary care record use
We observed clinical consultations in which the SCR seemed to add value, especially when patients had been prescribed multiple medications and were unsure what these were (table 2⇓). We observed no cases in which an SCR unequivocally made care safer, or in which absence of one seemed to make care unsafe. This seemed to be because doctors, nurses, and pharmacists tended to err on the side of caution and took additional safety measures if key data were not available. However, because the incidence of harm from medication errors appears to be low in relation to the number of prescriptions issued in the primary care setting34 and the SCR was little used in any other setting, our findings do not exclude a significant positive impact on patient safety. When dealing with complex cases, clinicians sometimes found it stressful and challenging to make decisions in the absence of reliable data on medication and allergies and described the SCR as making such consultations “easier” even when it did not change management.
We encountered no examples of SCR use influencing onward referral (for example, preventing a hospital admission or ambulance call), nor of a patient without an SCR whose onward referral decision was likely to have been different had one been present. We found it impossible to assess whether patients were more satisfied when their SCR was accessed, because satisfaction was a function of the consultation as a whole, not of the presence or absence of a particular technology.
Risks associated with summary care record use
We encountered examples of incomplete and inaccurate data on the SCR (table 2⇑). However, we did not see any cases where this led to harm or risk of harm to the patient—precisely because clinicians did not view the SCR as the sole source of reliable data. Rather, they drew eclectically on multiple sources, including the patient, electronic and paper records, and their own observations and measurements. When these sources conflicted, they made a contextualised judgment about which data source was most trustworthy and the level of residual uncertainty in the case.
Explaining longer consultations with the summary care record
One possible confounding variable explaining this unexpected finding is that the SCR was more likely to be accessed for complex illnesses or when the patient had more than one complaint. Our qualitative data also showed that when information held on the SCR was poorly matched with the scope of practice of the clinician (such as a long medication list viewed by a nurse without senior level prescribing training), its use sometimes led to a time consuming attempt to make sense of the information viewed by consulting a formulary, asking a colleague, or questioning the patient.
The socio-technical network
The SCR programme can be conceptualised as a complex, dynamic, and unstable socio-technical network with multiple interacting sub-networks (fig 1⇑ and box 3). Individuals and organisations in the network represented four different institutional “worlds”—political, clinical, technical, and commercial.35 The personal world of the patient formed a fifth perspective. In the political world, the programmes were an exercise in modernising the NHS by improving efficiency in the public sector and delivering measurable benefits to patients and taxpayers.2 In the clinical world, they were an initiative to improve quality of care in an area (unscheduled care) where concerns had been raised about standards.28 In the technical world, they were a software development project for use in cases characterised by unpredictability and a high degree of exceptionality. In the commercial world, many IT companies viewed them as high risk but potentially high revenue business contracts. In the personal world, the SCR was a potential encroachment (for good or ill) of the system into the patient’s lifeworld.36
Box 3: The socio-technical network for the summary care record programme
Five interacting sub-networks were evident:
1. The design network
Software developers (based variously in Connecting for Health, commercial IT companies, and academic institutions)
A large, complex, and evolving technical infrastructure
2. The implementation network
National and local managers
Frontline NHS staff
Individuals and groups who sought to “resist” implementation in different sectors
Patient organisations and other groups who encouraged implementation from the user perspective
3. The governance network
Professional, legal, and regulatory bodies (such as the Information Commission)
Technical security features and the Connecting for Health staff who designed and built them
Security testing contractors
Business processes, tools, and systems that supported information governance activity
Individuals such as Caldicott Guardians and privacy officers
4. The frontline user network
Local administrators and call handlers
“Front end” software linking local systems to the SCR infrastructure
Terminals and smart card readers
Patients who were consulted on access to their record
5. The evaluation network
Policy makers and business managers who constructed the “benefits realisation” case
Teams and systems involved in in-house monitoring
Official bodies such as the Public Accounts Committee
Lobbyists (including the press) who made claims and counter claims about the justification and progress of the programme
Communications staff and systems within Connecting for Health
Our own evaluation team and governance structures, including the Connecting for Health Evaluation Programme
Differences in norms, values, priorities, expectations, and ways of working between these five worlds, and imperfect attempts to bridge these differences, accounted for much of the instability in the socio-technical network, and this in turn explained many of the challenges and frictions encountered as the collaborative tasks of design, implementation, governance, frontline use, and evaluation (box 3) were pursued.
Change agents who proved most pivotal in the programme held boundary roles between different organisations and sectors and managed to align—to some extent at least—the complex and competing institutional logics that characterised the network. They achieved this by engaging actively in what has been called “translation,”37 which involves four stages—problem construction (defining a problem for which the SCR offered a solution), selling the idea (getting others to accept this problem-solution link), enrolment (defining key roles and practices in the socio-technical network), and mobilisation (engaging people in fulfilling the roles, undertaking the practices, and linking with others in the network).
The PRINCE 2 approach to project management (www.prince2.com/) adopted by Connecting for Health, strategic health authorities, and some primary care trusts—in which explicit goals and milestones are systematically defined, pursued, and signed off—appeared to be an efficient business tool for managing the parts of the programme that could be controlled, isolated into discrete work packages, and “managed” in the conventional sense of the word. But it seemed that the sheer complexity of the socio-technical network and its embeddedness in wider institutional structures, some heavily contested, limited the impact of this change model in critical parts of the programme.
At the micro level, frontline staff from different “worlds” brought different beliefs, values, motives, and meaning-systems to their organisational roles (“normative” influences). Their actions were also shaped and constrained by such things as job descriptions, access privileges, and the functionality and limitations of technologies (“causal” influences). Relatively small differences in normative and causal influences on individuals sometimes explained wide variations in actions and outcomes (see examples in box 2). Inbuilt in the SCR were assumptions about individual behaviour and the culture and work practices of NHS organisations—for example, that fear of surveillance would not deter staff from accessing them and that general practice staff would enter key data in coded fields on the local records from which they were created. Mismatches between these inscribed assumptions and the reality of clinical work explained much of the non-adoption, partial adoption, and abandonment of these technologies at the level of the individual user.
This evaluation has shown that some progress has been made in introducing shared electronic summary records in England and that some benefits have occurred. However, significant social and technical barriers to the widespread adoption and use of such records remain, and their benefits to date appear more subtle and contingent than early policy documents predicted.
The largest single factor explaining non-use of the SCR at the clinical frontline was the identity of the clinician. As previous researchers have found, “most CIS [computerised information systems] implementations fail because, despite high investments in terms of both time and financial resources, physicians simply do not use them.”38 The finding that many out-of-hours clinicians do not access SCRs even when these are available with “one-click” access suggests that potential safety benefits (such as in preventing medication errors) may fail to accrue. Coiera has argued that e-health initiatives may focus too much on electronic records and too little on associated functionality such as automated alerts, though the extent to which this potential solution would overcome the problem of non-adoption by clinicians in practice is speculative.39
A key strength of this study was the use of a new theoretical and methodological approach, derived from a systematic literature review of electronic record research that deliberately went beyond the biomedical literature and embraced sociology, computer supported cooperative work, and empirical philosophy.40 This enabled us to combine qualitative and quantitative techniques to highlight the competing conceptualisations and complex interdependencies of the SCR programme and to bring into frame numerous social, technical, ethical, and political explanations for why particular goals and milestones set by policy makers and implementation teams were or were not reached.
The main limitation of the study is perhaps the extent to which this approach will be viewed as valid by the e-health community. Our analysis of the programme in context was based on a widely accepted methodological standard in information systems research.19 However, this standard is not accepted by some healthcare information systems researchers, who define robustness in terms of experimental or quasi-experimental studies of the “deployment” of a technology and its “impact” on predefined outcomes.41 The profound epistemological differences and lack of dialogue between healthcare information systems research (led largely by doctors with an interest in information technology) and mainstream information systems research (led by interdisciplinary teams of organisational sociologists, computer scientists, and political scientists) have been highlighted previously.40 42 The findings reported here will not resolve these differences but may promote a long overdue debate on the applicability of the epidemiological hierarchy of evidence to studies of complex change in health care.
Examples of apparently successful national programmes to share records between primary and secondary care exist in Scotland, New Zealand, and Denmark.43 44 45 46 47 48 49 50 In Denmark, for example, all the country’s general practices, hospitals, pharmacies, and laboratories are linked via a single national network, and electronic transfer of prescriptions, laboratory results, letters, and emails is the norm, though (perhaps significantly) there is no centrally held record. These national schemes differ in some fundamental technical features, but all report similar critical success factors: a pace of progress commensurate with levels of engagement and tension for change; early and frequent dialogue between key stakeholders to develop a culture of collaboration; systematic attention (such as via focus groups or discussion networks) to potentially contentious issues; strong peer influence (such as through clinical user groups); careful alignment of incentives for both individuals and organisations; transparency in monitoring and evaluation; and balancing central and local leadership. These key influences are reflected in three recent systematic reviews on implementing electronic record programmes more generally.40 51 52
The scale and scope of the SCR programme inevitably increased its complexity. Because the socio-technical network is heterogeneous and potentially unstable, alignments between stakeholders must be actively built and creatively managed—a task that gets exponentially more difficult as the network gets bigger.53 An industry report on the Danish shared record scheme recommended focusing on a simple, basic design and concluded that “increasing the level of complexity does not bring a corresponding increase in benefits.”50 The three countries whose shared record systems are the most advanced all have populations of around 5 million, though it is not known whether this figure is a critical ceiling or a coincidence.
The findings of this evaluation suggest that large scale, centrally driven models for innovation in healthcare IT systems are—perhaps inevitably—associated with multiple competing perspectives, complex interdependencies, inherent tensions, and high implementation workload. Information system development is sometimes depicted in terms of a tension between the “cathedral” model (large scale, top down, highly standardised) and the “bazaar” model (organic, laissez-faire, bottom up).54 The former are more readily standardised and controlled but are typically complex, expensive to implement, and inflexible. The latter risk inconsistent quality control and limited interoperability. A third option, “middle out”—whose defining feature is collaboration between government, industry, and clinicians to create an evolving set of standards and promote dialogue across sectors—has been proposed as a potential way forward for the National Programme for IT.54 Such an approach would require a fundamentally different change model and changes to the nature of relationships between the state, commercial suppliers, clinicians, and service users, but could potentially build fruitfully on work done in the SCR programme to date.
What is already known about this topic
Policy makers, clinicians, and citizens in many countries hope that shared electronic records will improve quality, safety, and efficiency of health care by making key information accessible across multiple care settings
No country with a population above 5 million has successfully introduced a nationally shared electronic patient record
In 2007, the English Department of Health began a programme to introduce the nationally shared summary care record (SCR), intended primarily for use in emergency and unscheduled care
What this study adds
Between 2007 and 2010, the SCR programme met numerous challenges, both social and technical, and was characterised by high implementation workload and complex interdependencies
When SCRs were available in unscheduled primary care settings, some clinicians accessed and used them some of the time
Benefits associated with use of SCRs were more subtle and contingent than early policy documents had predicted
Technical and operational aspects of the programme could not be meaningfully isolated from subjective and contextual issues such as what the introduction of the SCR meant politically, professionally, practically, and personally
Cite this as: BMJ 2010;340:c3111
This evaluation would not have been possible without the cooperation of a number of people and organisations. Patients and staff in the various field sites allowed us access to their work and experiences, gave freely of their time, and engaged with our desire to gain an honest, richly informed account of events. Adastra provided a large quantitative dataset and background information which allowed us to undertake an independent analysis. The evaluation benefited from the Summary Care Record Independent Evaluation (SCRIE) External Advisory Group, chaired by lay member Jenni Bowley, which served as a diverse, critical, and questioning audience for our emerging findings. Members (listed in full elsewhere14) included representatives from patient groups, academic institutions, professional organisations, and Connecting for Health. The study was part of the Connecting for Health Evaluation Programme coordinated by Richard Lilford’s team at the University of Birmingham. Academic colleagues too numerous to list provided feedback and guidance throughout the study.
Contributors: TG designed the study, led the development of the theoretical and methodological approach, helped with data collection, led the qualitative data analysis, drafted the paper, and is responsible for the integrity of the work. She is the guarantor. KS and TB led the qualitative data collection and helped with analysing the data and writing and revising the paper. HWWP led the quantitative data analysis and helped write and revise the paper. EB provided technical input to the study and helped write and revise the paper. JR helped develop the theoretical and methodological approach and provided expertise in evaluation methodology. Rob Stones from University of Essex provided theoretical and methodological advice. Deborah Swinglehurst assisted with data collection and analysis. Enrico Coiera, Mike Pringle, and Ole Hanseth reviewed an earlier draft of the paper and added additional insights and references.
Funding: The independent evaluation of the summary care record programme was funded by a research grant from the UK National Institute of Health Research (ref CFHEP002 and 007). The Medical Research Council funded a parallel research study (“healthcare electronic records in organisations,” ref 07/133) by our team in which the SCR formed one case example. The analysis presented in this paper draws on both those studies. University College London (UCL) held the research grants; the empirical work was undertaken while all authors were employed at UCL. Queen Mary University of London (QMUL) funded the salary of TG during the writing-up phase.
Independence from sponsors: All aspects of the study (including design; collection, analysis, and interpretation of data; writing of the report; and decision to publish) were led by the authors. Sponsors, along with other stakeholders, were given an opportunity to check drafts for accuracy, but the final decision on whether to amend drafts was made by the authors in consultation with the SCRIE External Advisory Group. The views expressed in this publication are those of the authors and not necessarily those of our employing institutions, the individual members of our advisory group, the National Institute for Health Research, Connecting for Health, or the Department of Health.
Competing interests: All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that (1) all authors have support from their respective institutions (QMUL and UCL) for the submitted work; (2) none of the authors has any relationships with any companies that might have an interest in the submitted work in the previous 3 years; (3) their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (4) no authors have any non-financial interests that may be relevant to the submitted work.
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.