How can we get high quality routine data to monitor the safety of devices and procedures?BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f2782 (Published 07 May 2013) Cite this as: BMJ 2013;346:f2782
- Bruce Campbell, chair of the interventional procedures advisory committee 1,
- Andrew C Stainthorpe, associate director 2,
- Carole M Longson, director 2
- 1National Institute for Health and Care Excellence, London SW1A 2BU, UK
- 2Centre for Health Technology Evaluation, National Institute for Health and Care Excellence, London, UK
- Correspondence to: B Campbell
Difficulties with medical devices and discussions at the European Commission about new regulations have focused attention on how to introduce devices and procedures safely.1 2 Both medical devices and new procedures are typically introduced with little evidence from clinical trials or observational data. The regulatory requirements for research evidence on the efficacy and on the short and long term safety of new devices are universally less stringent than those for medicines.3 To protect patients from harm we need to gather as much information as possible about adverse events; for implanted devices the data should be over the long term.4 Recent events such as the Poly Implant Prothèse (PIP) breast implant scandal show that data gathering is still not happening as it should. We examine the reasons for inadequate collection of data and explain what steps might be taken to improve it.
Why is the evidence on safety so limited?
Many medical devices are produced by small specialist companies that lack the finance and experience to conduct adequate research, and the relatively short “market life” of many devices also militates against continuing research. Within Europe, regulations require manufacturers to obtain a CE mark for a new device from any of the many “notified bodies” throughout the European Union. The degree of scrutiny for CE marking depends on the type of device, but the amount of clinical evidence needed is typically small.
Not all new procedures use a new medical device, and vice versa. For example transcatheter aortic valve replacement (TAVI) was a new procedure generated by production of a specially designed valve and delivery system. By contrast, injection of autologous blood to treat tendinopathies or fasciopathies is a novel procedure but involves no new device. There are no legal frameworks governing the introduction of new procedures, and the evidence base for them is seldom good. Reasons for this include absence of commercial sponsorship, the difficulties of setting up research trials, and clinical culture (not least a perceived lack of equipoise).5 Further complexities for procedures are defining when a procedure is new, rather than a minor modification of an existing one, and working out how outcomes are influenced by the person who does it.
All these factors mean that when a device or procedure enters clinical use there are limited data on its efficacy and short term safety. Adoption of the new device or procedure is typically driven by marketing and the enthusiasm of clinicians, rather than by evidence. Information about the long term safety of a device or procedure is unlikely to be available at this stage. As the device or procedure is used in routine clinical practice more evidence may accumulate about its efficacy and short and long term safety. However, there is no organised system—in the UK or elsewhere—to collect data on new devices or procedures each time they are used. Manufacturers are required to record and act on reports of adverse events, but they depend on clinicians and other users to inform them.
What has been done to improve safety?
Countries across the world are developing frameworks to try to guard against the use of devices and technologies that might harm patients. The European Commission, which is responsible for producing EU legislation, has adopted proposals for two new regulations on medical devices and in vitro diagnostic devices.6 The aim of the regulations is to improve patient safety, advance the functioning of the life sciences sector within the EU, and support innovation and competitiveness. If the proposed regulations are agreed before the European parliament elections in June 2014, the revised legislation is likely to apply from 2017.7
Procedures have posed a real challenge because of the lack of any legal regulation and any system to collect data to inform their use. Internationally, various organisations evaluate new procedures but have differing methods, outputs, and aims (such as informing reimbursement).8 All these organisations are hampered by the frequent paucity of evidence and would benefit from more information about long term outcomes.
Taking the UK as an example, in response to high complication rates of early laparoscopic surgery and the Bristol children’s heart surgery scandal of the 1990s, the National Institute for Health and Care Excellence (NICE) set up the interventional procedures programme in 2002. Its remit is to evaluate new or controversial procedures using published evidence on efficacy and safety, together with input from specialist clinical advisers and patients.9 10 NICE then publishes guidance on how new procedures should be used in the National Health Service.11
When the evidence on safety or efficacy of a procedure is inadequate, NICE guidance commonly recommends that clinicians tell their hospital that they are using it, tell patients about the uncertainties, and critically review their results. The guidance may also stipulate whom to involve in patient selection and treatment, the facilities and training needed to perform the procedure safely, and the additional evidence needed to guide future use. The fundamental aims are to protect patients from harm while fostering access to innovative procedures in a safe and controlled way.
When the evidence on a procedure is insufficient, NICE would like to see collection and ultimate publication of information every time the procedure is done in the UK. This pragmatic approach would provide observational data to supplement, and hopefully validate, the findings of clinical research and support robust conclusions when NICE guidance is updated (box 1). NICE has recommended data submission to specific registers for 72 of 367 procedures it has evaluated since 2002. It has used data from registers in its evaluation of procedures such as minimally invasive total hip replacement, off-pump coronary artery bypass grafting, and carotid artery stenting.11
Box 1: Value of well designed collection of observational data (registers)
Provides data on “real world use”
Supplements the (often sparse) research literature
Highlights safety problems
Can be linked with other data sources (readmissions, deaths)
Such comprehensive data collection can provide information about where procedures are being done and the results of different hospitals and clinicians. Importantly for patients, it can identify devices or procedures associated with excess adverse events. The withdrawal of a hip prosthesis as a result of data from the National Joint Registry in the UK and registers in other countries is a prime example of how successful this approach can be.12
What difficulties remain?
For procedures, one reason for poor data collection in the UK is the time it takes to produce codes to identify new procedures as part of the Hospital Episodes Statistics recorded when every patient is discharged from hospital. Any good record of a new procedure includes details of devices that have been used, such as implants. The delayed introduction of an efficient electronic patient record is another factor.
Setting up registers for specific procedures or devices is challenging, and getting comprehensive submission of data is difficult (box 2). Clinicians may have insufficient incentive (in terms of personal benefit and feedback of useful data), inadequate time, and insufficient support to submit data routinely. Allocating the time and funding for staff to perform data entry to registers has traditionally been a low priority.
Box 2: Obstacles to comprehensive data collection
Clinicians—lack of enthusiasm and time
Payers—uncertainty about the quality and value of data
Hospitals may not appreciate the need to support data submission
No commercial sponsor (for procedures with no device) or several manufacturers of competing devices
Large amount of work required compared with reward for low volume procedures
Complexities of international collaboration in gathering data
Information about new devices and procedures could be accrued more rapidly if the data from registers in different countries were aggregated.13 This would require major investment of time by clinical leaders, persuasion of many clinical specialists, sufficient financial and organisational support in hospitals, and attention to the legislative constraints governing consent, data confidentiality, and electronic sharing of information in different countries.
How can these problems be solved?
Below we describe solutions to some of these problems that could be implemented in the foreseeable future.
Device tracking—Universal adoption of an effective device tracking mechanism would make it easier to trace individual patients, recall devices, and improve monitoring and data collection on the functioning, durability, usage, and costs of a device.14 This would require every device to carry a unique machine readable identifier (such as a bar code). In the United States, the Food and Drug Administration published proposals in July 2012 for unique device identifiers on all medical devices, and the European Commission has included them in its draft revision of regulations.
Coding procedures—For new procedures, the early provision of specific codes to allow differentiation from other, similar, interventions is important. The move towards the SNOMED (Systematised Nomenclature of Medicine) coding system may help to achieve this. SNOMED Clinical Terms is part of an initiative by the International Health Terminology Standards Development Organisation to create international consistency in development and use of clinical terminology.15 SNOMED aims to offer storage, retrieval, and aggregation of clinical data in a way that will work well in electronic health records systems.
Use of registers—Within individual countries and health systems, moves to encourage use of existing procedure registers (like NICE guidance in the UK), to adapt registers for new procedures, and, where necessary, to set up new ones, must continue despite the challenges highlighted above.13 Funding is key, and the recent PIP breast implant scandal may provide a stimulus by emphasising the risks of failing to support good data collection. The model of the National Joint Register in the UK, which includes a small amount for data submission in the cost of each procedure, could perhaps be considered more widely.16 Dedicated staff to submit all the required data have been key to the success of the UK National Adult Cardiac Surgery Audit and the Swedish Arthroplasty Registers.
Data linkage—New registers should be linked to routinely collected health services data, national mortality statistics, and other possible sources of relevant information. Such links can greatly enhance the overall value of the data collected by registers. For example, links to the Clinical Practice Research Database in the UK, in which general practitioners enter data about their patients, could allow follow-up of patients who have received procedures and devices.
Consent—One potential problem with registers is gaining patients’ consent to use their data. In the past, potentially valuable registers have failed because patients did not agree to their data being entered: the breast implant register in the UK is a topical example. Some countries, such as Sweden and Denmark, do not require the consent of patients for inclusion of data.17 When data collection is in the public interest such a policy seems justified, but introducing it in other countries will require resolve and political pressure.18
International collaboration—International collaboration has already begun in some areas. Examples include the work of the FDA in the United States with the International Consortium of Orthopaedic Registries.19 The FDA’s recognition of the importance of this kind of collaboration is a boost to sharing data from different countries and may help to stimulate similar initiatives elsewhere.
Post-market surveillance—Post-market surveillance data collected by device manufacturers, most of which trade internationally, could provide useful information about use of products worldwide if manufacturers agreed relevant outcomes and allowed independent oversight and transparency of the data. However, manufacturers are reticent about the risk of exposing their data to competitors. Furthermore, there is concern that manufacturers could be selective about the data they release and that the data they collect may not include the most relevant outcomes. Negotiations to overcome these difficulties would be worthwhile.
New procedures—There is now a well recognised framework for generating evidence on any new procedure from its first use into the long term. The IDEAL principles (idea, development, evaluation, assessment, and long term) include registers as the main way to acquire long term data.20
Will it happen?
None of the currently proposed changes in legislation, in Europe or elsewhere, includes all of our suggestions to improve acquisition of data. But developing the systems we have described would go a long way towards producing observational data that would enhance the safety of devices and procedures and facilitate decisions about their place in healthcare.
Cite this as: BMJ 2013;346:f2374
Contributors and sources: BC and CL have worked together since 2002, leading the establishment and evolution of the NICE interventional procedures programme and the work aimed at promoting data collection on procedures. AS is associate director of the interventional procedures programme. BC is the primary author who planned and developed the text. AS conceived the idea of the article, contributed to the literature search, made suggestions about the text, and dealt with submission. CL edited and helped to develop the text.
Competing interests: We have read and understood the BMJ Group policy on declaration of interests and have no relevant interests to declare.
Provenance and peer review: Not commissioned; externally peer reviewed.