Does certification improve medical standards?
BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38933.377824.802 (Published 24 August 2006) Cite this as: BMJ 2006;333:439All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Having practised medicine in Asia,
Europe, Africa and North America, I found the recertification
process and the continuing medical education system in the United States
of America to be the most helpful in keeping up with the fast advancing
frontiers of my chosen profession.
At 72, I practise alongside newly minted, young men and women doctors. I
love it !
Competing interests:
Associate Professor of Psychiatry and Colonel United States Army.
Competing interests: No competing interests
In his rapid response Dr Adam pringle refers to "immigrants in the
homeland of litigation and the KluKluxKlan". I suspect that the many
American readers of the BMJ find this as offensive as I do. Moreover, it
is likely that Dr Pringle would be equally offended if an American
corresepondent with this journal referred to Britain as the "homeland of
the British National Front".
As it happens, historically America's treatment of her immigrants has
arguably been far better than Great Britain's.
Competing interests:
None declared
Competing interests: No competing interests
Few will dissent from the thesis put forward by Sutherland and
Leatherman that regular recertification of doctors is likely to raise
standards of practice, even if the evidence they quote in support of it is
less than convincing. The primary purpose of the GMC’s proposals for
revalidation has from the outset been to raise standards by requiring all
doctors to collect information about what they do, to reflect on it and to
identify areas for improvement. The revalidation proposals are, however,
more ambitious than those for recertification ratified by the American
Board of Medical Specialties in that revalidation will be compulsory, not
voluntary, may in a minority of cases lead to restriction of a doctor’s
right to practise and will focus on what doctors do rather than what they
know.
It is wholly mistaken to state that revalidation is intended to be “a
peer appraisal process”. Evidence of appraisal will be a requirement for
revalidation but will not determine the outcome. Rather, revalidation will
be based upon evidence drawn from a variety of sources and, for most
doctors, evidence drawn from clinical governance will be crucial. The
process of recertification designed by ABMS is unable to draw on clinical
governance data since there is no standardised and quality assured system
of clinical governance in place across the USA, and alternative approaches
were therefore inevitable.
Of the four types of evidence required for recertification by the
ABMS, “professional standing” will be assessed under revalidation through
clinical governance data and/or colleague and patient questionnaires.
“Commitment to lifelong learning and … self assessment” will be assessed
through evidence of continuing medical education and appraisal. The
principal difference between the American and British approaches concerns
the use of examinations and “standards based evaluation of performance”.
The extreme variability of doctors’ range and scope of practice calls into
question the feasibility and fairness of examinations. Valid tools for
assessing standards of practice are scarce and assessments are liable to
be confounded by resource issues and the performance of other members of
the team in which doctors work.
This paper makes a useful contribution to the debate on how best to
ensure the highest possible standards of medical practice through a
process of recertification/revalidation. Comparisons between different
healthcare systems in different jurisdictions are however liable to lead
to erroneous conclusions. It is not clear that a process tailored to the
needs and circumstances of the US healthcare system is the ideal one to
adopt in the UK.
Competing interests:
As chairman of the GMC's Registration Committee,the author has been extensively involved in developing the Council's proposals for revalidation
Competing interests: No competing interests
Individual knowledge assessment is useful but it is easy to have good
knowledge and be a bad doctor. General practice is about making multiple
decisions, under pressure and trying to do no harm. In group practices the
systems within the practice and the opportunity for peer review are
probably much more relevant than the an individual GP's performance at an
examination or during a video consultation. A practice should demonstrate
a team approach to continuing development and maintenance of the
knoweledge base and show evidence of attempts to follow EBM. This should
be combined with frequent exposure of work to peer review. Those factors,
I suspect, are far more useful in terms of accrediting the doctors within
that team than individual appraisal and examination.
Focusing on the
individual, outwith his or daily practice, risks overvaluing a doctor's
knowledge base over his or her effectiveness and efficiency. On the latter
note, being efficient allows one to provide more accessible services and
being more available which should form part of reaccreditation. An
accessible service is putting itself in a position to diagnose early
disease and have the greatest impact. That accessibility probably would
not usually form part of the individual assessment of a doctors potential
performance. Similarly in terms of efficiency, GP is an increasing a
clerical job both at individual and team level and assessment of the
team's clerical organisation as well as the individual's could point to a
more effective doctor than knowledge base and abstract assessment.
In summary, is the doctor working within the team, is he exposed to peer
review, is he showing a willingness to have his work examined on a regular
unselected basis, has the team good clerical organisation which encourages
good practice in its members.
I have doubts that individual appraisal can indentify the effectiveness of
individuals as practitioners within general practices
Competing interests:
None declared
Competing interests: No competing interests
Leatherman and Sutherland pose the question- does certification
improve medical standards?- and attempt to answer the question in the
affirmative. However there are a few lingering points that require
comment.
First the authors assume a universality of practice across the UK
royal colleges in terms of membership examinations which is not correct-
surgery and public health, until very recently- for instance, differ from
the model described.
Second, there is no mention in the paper of the Postgraduate Medical
Education and Training Board (PMETB) which is currently in the process of
setting standards and approving curricula for specialist training across
specialties in the United Kingdom.
Third, the authors quote the systematic review that found that 52 per
cent of studies found a negative association between clinical performance
and years of qualification without mentioning some of the important
methodological limitations of that review.
Fourthly, the authors quote statistics that show that problem doctors
are less likely to be certified- but this raises the chicken and egg
question- are they uncertified because they have problems that prevented
them from being certified or vice versa?
Finally, with regard to revalidation which is a major component of
the proposals- these have only been introduced in the US in 2000 and no
evidence has been presented to show that it is effective. Even if one
accepts that certification leads to better performance, there is little in
this article to show that recertification also works.
There is no doubt that transparent,robust, valid and reproducible
systems are needed to ensure that doctors are well trained and provide
quality care, but it is important that in making changes, we are guided by
the evidence of what works.
Competing interests:
None declared
Competing interests: No competing interests
The authors present a glossy puff in favour of certification, but
only by the most gross selection of evidence, which should not have passed
the Editor.
They state that Shipman Ledward Ayling et al demonstrate failures in
our system of regulation - they do not. They demonstrate failures to
implement our systems. Nobody suggests that cars that crash do so because
they have no brakes.
They ignore the fact that the 15% of doctors not certified in the US
will differ from those who are certified, thus any difference may be
unrelated to the certification process. I suggest uncertified doctors are
more likely to be recent immigrants, in rural or isolated areas, in poorer
public hospitals in poor parts of town, recently qualified or on the brink
of retirement - all of which are likely to have a bearing on morbidity and
mortality.
They make no mention of the fact that certification is an income-
generating process - thus those issuing certificates have an incentive to
produce research demonstating that they are beneficial - and we already
know positive results are more likely to be published than negative ones.
Despite the impact of selection bias, nearly half (13 out of 29, and
30 out of 62) of the analyses show no difference (or perhaps a negative
impact?) for certification. Might I suggest the absence of a meta-analysis
demonstrating significant benefit is because it failed to do so?
Most importantly, the authors make much of the stated fact that
mortality from lower bowel surgery is lower in the hands of certified
surgeons - yet gloss over the fact that sub-specialty certification shows
no benefit. Surely these facts are of equal merit, and between them show
no clear benefit?
They state that lack of certification is associated with disiplinary
action. Surely this includes a small group of failing doctors, who allow
certification to lapse as they know action is pending, and a much larger
group of immigrant doctors. We know already that our own disciplinary
process falls disproportionately heavily on immigrants, and immigrants in
the homeland of litigation and the KluKluxKlan are unlikely to fare
better.
Finally, and most importantly, they ignore the massive difference
between a voluntary process and a compulsory one. The history of the
Western World shows that monopolies routinely exploit their customers,
provode a poor level of service, and fail to protect their customers - the
performance so far of the PMETB being a typical example, while one
volunteer is worth ten pressed men.
Competing interests:
None declared
Competing interests: No competing interests
Specialty certification in developing countries is a new
cocept,however it was started in the Middleast ten years ago,mainly in the
Arab countries through the Arab Countries Medical Specialities Board.
The concept of Appraisal and Revalidation is yet to be established.
Competing interests:
None declared
Competing interests: No competing interests
I read with interest Sutherland and Leatherman's analysis of
certification and revalidation in the UK. They describe the US boards of
the medical specialties 'broadly analogous' to the royal colleges in the
UK, however this is simply not true. The royal colleges in the UK are
each very different, and indeed inconsistent in terms of the level of
certification they confer. Take for example the Membership of the Royal
College of Physicians (MRCP) examination taken by thousands of Senior
House Officers (SHOs) or equivalent during their rotations through the
medical specialties. The examination does nothing to assess specialist
training or practice - it is the same qualification regardless as to
whether you are an SHO with 6 months experience in chest medicine or a
consultant rheumatologist with 30 years experience! Other royal colleges
(namely General Practice, Psychiatry, Paediatrics and others) operate
their certification process similarly to the physicians.
In contrast, the surgical royal colleges currently confer Membership to
the Royal College of Surgeons (MRCS) to appropriately trained and examined
SHO's in surgery (an examination that is to be phased-out with PMETB) and
only at the end of 5-6 years of sub-specialist training and examination
confer a sub-specialty FRCS (eg FRCS Orth, FRCS Plast etc). Thus the
surgical model confers the Certificate of Completion of Specialist
Training (CCST) only when Specialist Training has been completed and
EXAMINED, not simply at the end of registrar work!
Certification of doctors in this country is hopelessly inconsistent.
Hospitals, doctors and patients deserve a far more structured and
consistent certification process from the royal colleges. Only when we
have proper certification in place, can we address re-certification and
revalidation with all the merits it offers.
Competing interests:
None declared
Competing interests: No competing interests
There is a strong evidence that patient care has improved because of
certification. The PMETB in UK which is a relatively new regulatory body
is doing commendable job in assessing the specialty status of doctors. At
a later stage, PMETB may recommend and implement a rather simple and
benign process for re-validation after 5-10 years of initial certification
which will go a long way in ensuring good standard of care and
performance. Unfortunately, the developing countries are still not
seriously considering the benefits of re-validation process where doctors
are responsible for the treatment of large number of people. In a country
like Pakistan, there is an acute need for monitoring and evaluation of
practice methods. The need for establishment of an appropriate
"specialists' register"
following the UK's PMETB model will greatly help the health system of the
country and will go a long way in not only benefitting the population but
also the doctors who will endeavour to keep themselves updated with
latest knowledge and get the skills sharpened.
Competing interests:
None declared
Competing interests: No competing interests
Recredentialling - linking recertification to QI
Recredentialling – linking recertification to QI.
Having read the paper on recertification of doctors1 (BMJ 2006; 333:
439-41) may I suggest that Britain could learn from the experience of
recredentialling of senior hospital medical staff developed in New
Zealand. This has been based on a Ministry of Health working party report
(Toward clinical excellence. Ministry of Health, 2001:
http://www.moh.govt.nz)
Like recertification, recredentialling is repeated every 3-5 years.
Whilst its primary goal is to ensure patient safety through examination of
the competence of individual doctors, it also focuses on their
professional and personal well being and development. These may be
addressed in a number of ways including further training and improved work
conditions - excessive workload, limited work space and poor equipment
may, for example, all diminish the quality of a doctor’s work.
Whilst recredentialling is required by the organization the process
is owned by senior doctors. The interests of
consumers are being recognised through the increasing inclusion of a
consumer representative on the recredentialling committees which are
composed of senior doctors, not only from within but also outside the
organization. All information received or discussed with individual
doctors is confidential and undiscoverable but the committees’ final
reports are publicized. These include defining individual doctors’ scopes
of practice, particular development needs and recommendations to enhance
their work through changes within their clinical services and working
environments.
Support for credentialling by both senior medical staff and
management within this DHB is reflected by the whole or partial
achievement of over 83% of recommendations made by credentialling
committees. Recredentialling links the rigorous scrutiny of individual
doctors’ practice to quality improvement by also involving others,
including management, to improve their working environments
Competing interests:
None declared
Competing interests: No competing interests