Rapid Responses to:

EDITORIALS:
Phil Alderson and Trish Groves
What doesn't work and how to show it
BMJ 2004; 328: 473 [Full text]
*Rapid Responses: Submit a response to this article

Rapid Responses published:

[Read Rapid Response] Over-reliance on pharmacotherapy in prison-based smoking cessation programs doesn’t work: to prove it, just ask smoker prisoners and prison anti-tobacco workers.
Niyi Awofeso   (27 February 2004)
[Read Rapid Response] Cause and effect?
Susan R Corby   (27 February 2004)
[Read Rapid Response] I agree
Emmanuel UDEZUE   (27 February 2004)
[Read Rapid Response] ‘Well after all what do any of us really know?’ ‘That’ll be enough of that, Wilson’
David G Connell   (2 March 2004)
[Read Rapid Response] Unknown Unknowns
Christopher E Nancollas   (2 March 2004)
[Read Rapid Response] So many unknowns, such serious consequences
Elizabeth M Witherington   (2 March 2004)
[Read Rapid Response] What doesn't work and how to show it
Alan R Giles   (5 March 2004)
[Read Rapid Response] Patient uncertainty - scientific uncertainty
Robert Fox   (7 March 2004)
[Read Rapid Response] I don't know
Simon M Loader   (7 April 2004)
[Read Rapid Response] Response to "What doesn't work and how to show it"
Joaquim P Ribeiro, R. da Beneficência, nº8 - 1069-166 Lisboa Portugal   (14 June 2004)

Over-reliance on pharmacotherapy in prison-based smoking cessation programs doesn’t work: to prove it, just ask smoker prisoners and prison anti-tobacco workers. 27 February 2004
 Next Rapid Response Top
Niyi Awofeso,
Conjoint Senior Lecturer, University of New South Wales, Sydney 2052, Australia
NSW Corrections Health Service

Send response to journal:
Re: Over-reliance on pharmacotherapy in prison-based smoking cessation programs doesn’t work: to prove it, just ask smoker prisoners and prison anti-tobacco workers.

“Professor Aristotle would have avoided the tragic misconception that men had three more teeth than women if he had simply, and courteously, asked Mrs. Aristotle to open her mouth” - Anonymous.

Although there is a clear political and professional risk in trying to challenge medical and public health zeitgeists1, failure to do so when proof of their ineffectiveness is evident can only harm the progress of the health sciences, as reasons for the lag of polio research in the early decades of the 20th century demonstrate.2

Over the past decade, interest in implementing tobacco control programs, and basic smoking cessation (‘Quit’) programs in Western prison settings has heightened, due largely to litigious and public health reasons. For instance, the United Kingdom’s Department of Health has recently agreed to fund Nicotine Replacement Therapy (NRT) for 73,000 British prisoners, about 80% of who smoke (http://www.sepho.org.uk/viewResource.asp?).

The addiction model is a dominant framework adopted by most health workers involved with smoking cessation programs (http://www.rcplondon.ac.uk/pubs/books/nicotine/4-addiction.htm), and all but two of BMJ’s “ABC of smoking cessation” series have emphasized this model. Consequently, provision of NRT to prison populations - epicenters of an ignored public health epidemic3 – is ‘naturally’ regarded by health workers as the antidote to one of the greatest hurdles to setting up prison Quit programs. However, inmates have markedly different perceptions, as a recent survey in an Australia prison shows (Table 1).

Table 1: Smoking cessation obstacles in an Australian prison – prisoners’ survey.

	Smoker prisoners	Non-smoker prisoners
Average Inmates’ count during survey week     160		
Number of survey respondents	67	20
Number that believe it is more difficult to quit in prison than on the ‘outside’	58	7
Of participants that believe that it is more difficult to
quit in prisons than on the ‘outside’, reasons given (i.e.
frequency of each listed response):		
1) Boredom	32	6
2) Stress	39	4
3) Everyone else smokes	24	5
4) Addiction	10	
5) Helps cope with depressing prison environment	9	1
5) Nothing better to spend money on in prisons	6	

Only 7% of responses by smoker inmates were related to addiction as an obstacle for smoking cessation, compared with 23% for boredom, 28% for stress, and 17% for high smoking prevalence. Although the enhanced accessibility of NRT in this prison (for the 40% of smoker inmates serving sentences of six moths or greater) explains the fact that only one of 139 responses cited NRT cost as an obstacle to smoking cessation, only about 9% cited NRT availability as a major obstacle in the author’s unpublished 2001 survey of 1068 Australian prisoners, when NRT was not provided free of charge to inmates in the surveyed prisons. It is also noteworthy that while 88% of smoker prisoners surveyed believed that it was more difficult to quit tobacco use in prison compared with community settings, only 35% of non-smoker prisoners shared the same view.

If prison-based smoking cessation efforts are to yield optimal results it is important for health workers to adopt a social marketing framework4, and address Quit obstacles through smoker prisoners’ perspectives. This paradigm shift should involve a greater appreciation of the core values of prisoners, and the promotion of smoking cessation programs in such a manner that it accords with those core values. Furthermore, the primary objectives of prison-based smoking cessation programs should be framed as having a clear benefit for inmates not necessarily from a health perspective, but perhaps more importantly, in terms of satisfying prisoners’ core values.

Over-emphasis on an addiction model, or a health education model (inadequate knowledge about tobacco’s health effects was regarded as a major impediment to Quit efforts by only about 2% of inmates in the author’s 2001 unpublished survey), at the expense of obstacles accorded greater importance by inmates, such as boredom or stress, is indicative of a “selling approach” to facilitating social change, and is therefore less likely to result in successful prison-based Quit program outcomes. Admittedly, the reality of prison life is such that most major obstacles perceived by inmates cannot be fully surmounted without compromising prison security. Thus, the use of strategies to enhance motivation and self-efficacy to sidestep such obstacles5 should be integral to prison Quit programs.

References

1. Alderson P., Groves T. What doesn’t work and how to show it. British Medical Journal 2004; 328: 473.

2. Benison S. Polio research in the United States: appraisal and lessons. In: Holton G, editor. The 20th century sciences – studies in the biography of ideas. New York, W. W. Norton & Company, 1972: 308-343.

3. Cropsey K., Eldridge GD, Ladner, T. Smoking among female prisoners: an ignored public health epidemic. Addictive behaviors, 2004; 29: 425-31.

4. Awofeso, N. Implementing smoking cessation programmes in prison settings. Addiction Research and Theory, 2003; 11: 119-130.

5. Secker-Walker RH, Flynn, BS, Solomon L.J. et al . Helping women quit smoking: baseline observations for a community health education project. American Journal of Preventive Medicine, 1996; 12: 367-377.

Competing interests: None declared

Cause and effect? 27 February 2004
Previous Rapid Response Next Rapid Response Top
Susan R Corby,
Reader in Employment Relations
University of Greenwich London SE10 9LS

Send response to journal:
Re: Cause and effect?

Human resource management interventions are even more problematic than healthcare interventions. Nevertheless, the Department of Health is commissioning research on improving health though human resource management (HRM). It wants to find out which human resource (HR) interventions are most effective in “increasing productivity and achieving better outcomes for patients”.

Meanwhile researchers from Aston University UK in a study funded by the National Health Service (NHS) Executive (north Thames region) have already found “strong links” between the presence of sophisticated personnel practices, particularly appraisals, teamwork and training, and lower patient mortality in acute hospitals. Between 1999-2000 the researchers collected information on personnel policies, procedures and practices from 61 NHS trusts. In addition, they collected performance data on patient mortality, using complex statistical procedures which included standardising for such variables as patients’ ages and primary diagnosis.

Yet just because the researchers found an association does not mean that there is a link, whatever their claims. Statistical associations can be spurious and misleading as King’s Fund researchers demonstrated. They found an association between the World Health Organisation’s (WHO’s) rankings of the health system performance of a country and the country’s football rankings by the Federation Internationale de Football Association (FIFA).

Also, there could be reverse causation. The Aston researchers’ information on personnel practices referred to the period 1999-2000 and the mortality data spanned the period 1995-2000. So lower patient mortality could have caused sophisticated personnel policies.

Unfortunately an HR intervention is even more difficult to measure than a medical intervention, not least because of the problems of measuring health service productivity. (Is it quality or quantity?) Indeed, research by the University of Greenwich into how one HR intervention, a new pay system, was evaluated found that health service HR practitioners were sceptical that links could be demonstrated between a personnel policy and practice and productivity or health outcomes.

The Department of Health, however, is not deterred. In conjunction with the Chartered Institute of Personnel and Development and the Association of Health Human Resource Management it is offering funds of up to £250,000 for research on how human resource management can contribute to “positive employee attitudes and behaviours and to improved health service performance” with the possibility that the research may be extended with additional funding.

Researchers from cash strapped universities are queuing up. Perhaps the money would be better spent on reducing patient queues.

References

www.doh.gov.uk/callsforproposals.htm (accessed 18 Feb 2004)

West MA, Borrill C, Dawson J, Scully J, Carter M, Anelay S. et al. The link between the management of employees and patient mortality in acute hospitals. Int. J. of Human Resource Management 2002;13:1299-1310

Appleby J, Street A. Health system goals: life, death and… football. J of Health Services Research & Policy 2001;6:220-225

Corby S, White G, Dennison P, Douglas F, Druker J, Meerabeau E. Does it work? evaluating a new pay system. London: University of Greenwich, 2002

Competing interests: None declared

I agree 27 February 2004
Previous Rapid Response Next Rapid Response Top
Emmanuel UDEZUE,
Consultant Physician
ARAMCO MEDICAL SERVICES, SAUDI ARABIA

Send response to journal:
Re: I agree

Doctors need to be aware of, and avoid, intellectual pride. True education should humble us, not make us proud. More knowledge makes us more aware of the things we do not know. And it is the quest to find out more about these unknowns that extend the frontiers of knowledge.

Competing interests: None declared

‘Well after all what do any of us really know?’ ‘That’ll be enough of that, Wilson’ 2 March 2004
Previous Rapid Response Next Rapid Response Top
David G Connell,
Principal in General Practice
Fyvie Oldmeldrum Medical Group

Send response to journal:
Re: ‘Well after all what do any of us really know?’ ‘That’ll be enough of that, Wilson’

‘Well after all what do any of us really know?’ ‘That’ll be enough of that, Wilson’(1)

I write to offer my wholehearted support to a BMJ issue exploring ‘What we don’t know we don’t know.’ Could I suggest en passant for the dismissive non-believer that some small consolation is offered by a print run on a slightly more absorbent paper?

I was ashamed to speak up that day and admit that Rumsfeld’s cumbersome statement held distinct resonance.

As a GP I spend my day being appraised of those areas of my knowledge that are deficient. It is very educational. I am now being rewarded for keeping a running tally of all those things I didn’t know.

It strikes me that it is harder for my Specialist colleagues to speak up about their educational needs, but how would I know that?

It strikes me that Richard Feynman (2)was close to the truth in 1963:

‘In talking about the impact of ideas in one field on ideas in another field, one is always apt to make a fool of oneself.’

‘Some people say ‘How can you live without knowing?’ I do not know what they mean. I always live without knowing. That is easy. How you get to know is what I want to know’

There is much to discuss in this arena – perhaps too much, but let us at least sow the first seeds of humility for future generations of patients and doctors to reap.

And I do still like the tale of the ‘Emperors New Clothes’- and if you are sitting comfortably children our story today will take us through – the Johari window.

David G Connell

1 Dad’s Army BBC All of our lives

2 Feynman, Richard P. A Scientist looks at Society Cal.Tech 1963

Competing interests: Ego,Id and the ridicule of my colleagues

Unknown Unknowns 2 March 2004
Previous Rapid Response Next Rapid Response Top
Christopher E Nancollas,
GP, Newnham, Glos
Newnham surgery, High St, Newnham, Glos GL14 1BE

Send response to journal:
Re: Unknown Unknowns

Sir,

It would be an excellent idea to have a theme issue on medical unknowns.

After 25 years in medicine, it seems to me that there are far more unknown unknowns than there are known knowns, or, to put it slightly differently, there are very few 'facts' I learnt at medical school which are facts now.

In a world where targets and NSF's are driving the practice of medicine, and patients expect certainty, it is important to acknowlege what we don't know.

You must be the only publication which has used Donald Rumsfeld's phrase seriously, which I admire because I always thought it made sense. In Cornwall we used to have a phrase ' He don't know enough to know he knows bugger all' which is sort of the same thing.

Yours sincerely,

Chris Nancollas

Competing interests: None declared

So many unknowns, such serious consequences 2 March 2004
Previous Rapid Response Next Rapid Response Top
Elizabeth M Witherington,
Clinical Assistant
Nottingham City Hospital, Hucknall Rd, Nottingham NG5 1PB

Send response to journal:
Re: So many unknowns, such serious consequences

EDITOR- 'What we don't know we don't know' is a familiar concept in General Practice. I remember being introduced to this idea as a GP trainee in 1989: at first it seemed almost a nonsense - of course we know what we don't know- how could we not, after so many years' training? It's only through experience that we come to appreciate the extent of our ignorance. It is a further, more difficult step to be able to accept this in our work: we are trained to 'know'. In medical training and practice, ignorance is so often equated with failure.

Perhaps this is why we are reluctant to admit our limitations. Yet, what we don't know we don't know is at the heart of so many examples of poor communication and adverse outcomes for patients, eg

-the doctor who is 'often wrong but never in doubt'.

-the attitude that, because the patient's symptoms don't add up to a diagnosis I can recognise, the patient must be either a) depressed b) mad or c) 'swinging the lead'. There is nothing so instantly depressing as being told by a doctor that troubling symptoms do not exist or matter, and that the consultation is closed. It is more time-consuming, but ultimately more constructive to admit ignorance, (having, as far as possible, excluded important serious conditions), enable patients to get on with their lives in spite of their symptoms and permit review if things change or get worse.

- the classic GP consultation where the presenting symptom is just an 'entry ticket' for the patient to test out the doctor's receptiveness to a much more troubling or embarrassing problem. How many brief 'pointless' consultations are required before the true complaint is broached?

-communication between medical colleagues: the absent or vanishingly brief GP letter which compromises care for emergency admissions.

-the absent or incomplete hospital discharge communication which contributes to adverse events. Local examples include patients who died because the discharge information did not include significant results or monitoring recommendations. The GP cannot know what information is missing. Our local audit shows that about 15% of older patients had problems with medication within 4 days of discharge.

With the new GP contract, patients are less likely to see a doctor who knows them out of hours. With changes in junior doctors' working patterns, immediate discharge information (the 'TTO form') is less likely to be written by a doctor with extensive knowledge of the patient. This combination increases the risks for those already at greater risk of adverse events, ie complex, often older patients, who are more likely to be discharged to different destinations (intermediate and residential care), and new GPs. The TTO form is often the only form of discharge communication available for several days or weeks. It is at the same time more important and more difficult to get it right for these patients.

In summary, communication between doctors has been shown to be poor for at least 20 years. New working practices mean that there are even more things that we don't know we don't know about patients, which are putting lives at risk. If ever there was a time to push for improvement, it is now.

Competing interests: None declared

What doesn't work and how to show it 5 March 2004
Previous Rapid Response Next Rapid Response Top
Alan R Giles,
Professor Emeritus Dept. Pathology & Medicine
Queen's University, Kingston, Ontario K7L 3N6, Canada

Send response to journal:
Re: What doesn't work and how to show it

It is a pity that in what is otherwise an excellent and thought provoking editorial, Alderson & Groves appear to provide credibility for the ruminations of Donald Rumsfeld by suggesting that most of us have missed his point.1 Surely we can only be unaware of what we do not know by simply not asking the question either by default or deceit. We would be deluding ourselves, however, if we assumed that politicians have the monopoly of such manoeuvres.

During my career in medicine, I have oscillated through a number of roles, starting in very basic clinical science and then passing through primary care and pharmaceutical medicine before settling in academic medicine as, I hope I can describe myself, a clinical scientist. During these varied experiences I have seen many examples of the situations described in the articles of this excellent issue of the BMJ. I would draw on two that I believe are germane to the debate.

During my stay in primary care, which came relatively soon after qualification, I became very quickly aware and threatened by how relatively little I actually knew. This was particularly with regard to the recurrence of rubellaform rashes in otherwise relatively well children where rubella had been ‘diagnosed’ some months or years previously. This lead me to adopt an ‘educational’ policy including an open discussion with the anxious parent and a frank admission of how little we knew about such things thus denying a firm diagnostic title. Sometimes it worked but more often than not it did not. The likely reason for this became clear when I noticed that one particularly dissatisfied parent had re-consulted my senior partner the same day and left satisfied with a ‘diagnostic title’ and prescription for antibiotics for her child. On challenging this decision, I was given paternalistic advice on the ‘realities’ of general practice and the need to prioritise my time commitment between education and action. Fortunately, I left general practice before I was forced to consider this advice seriously but I am sure my advisor was not alone in following the path of least resistance rather than “just standing there and doing nothing”.

The second, more positive, experience occurred when I re-entered academic medicine. Perhaps overly sensitive to criticism of my time spent off the academic ladder, initially I had a tendency to bluster when on uncertain ground. My mentor, who had a propensity for scientific and personal directness, quickly provided the diagnosis and the correct remedial treatment. Simply put, I was advised never to be afraid of admitting ignorance (an ugly word but to the point!) no matter how ignominious the immediate consequences could be. The long-term benefits were twofold. Firstly it was a question of self-preservation. History tends to judge individuals kindly when ignorance has been freely admitted. It is less charitable when this is initially denied but uncovered later. Secondly, and probably of primary importance, it is only by admitting what is not known that the need to know is appreciated and investigated to the limit of each individual’s resources. It is not good enough to say, “we don’t know what we don’t know”. Denial is a powerful force in both the clinical state and its managers. In both cases, its release, when handled appropriately, can be extraordinarily beneficial.

It would be foolish to underestimate the pressures under which physicians and primary care physicians in particular work and the burgeoning responsibility imparted by the general public’s faith in their physician’s complete understanding of the ‘secrets of life’. Over the history of medicine we have used this faith, or should we call it trust, to benefit our patients in their hours of medical need.

Unfortunately, it is also open to significant abuse. Although still woefully incomplete, there has been an exponential increase in our knowledge of human disease and its management. Now is surely the time for us to shed defensive postures and admit we do not know when we do not because we have asked the question and the answers were not forthcoming. This lesson should be taught in our medical schools. It was not a message I was taught in the sixties and I suspect nothing has changed. It should.

1. Alderson P. Groves T. What doesn’t work and how to show it. BMJ 2004;328:473

Competing interests: None declared

Patient uncertainty - scientific uncertainty 7 March 2004
Previous Rapid Response Next Rapid Response Top
Robert Fox,
Consultant Obstetrician
Maternity Unit, Taunton & Somerset Hospital, Taunton TA1 5DA

Send response to journal:
Re: Patient uncertainty - scientific uncertainty

I read the collated articles about the assessment of clinical effectiveness in the February 28th edition with great pleasure. Alderson & Groves stated that ‘the rules for deciding "yes" are relatively clear and well known, but less has been written about deciding that something doesn't work.’ They discuss the reasons why it can be difficult to determine that an intervention is ineffective. I would like to propose a possible reason why we cannot be certain about effectiveness.

The double-blind, placebo-controlled, random allocation trial is seen as the gold standard of study design for testing new interventions. It is thought to circumvent the problems of observer bias and to control for the placebo effect. However, as well as a positive placebo effect, can we be sure that the effect of any uncertainty of random allocation on the patient does not have a negative effect on the efficacy of the active agent? And could such an effect would underscore the effectiveness of the active agent?

One could imagine a study of a novel intervention with a 40% effectiveness, which is reduced by one third by the uncertainty of allocation, for a symptom with a placebo response rate of 40%. The result of a placebo trial would show as a 51% response for the active arm (27% true effect and 24% placebo effect) and 40% for the placebo. If properly powered, the research would conclude that the active agent was effective. An open study would have tended to overestimate the effectiveness but a blinded study would underestimate the value of the drug. The result of such an effect could erroneously lead to a conclusion that an agent is insufficiently effective (NNT too high), or result in an underestimate of cost-effectiveness. It might also limit conclusions of n=1 trials. If such an effect does exist one might expect it to apply more to the subjective assessment of symptoms such as pain or nausea as opposed to objective disease markers.

We should always bear in mind that it is not nature we onserve but nature exposed to our method of questioning.

Competing interests: None declared

I don't know 7 April 2004
Previous Rapid Response Next Rapid Response Top
Simon M Loader,
Consultant RAWM Hospital, Khulna, Bangladesh
RAWM Hospital, PO Box 91, Khulna, Bangladesh

Send response to journal:
Re: I don't know

Alderson & Groves suggest that ‘what we don’t know we don’t know would be a good topic for a BMJ theme issue’. But, do you know what? It couldn’t be done. For to write about what we don’t know, we must sureley know we don’t know it first, otherwise how could it be an issue? The only way it could work would be that those who know they don’t know something, but think the rest of us don’t know we don’t know it, write about it so that the rest of us then also know we don’t know it. Then everything in that issue will no longer be unknown unknowns, but known unknowns. Do you know what I mean?

Competing interests: None declared

Response to "What doesn't work and how to show it" 14 June 2004
Previous Rapid Response  Top
Joaquim P Ribeiro,
director of UCI- medical doctor
Hospital de Curry Cabral- UCI,
R. da Beneficência, nº8 - 1069-166 Lisboa Portugal

Send response to journal:
Re: Response to "What doesn't work and how to show it"

I agree with you when you say "Perhaps what we don't know we don't know".

I found it curious that I also finish my article "Normalities, abnormalities and interspace diseases" with a similar phrase. It was publised in "Anamnesis, year 9, nº 94, November 2000, page 15".

I said in portuguese: "desconhece-se muito do que se julga conhecer e desconhece-se ainda mais do que se julga desconhecer"; (In english something like: "great is our unawareness of how little we actually know, but greater is still our unawareness of how much we do not know".)

Best Regards
J Palmeiro Ribeiro

Competing interests: None declared