Being old, depressed, and disabled is to be in triple jeopardy

EDITOR—Llewellyn-Jones et al have provided strong evidence that a multifactorial intervention for late life depressive illness has a measurable beneficial effect.1 I have provided specialist medical services to the community that Llewellyn-Jones et al studied and as a researcher have tried to study similar participants in clinical trials of multifactorial interventions. Research into rehabilitation, falls, and geriatric evaluation and management share the same issues as depression.

Haynes (in his editorial accompanying the paper)2 and Deeks and Juszczak (in their commentary)1—and the rapid responses to the paper3—raise important issues. Although this area of clinical investigation remains in development, it is clinically relevant research. The researchers did well to follow up the percentage of participants that they did. The number eligible was 220, and they managed to have outcomes for 185 (85%). This included 15 participants who died: death is a legitimate end point for the frail older people studied.

The study showed an improvement of about 2 points on the 30 item geriatric depression scale. Is this worthwhile? As a clinician I vote yes. Remember that this is the real world of care of older people, with limited resources and hard pressed nursing staff, personal care staff, and general practitioners. If the intervention works in Llewellyn-Jones et al's large and architecturally outdated facility in Sydney it will be even more effective in well resourced retirement communities. In the United Kingdom the structure of general practice (which encourages closer medical supervision of frail older people) should also improve the effectiveness of the intervention.

The intervention has components that should be available to all older people as a right. Callahan argues for a basic humane health service as a minimum for all older people.4 The intervention falls into this league. Cost effectiveness analyses are unlikely to support the types of programmes pioneered by Llewellyn-Jones et al unless they reduce the need for admission to hospital or increased help with activities of daily living. Because admission to hospital with depressive illness is uncommon in the population studied and most participants already required some help with activities of daily living, sample sizes for a cost effectiveness study are likely to be large. In a population with a genuine unmet health need it is almost axiomatic that it will cost more to meet this need.

Evidence based health care seems to be better accepted if the evidence supports a lower cost intervention. If the evidence supports the efficacy of a more costly intervention healthcare managers and planners seem less interested.

Important research seems to have been greeted with only two faint cheers

EDITOR—I read the paper by Llewellyn-Jones et al on multifaceted shared care intervention for late life depression in residential care,1 together with Haynes's accompanying editorial2 and the commentary by Deeks and Juszczak.1 I was left with the impression that this important piece of research had been greeted with two faint cheers rather than the three heartier ones it probably deserved.

Neither the editorial nor the commentary makes the important points that, firstly, depression among elderly people is common, underdetected, undertreated, and an appreciable public health problem; secondly, depression among elderly people in residential care is extremely common, grossly underdetected, and grossly undertreated and seems to have an appalling prognosis3; and, thirdly, because the aetiology of such late life depressions is usually multifactorial we need to evaluate multipronged interventions among large populations.

It is difficult to do good quality research on depression in residential care, and the editorial and the commentary on the paper emphasise this. Although the design of Llewellyn-Jones et al's study can be criticised, it would seem virtually impossible to mount both arms of such a trial simultaneously within one large residential institution.

The variability that would be produced by using two or more institutions would probably outweigh the temporal variability introduced by Llewellyn-Jones et al's methodology. Even modest improvements in depression scores and modest changes in general practitioners' behaviour may have considerable impacts on overall population morbidity from depression. A small decrease in alcohol use in a community is associated with a considerable benefit to some individuals at risk, and the same may be true of small improvements in depression scores.

Finally, it seems unfair for Haynes to criticise the dropout rates in the study. If you are going to do research with very old people some of them are going to die and any intervention which prevented that would certainly be worth a headline.

How much trial and error should we tolerate in community trials?

EDITOR—Haynes's editorial1 refers to a trial by Llewellyn-Jones et al in which the modest result is attributed to a variable degree of programme implementation.2 Haynes assures us that trial and error is a necessary part of the evolution of trials. This ignores the broader question of how much effort we should devote to getting an intervention right before we put it to the ultimate test (the randomised controlled trial)—particularly ambitious interventions that set out to “change the care culture.”2

When we depart from the tidy world of drug trials to the murky world of community trials, where do we draw the line between efficacy and effectiveness? If we are too ready to accept the “real world” conditions of effectiveness trials we may risk a proliferation of state of the art evaluations of far less than state of the art interventions. This prospect seems extremely wasteful.

These issues are felt acutely in health promotion, where some spectacular failures have led to intensive soul searching about the differences between programme failure and evaluation failure and, in the event of programme failure, whether this is accounted for by implementation failure or theory failure. A randomised controlled trial is an unnecessarily expensive way of learning about implementation failure. This has led to the view that a cycle of implementation and review, of getting the implementation as right as feasibly possible, should precede the evaluation of programme outcomes. In other words, randomised controlled trials should have starting rules as well as stopping rules.

Haynes argues that we are only learning to run with community trials. This may be the case, but we are certainly not just learning to run with community interventions. Nor are we ignorant of methods to assess contextual factors in programme environments,3 or methods to guide change processes,4 or methods to assess implementation.5 This means that we are better equipped than ever to introduce programmes and optimise their functioning before testing.

Undoubtedly, professional judgment is required to determine whether implementation is as right as feasibly possible. What range and type of evidence and skill should be called on? How, for example, might we distinguish naturalistic conditions from poor programme management within a trial? A lot of this has not been assessed because many investigators seem to pay it scant attention. Expert criticism of intervention theory and strategy, as well as scrutiny of the criteria to be used to define intervention integrity, must be part of trial design and review.

Cochrane preferred to use “effective” where other people used “efficacious”

EDITOR—In his editorial on the testing of healthcare intervention1 Haynes quoted definitions of efficacy and effectiveness which he attributed to Archie Cochrane.2 Last has made the same attribution for his definition of effectiveness.3

But in referring to the need to apply the randomised controlled trial “to measure the effect of a particular medical action in altering the natural history of a particular disease for the better,” Cochrane stated: “It is in this sense that I use the word ‘effective’ in this book, and I use it in relation to research results, as opposed to the results obtained when a therapy is used in routine practice in a defined community. Some people would like to use the word ‘efficacious’ for this measurement. This seems reasonable, but as I dislike the word I have not used it here.” Hence he used the terminology that is almost exactly the opposite of that attributed to him.

Cochrane may not have been first to define efficacy and effectiveness

EDITOR—Haynes credits Archie Cochrane with first defining the terms efficacy, effectiveness, and efficiency as applied to health services.1 Archie did much to popularise the application of the terms, but they were first promulgated two years earlier at the 14th meeting of the World Health Organisation Expert Committee on Health Statistics in December 1970.2

The late Sir John Brotherston was in the chair and I, as the rapporteur, wrote the report. The concepts and terms, however, came from two other members of the committee: Dr A Sakari Härö (chief of the department of planning, National Board of Health, Helsinki, Finland) and the late Dr Georges Rösch (deputy director of the Centre de Recherches et de Documentation sur la Consommation, Paris, France). The definitions were as follows.

Efficacy: the benefit or utility to the individual of the service, treatment regimen, drug, or preventive or control measure advocated or applied.

Effectiveness: the effect of the activity and the end results, outcomes, or benefits for the population achieved in relation to the stated objectives.

Efficiency: the effects or end results achieved in relation to the effort expended in terms of money, resources, and time.

Any member of our committee could have discussed the terms with Archie, but the most likely candidates are John Brotherston or Dr W P D Logan from the United Kingdom; Dr Logan at that time was the director of the Division of Health Statistics at the World Health Organisation. Alternatively, Archie may have conceived the terms independently, but unfortunately he did not distinguish clearly between the first two.

Last is correcting the entry for the next (fourth) edition of the Dictionary of Epidemiology published by Oxford University Press.