Primary Care

Evaluation of aid to diagnosis of pigmented skin lesions in general practice: controlled trial randomised by practice

BMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7411.375 (Published 14 August 2003) Cite this as: BMJ 2003;327:375
  1. Dallas R English (dallas.english{at}cancervic.org.au), associate professor1,
  2. Robert C Burton, Professor2,
  3. Chris B del Mar, professor of general practice3,
  4. Robert J Donovan, professor4,
  5. Paul D Ireland, deputy director2,
  6. Geoff Emery, general practitioner5
  1. 1 The Cancer Council Victoria, Carlton, VIC 3053, Australia
  2. 2 National Cancer Control Initiative, Carlton, VIC 3053, Australia
  3. 3 Medical School, University of Queensland, Herston, QLD 4006, Australia
  4. 4 Centre for Behavioural Research in Cancer Control, Curtin University of Technology, Bentley, WA 6102, Australia
  5. 5 Perth Central Coastal Division of General Practice, Hollywood Private Hospital, Nedlands, WA 6009, Australia
  1. Correspondence to: D English
  • Accepted 9 June 2003

Abstract

Objectives To determine whether an aid to the diagnosis of pigmented skin lesions reduces the ratio of benign lesions to melanomas excised in general practice.

Design Controlled trial randomised by practice.

Setting General practices in Perth, Western Australia.

Participants 468 general practitioners in 223 practices.

Interventions Intervention practices were given an algorithm and instant camera to assist with the diagnosis of pigmented skin lesions. All practices were given national guidelines on managing melanoma.

Main outcome measures Ratio of benign pigmented lesions to melanomas excised. Analyses conducted with and without inclusion of seborrhoeic keratoses.

Results At baseline the ratios of benign to malignant lesions were lower in the intervention group than in the control group. During the trial period the ratios were higher in the intervention group (19:1 v 17:1 without seborrhoeic keratoses and 29:1 v 26:1 with seborrhoeic keratoses). After adjustment for patients' age, sex, and socioeconomic status, the ratio was 1.02 times higher (95% confidence interval 0.68 to 1.51, P = 0.94) in the intervention group when seborrhoeic keratoses were not included and 1.03 times higher (0.71 to 1.50, P = 0.88) when seborrhoeic keratoses were included. General practitioners in the intervention group were less likely than those in the control group to excise the most recent pigmented skin lesion they managed (22% v 48%, P < 0.001) and to refer the patient to a specialist (16% v 27%, P = 0.06).

Conclusions Provision of the algorithm and camera did not decrease the ratio of benign pigmented skin lesions to melanomas excised by general practitioners.

Introduction

Doctors are anxious not to miss melanomas because early diagnosis is associated with good prognosis.1 2 A melanoma screening trial with Australian general practitioners showed high sensitivity (0.95, 95% confidence interval 0.90 to 1.0) for the diagnosis of melanoma but low specificity (0.49, 95% confidence interval 0.41 to 0.57).3 Low specificity and low prevalence of melanoma, even in Australian general practice, means that for every melanoma excised between 10 and 28 benign naevi are excised; this rises to 35 if seborrhoeic keratoses are included.47 Similar ratios have been found elsewhere.8 Thus, reducing the number of benign pigmented skin lesions removed without decreasing sensitivity would reduce unnecessary surgery.

A trial of an algorithm and instant camera to improve general practitioners' skills in diagnosing pigmented skin lesions was conducted in two provincial cities in Queensland, Australia. The rationale was that patients would be reassured if steps are taken to check that a pigmented lesion shows no change.6 In the six months before the trial, the ratios of benign lesions to melanomas excised in the two cities were similar, but the ratio was significantly lower in the intervention city after the intervention. The number of melanomas excised in the intervention city, however, was higher during the intervention period than the baseline period. We replicated this intervention in urban general practice in Australia, using a controlled trial that randomised by general practice rather than by city.

Methods

General practitioners on the mailing lists of the divisions of general practice (geographically based groups with some similarities to UK primary care groups) in Perth were eligible. They had to agree to their practice being randomised and to pathology laboratories releasing data on pigmented skin lesions that they excised. None were planning to retire or relocate in the next 12 months or were already using equipment to monitor pigmented skin lesions. General practitioners who joined a practice after randomisation or with whom we had had no contact before randomisation (usually because the mailing lists were incomplete) were also eligible. In Australia general practitioners often work at multiple practices. Those who did so could participate at each practice, though they were asked to follow the protocol allocated to that practice.

During the randomisation visit, all practices were given national guidelines on managing melanoma.9 Randomisation occurred after the trial was explained and the doctors present had consented. The research assistants then gave doctors in the intervention group the aid to diagnosis and trained them to use it. The aid included an algorithm slightly modified from the original (fig 1)6 and an instant camera (Polaroid Spectra AF, Polaroid Australia Pty Ltd, North Ryde, NSW) with stand and film. The algorithm centres on change in appearance as a key to distinguishing melanomas from benign lesions.10 The doctors were informed that the aid did not replace normal clinical responsibility.

Fig 1
Fig 1

Algorithm to assist with the management of patients with pigmented skin lesions

We used a computer generated randomisation code consisting of random permuted blocks and stratified by practice size (one, two to three, and four or more general practitioners). A programmer prepared the codes and sealed them in envelopes that were labelled with the practice size. After randomisation, participants and research assistants who visited practices were not blinded to assignment. All coding of outcome data was done blind to assignment.

Evaluation was based on the ratio of benign to malignant pigmented skin lesions excised. We defined a malignant lesion as an in situ or invasive melanoma and a benign lesion as a naevus (including dysplastic naevus) or (in some analyses) a seborrhoeic keratosis. We included seborrhoeic keratoses because they are commonly mistaken for melanomas.7 All pathology reports on excisions of pigmented skin lesions from 1 November 1998 to 31 August 2000 were obtained from pathology laboratories.

At the end of the study we sent the general practitioners a questionnaire on how they had managed their last three patients with pigmented skin lesions. Because the management of the three lesions was similar we have reported information relating only to the last lesion.

Statistical analysis

The primary analysis was based on comparisons between groups. For consistency with the earlier trial6 we did not include seborrhoeic keratoses in this analysis. Lesions were counted only if they were excised by a participating doctor at a study practice. Because not all doctors in each practice took part in the trial we could do a strict intention to treat analysis. If practices merged we used data until the date of the merger.

We used logistic regression with generalised estimating equations and the sandwich estimator of the variance to analyse data on individual lesions, allowing for the clustering by practice.11 12 Odds ratios, their 95% confidence intervals, and Wald test P values were obtained from these models. The odds ratio was calculated as the ratio of benign to malignant lesions in the intervention group divided by the ratio in the control group.

All models included practice size and the patient's age, sex, and socioeconomic status. Age was analysed in four groups (< 30, 30-39, 40-54, and ≥ 55 years). Socioeconomic status was estimated from the patient's postcode and grouped into approximate quarters.13 Data for these covariates were missing for eight lesions (of 3822) in the trial period and four (of 4741) in the baseline period.

Four secondary analyses of the primary outcome were undertaken. Firstly, we included seborrhoeic keratoses. These lesions were included in all subsequent secondary analyses. Secondly, we excluded doctors who joined the trial after practices were randomised. Thirdly, we excluded five “specialist” general practitioners to whom other doctors refer patients with pigmented skin lesions because these specialists performed a substantial proportion of all excisions and four were in the intervention group. Fourthly, we tested an interaction between intervention group and the patient's age, which might modify any effect of the intervention because of its inclusion in the algorithm (fig 1).

To assess the effect of the intervention on the numbers of lesions excised (including seborrhoeic keratoses), we calculated the annual number of excisions for each practice and grouped them into categories (melanomas: 0, > 0 and < 1, 1, 2, ≥ 3; benign lesions: 0, > 0 and < 5, 5-12, 13-27, 28-49, 50-99, ≥ 100). (The category of > 0 covers those practices that excised one melanoma in a period of over one year; therefore they had excised more than none in one year but less than one.) We used Pearson's correlation coefficients to estimate changes within each intervention group between the baseline and trial periods and performed all analyses in Stata/SE 7.0 (StataCorp, College Station, TX, USA).

Calculations of sample size

Seborrhoeic keratoses were not included in the sample size calculations. We assumed that the ratio of benign to malignant lesions in the control group would be 23, which was the baseline average in the Queensland trial. The trial had 80% power (P = 0.05, two sided) to detect a 36% lower ratio in the intervention group during the trial period.

General practitioners in the Queensland trial (provincial cities with greater ratios of general practitioners to patients) excised 27 pigmented lesions per year; we estimated that general practitioners in Perth (a city of 1.2 million people) would remove 16.2 per year (60% of 27). We assumed that the intraclass correlation within practices for the diagnosis of melanoma was 0.015. We estimated that the maximum we could recruit was 450 general practitioners, or about 225 practices with two doctors per practice. To achieve 80% power required nine months of follow up. No interim analyses were planned or performed.

Results

Participation and randomisation

We identified 488 practices, of which 223 participated (fig 2). Practices were randomised between 14 September 1999 and 8 February 2000. The two groups had similar durations of baseline and trial periods and were similar in terms of total general practitioners per practice and the number that participated within each practice (table 1). The intervention group had slightly more practices in areas of highest socioeconomic status, and the control group had slightly more practices in the middle two quarters.

Fig 2
Fig 2

Flow diagram for practices, general practitioners, and excised pigmented skin lesions based on practices. *General practitioners counted once at each practice whereas all others counted once only in each group (ICC=intraclass correlation)

TABLE 1

Characteristics of practices in each group. Figures are numbers (percentage) of practices unless stated otherwise

View this table:

We identified 1221 GPs, of whom 468 participated in the trial (fig 2). Fourteen GPs working at more than one practice participated at multiple practices. Of these, five GPs (one women and four men) worked at multiple practices that were allocated to both groups. More GPs working at intervention practices participated in the study, which was partly due to more GPs in intervention practices joining the trial after the practice was randomised (fig 2). The control group included 81 women (37%) and the intervention group 91 women (36%).

Similar numbers of GPs in the two groups left their practices during the trial (fig 2). No other GPs withdrew from the trial. However, only 302 (65%) GPs completed the questionnaire at the end of the study (fig 2).

Excision of pigmented skin lesions

During the two periods, the participants excised 8563 pigmented skin lesions: 295 (3%) melanomas (180 invasive and 115 in situ), 529 (6%) dysplastic naevi, 5065 (59%) other naevi, and 2674 (31%) seborrhoeic keratoses. The ratio of benign lesions to melanomas was 19 without and 28 with seborrhoeic keratoses. More than half the lesions were excised from females (4878, 57%) and almost half (3928, 46%) were from patients less than 40 years of age. The number of excisions per practice ranged from none (42 practices) to more than 1000, with a median of 15. The number of melanomas varied from none (130 practices) to 31 with a median of 0.

During the baseline period, the two groups had similar annual numbers of excisions of benign lesions and melanomas within practices (table 2). Overall, the intervention group excised more melanomas and more benign lesions and had more favourable ratios of benign lesions to melanomas (table 3). The apparent inconsistency between the figures within practices and the overall totals was due to the imbalance in specialist general practitioners (four of the total (five) were in the intervention group). When we excluded these GPs, the number of excisions of benign lesions was similar in the two groups, although the intervention group excised more melanomas and therefore had a lower ratio of benign to malignant lesions (table 3).

TABLE 2

Number (percentage) of excisions per year of benign pigmented skin lesions (including seborrhoeic keratoses) and melanomas performed in each practice during baseline and trial periods, by trial group

View this table:
TABLE 3

Odds ratios and confidence intervals for analysis of benign pigmented skin lesions and melanomas

View this table:

Neither group showed substantial changes in the excision rates within practices between the baseline and trial periods (table 2), and the correlation coefficients for the categorised rates in each group before and after randomisation were small and not significant (intervention group: benign lesions r = −0.01 (P = 0.9), melanomas r = −0.01 (P = 0.9); control group: benign lesions r = −0.01 (P = 0.9), melanomas r = 0.07 (P = 0.3)). The overall rates showed little change in the control group, but decreased in the intervention group between periods (table 3), largely because of substantial reductions in a few practices with large numbers of baseline excisions (data not shown).

The intervention group had a slightly higher ratio of benign lesions to melanomas during the trial period (table 3). After adjustment for practice size and patients' age, sex, and socioeconomic status, the odds ratio from the primary analysis was close to unity. Its confidence interval was consistent with at most about 32% lower ratio of benign to malignant lesions in the intervention group (table 2). The interaction between intervention group and patients' age was not significant (P = 0.71). The odds ratios were similar for all secondary analyses (table 3).

Compliance and reported management of pigmented skin lesions

We provided intervention practices with 482 packets of film (4820 possible photographs), and 85 practices requested at least one additional packet of film during the trial.

One hundred and twenty (82%) control GPs and 131 (85%) intervention GPs who returned the questionnaire reported that they had dealt with at least one pigmented lesion during the trial, though four of these doctors gave no details on their management. One hundred and thirty (84%) intervention GPs reported that they photographed at least one patient. The median number of patients photographed was eight.

When we asked about the last patient with a pigmented lesion that they managed, GPs in the intervention group reported that they referred fewer patients to specialists, photographed more lesions, and excised fewer lesions (table 4). When patients were referred, there was little difference between the groups in choice of specialist (table 4; P = 0.45).

TABLE 4

Management of most recent patient with pigmented skin lesion, by trial group. Figures are numbers (percentage) of events

View this table:

Discussion

The provision of a camera and algorithm to general practitioners to help them manage patients with suspicious pigmented lesions did not decrease the ratio of benign lesions to melanomas they excised. During the trial period, the ratio was actually slightly higher in the intervention group. There was some evidence, albeit not significant, that the intervention group excised fewer melanomas during the trial period, raising the possibility that some melanomas were missed.

How valid are these results? Chance is unlikely to explain the apparent lack of effect on the ratio of benign to malignant lesions as the lower bound of the confidence interval for the primary analysis (0.68) excludes strong effects and is barely consistent with the 32% lower ratio of benign to malignant lesions in the intervention group of the previous trial.6

The ratio of benign to malignant lesions was lower in the intervention group before randomisation, which should have favoured finding a lower ratio during the trial period. Exclusion of the specialist general practitioners, most of whom were in the intervention group, had no impact on the results. Although more GPs in the intervention group joined the trial after their practice was randomised, their exclusion also had little effect. All analyses were adjusted for patient characteristics that might affect the ratio of benign lesions to melanoma—namely, age, sex, and socioeconomic status.

Compliance seemed high, as indicated by use of film and general practitioners' reports on the number of patients that they photographed. Our objective evidence on number of excisions by the intervention general practitioners suggests that most practices showed little change, but that in practices where there were many excisions before baseline, there was a reduction. We do not know what proportion of this change was due to the intervention rather than to regression to the mean. Some contamination might have been present because five general practitioners were in both groups, but fewer doctors in the control group reported photographing skin lesions, and in the control group excision rates were similar before and after randomisation.

How do we reconcile the results from this trial with those from the previous apparently successful one?6 The design and settings of the two trials were different. Our design is stronger because we randomised multiple practices rather than two cities. It was conducted in a metropolitan rather than a remote provincial setting, where there was only one dermatologist. Perth has numerous specialists in skin diagnosis to whom general practitioners could refer patients. We had no objective data on referrals to specialists, although general practitioners in the intervention group reported that they were less likely to refer patients in the trial period. Perhaps it is more common for general practitioners to refer diagnostically worrying lesions to such services in Perth, in which case we might have been measuring the wrong sort of lesion. Despite these differences, there is no obvious explanation for the discrepancy.

What is already known on this topic

A previous trial in Australia showed that the use of an instant camera and algorithm for the management of pigmented skin lesions in provincial general practice reduced the ratio of excisions of benign lesions to melanomas without reducing the number of melanomas diagnosed

What this study adds

A similar trial in a capital city in Australia, with a stronger study design and randomisation by practice rather than town showed no reduction in the ratio of benign to malignant lesions excised

The reasons are not clear, but the contradictory results may be related to differences in large cities rather than small ones (where specialists are not so available)

We had few eligibility criteria that would limit the external validity of the study. GPs particularly interested in skin lesions may have been more likely to participate, but the ratios that we observed were similar to that found in an analysis of all excisions sent to a pathology service in Victoria.7 GPs in Australia practise on a fee for service basis, and excisions of malignant skin lesions attract a higher payment than excisions of benign skin lesions. Our results may not be applicable to general practice in places where methods of remuneration differ or where melanomas are uncommon. The results are not relevant to other aids to the diagnosis of pigmented skin lesions such as dermatoscopy14 nor to whole body photography of patients with multiple atypical naevi.15 16

In conclusion, our results do not show that photography of pigmented skin lesions in general practice in Australia decreases the number of benign lesions excised without compromising sensitivity of the diagnosis of melanoma.

Acknowledgments

We thank the members of the Perth Project Team—Jim Annear, Roland Gaebler, John Hilton, Damien McCann, and Mary Surveyor—who represented the Perth Divisions of General Practice and facilitated the conduct of the study. We thank Mark Elwood, Peter Heenan, John Kelly, Richard Lovell (deceased), Graham Mason, and John Primrose, who were members of the Project Management Committee that provided scientific advice. Peter Heenan advised on the coding scheme for pigmented lesions. We thank Robin Marks, the independent evaluator of the trial, for his constructive criticism.

Footnotes

  • Contributors RCB, DRE, GE, CBDM, and PDI were responsible for the initial study design. DRE and GE supervised the conduct of the study. DRE performed the data analysis. RJD conducted focus groups with the general practitioners and designed the questionnaire. All of the authors contributed to the final report. DRE and RCB are the guarantors for the paper. Chris Costa was the senior project officer, and Jade Nolan assisted her. Helen Bartholomew programmed the database and developed the randomisation code.

  • Funding The work was supported by a contract awarded to the National Cancer Control Initiative by the Commonwealth Department of Health and Aged Care. The guarantor accepts full responsibility for the conduct of the study, had access to the data, and controlled the decision to publish.

  • Competing interests None declared.

  • Ethical approval The Human Research Ethics Committees of the Royal Australian College of General Practitioners and the University of Western Australia approved the study protocol. No consent was sought from patients, whose names were not sought.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
View Abstract