Monitoring adherence to drug treatment by using change in cholesterol concentration: secondary analysis of trial dataBMJ 2011; 342 doi: http://dx.doi.org/10.1136/bmj.d12 (Published 21 January 2011) Cite this as: BMJ 2011;342:d12
- Katy J L Bell, research academic1,
- Adrienne Kirby, biostatistician2,
- Andrew Hayen, senior lecturer, biostatistics1,
- Les Irwig, professor of epidemiology1,
- Paul Glasziou, director3
- 1Screening and Test Evaluation Program (STEP), School of Public Health, University of Sydney, NSW 2006, Australia
- 2NHMRC Clinical Trials Centre, University of Sydney
- 3Centre for Research in Evidence Based Practice, Bond University, QLD 4229, Australia
- Correspondence to: A Hayen
- Accepted 13 October 2010
Objective To estimate the accuracy of monitoring cholesterol concentration for detecting non-adherence to lipid lowering treatment.
Design Secondary analysis of data on cholesterol concentration in the LIPID (long term intervention with pravastatin in ischaemic disease) study by using three measures of non-adherence: discontinuation of treatment, allocation to placebo arm, less than 80% of pills taken.
Setting Randomised placebo controlled trial in Australia and New Zealand.
Participants 9014 patients with previous coronary heart disease.
Interventions Pravastatin 40 mg or placebo daily.
Main outcome measures Sensitivity, specificity, area under the receiver operating characteristics (ROC) curve, post-test probability.
Results Monitoring of cholesterol concentration had modest ability for detecting complete non-adherence. One year after the start of treatment, half (1957/3937) of the non-adherent patients and 6% (253/3944) of adherent patients had a rise in concentration of low density lipoprotein cholesterol. Accuracy was reasonable (area under the curve 0.89). Cholesterol monitoring, however, had weak ability for detecting partial non-adherence. One year after the start of treatment, 16% (34/213) of partially adherent and 4% (155/3585) of fully adherent patients had a rise in concentration of low density lipoprotein cholesterol. Accuracy was poor (area under the curve 0.65). For typical pre-test probabilities of non-adherence ranging from low (25%) to high (75%), the post-test probabilities indicate continuing uncertainty after lipid testing. A patient with no change in low density lipoprotein cholesterol concentration has a post-test probability of being completely non-adherent of between 67% and 95% and a post-test probability of being partially non-adherent of between 48% and 89%. A patient with a decrease in concentration of 1.0 mmol/L has a post-test probability of being completely non-adherent of between 7% and 40% and a post-test probability of being partially non-adherent of between 21% and 71%.
Conclusions Monitoring concentration of low density lipoprotein (or total) cholesterol has modest ability to detect complete non-adherence or non-persistence with pravastatin treatment and weak ability to detect partial non-adherence. Results of monitoring should be considered as no more than an adjunct to careful discussion with patients about adherence.
After starting patients on lipid lowering treatment, clinicians might monitor them for several reasons, not all of which are evidence based.1 Some evidence supports monitoring cholesterol concentration soon after the start of treatment to check the initial response2 3 and to evaluate long term trends.3 On the other hand, the value of such monitoring to detect non-adherence remains unproved. Despite this, several clinical guidelines recommend regular monitoring to assess and improve adherence. The National Cholesterol Education Program suggests that “Lipoprotein profiles should be assessed at least annually, and preferably at each clinic visit to promote compliance.”4 In the United Kingdom, the NHS CKS (National Health System Clinical Knowledge Summaries) recommend that clinicians can “review the lipid profile (e.g. to assess compliance)” after starting patients on treatment for the primary prevention of cardiovascular disease.5 In Australia, the National Heart Foundation recommends that “patients at high risk should have their lipid levels measured every 6-12 months as part of the ongoing assessment of adherence.”6
Non-adherence can be classified as partial or complete—that is, continuing to take treatment but at a dose that is considerably less than that prescribed, or stopping taking treatment altogether (also referred to as non-persistence).7 Though lipid lowering drugs such as statins are generally well tolerated,8 non-adherence is thought to be an important cause of decreased effectiveness in the community.9 Non-adherence increases with time from the start of treatment: the prevalence of non-adherence to statin treatment in patients aged 65 or older was estimated as 29%, 38%, 42%, and 56% after six months, one year, two years, and five years of treatment.10 Different groups of patients have different rates of non-adherence; at two years 75% of patients prescribed a statin for primary prevention and 40% of patients with an acute coronary syndrome were non-adherent.9 Many patients fail to take the full amount prescribed: of those taking at least 20% of the drugs prescribed to them, about four in 10 were taking less than 80% of the drugs after one year.10
For patients to receive the full clinical benefits observed in randomised trials, adherence to lipid lowering treatment needs to be maximised. For clinicians to target interventions to increase adherence11 to patients most in need, they need to be able to reliably detect non-adherence. Methods to detect non-adherence include “direct” methods such as directly observed treatment and blood drug concentrations and “indirect” methods such as questionnaires, pill counts, prescription refill rates, measurement of physiological markers (such as cholesterol concentration), and electronic medication monitoring.12
We evaluated the performance of monitoring cholesterol concentration during the 12 months after the start of treatment as a means of detecting non-adherence to statin treatment.
We analysed changes in cholesterol concentration seen in the LIPID trial.13 Patients in that trial were eligible if they had previous coronary heart disease and had a plasma concentration of total cholesterol of 4.0-7.0 mmol/L. After a dietary run-in period, they were randomised to 40 mg of pravastatin or matching placebo. We compared adherent and “non-adherent” patients in different ways.
Comparisons of “complete non-adherence”
To assess whether complete non-adherence could be detected we used two groups:
Discontinuation—Among patients in the active treatment group, we classified those who reported that they were still taking any pills as “adherent” and those who reported that they had stopped as “non-adherent.” This is a direct method of comparing (partially or fully) adherent with completely non-adherent patients.
Placebo—Regardless of their actual level of compliance, patients in the active treatment group were classed as “adherent” and patients in the placebo group were classed as “non-adherent.” This is an indirect method of comparing completely adherent and non-adherent patients in which the use of placebo is a proxy for a non-adherent group of patients. The larger number of non-adherent patients analysed with this method allows for more robust statistical inferences to be made.
Comparison of “partial non-adherence”
Pill count—To assess whether partial non-adherence could be detected we classed patients in the active treatment group who continued taking at least some of their pills as fully adherent if they took at least 80% (as assessed by pill count at the six month visit) and as partially non-adherent if they took less than 80%. Only 9.4% (411) of the patients who did not permanently stop their drugs took less than 80% of their tablets.
Our analyses were based on changes in cholesterol (low density lipoprotein and total cholesterol) concentration from the time of randomisation to 12 months after randomisation. For each type of analysis we first plotted the distribution of changes for adherent and non-adherent patients. To determine the ability of the change in cholesterol concentration to identify whether a patient was from the adherent or non-adherent group of the trial we then plotted receiver operating characteristics (ROC) curves. With this empirical approach, the population levels of non-adherence (pre-test probability) would be those of the trial setting, and these are unlikely to reflect clinical populations (for example, the placebo comparison has a population level of non-adherence that is 50%, reflecting the 1:1 allocation of patients to active and placebo drugs). Therefore, for each type of non-adherence we fitted models that allowed for different population levels of non-adherence. We estimated post-test probability of non-adherence for levels of population non-adherence that differed from those in the trial and were more likely to reflect clinical populations: 25%, 40%, 50%, 60%, and 75%. See the appendix on bmj.com for full details.
Comparisons of complete non-adherence
Discontinuation—Of the people randomised to active treatment, 3944 had measurements of low density lipoprotein cholesterol at randomisation and at 12 months. Of these, 3.2% (n=128) reported that they had stopped all randomised treatment by 12 months (table 1⇓). The mean reduction in low density lipoprotein cholesterol for the discontinuers (non-adherent) was 0.07 mmol/L (SD 0.78 mmol/L) compared with a reduction of 1.15 mmol/L (SD 0.67 mmol/L) in the group still taking treatment. Figure 1 shows the observed distribution of these changes in low density lipoprotein cholesterol⇓, indicating some overlap in the distributions. The area under the curve was 0.86 (95% confidence interval 0.83 to 0.90), indicating reasonable discrimination (table 2⇓ and fig 2⇓). Results were similar for total cholesterol concentration.
Placebo—Of the 9014 patients randomised to active or placebo, 7881 had measurements of low density lipoprotein cholesterol at randomisation and at 12 months. Of these, about half were in the active treatment group and half in the placebo group (table 1⇑). The mean reduction in low density lipoprotein cholesterol was 0.02 mmol/L (SD 0.61 mmol/L) in the placebo group and 1.11 mmol/L (SD 0.70 mmol/L) in the active treatment group. Figure 1 shows the observed distribution of these changes in low density lipoprotein cholesterol, indicating some overlap in the distributions.⇑
The accuracy of detecting adherence depends on the cut off used. If any rise in concentration of low density lipoprotein cholesterol at 12 months was taken as an indicator of complete non-adherence, the sensitivity would be 50% and the specificity 94% (fig 2⇑). That is, of those who were completely non-adherent, 50% would have recorded a rise in concentration (and 50% no change or a fall in concentration) at 12 months; but of those who were adherent some 6% would have recorded an apparent increase in concentration at 12 months, despite them taking their drugs. If a cut off of a reduction of more than 0.5 mmol/L is used, the sensitivity improves to 82% but the specificity drops to 83% (82% of completely non-adherent and 17% of adherent patients failed to have a reduction of 0.5 mmol/L or more). The sensitivity and specificity is 95% and 59%, respectively, for reductions of more than 1.0 mmol/L and 98% and 29% for reductions of more than 1.5 mmol/L. Figure 2⇑ shows estimates for all possible cut-off values for change in low density lipoprotein cholesterol at 12 months. The area under the curve was 0.89 (0.88 to 0.89), similar to the permanent discontinuation comparison and again indicating reasonable discrimination (table 2).⇑ Similar results were found for total cholesterol.
Comparison of partial non-adherence
Pill count—Of people who continued to take active treatment, 3798 had measurements of low density lipoprotein cholesterol at randomisation and at 12 months. Of these, 5.6% (213) were taking less than 80% of their pills by 12 months (table 1⇑). The mean reduction in low density lipoprotein cholesterol for partial non-adherers was 0.77 mmol/L (SD 0.79 mmol/L) compared with 1.17 mmol/L (0.65 mmol/L) in the fully adherent group. Figure 1 shows the observed distribution of these changes in low density lipoprotein cholesterol, indicating substantial overlap in the distributions.
If any rise in concentration of low density lipoprotein cholesterol at 12 months was taken as an indicator of partial non-adherence, the sensitivity would be 16% and the specificity 96% (fig 2⇑). That is, of those who were partially non-adherent, 16% would have recorded a rise in concentration (and 84% no change or a fall in concentration) at 12 months; but of those who were fully adherent some 4% would have recorded an apparent increase in concentration at 12 months, despite them taking all their treatment. If a cut off of a reduction of more than 0.5 mmol/L is used, the sensitivity improves to 37% but the specificity drops to 86% (37% of partially non-adherent and 14% of fully adherent individuals failed to have a reduction of 0.5 mmol/L or more). For reductions of more than 1.0 mmol/L the sensitivity and specificity are 60% and 62%, respectively, and for reductions of more than 1.5 mmol/L these are 81% and 30%. Figure 2⇑ shows estimates for all possible cut-off values for change in low density lipoprotein cholesterol at 12 months. The area under the curve was 0.65 (0.61 to 0.70), indicating only modest discrimination (table 2).⇑ Similar results were found for total cholesterol.
Effects of different population frequency of non-adherence
The prevalence (pre-test probability) of non-adherence found in the LIPID trial was about 3% for the discontinuation comparison, 50% for the placebo comparison, and 6% for the pill count comparison, but these levels are unlikely to reflect clinical practice. Figure 3⇓ shows the probability of detecting non-adherence given different changes in concentration of low density lipoprotein cholesterol for a range of background prevalences of non-adherence that can be found in clinical populations. The population prevalences shown are broadly based on those reported in the literature, with high levels of non-adherence for people who have been taking treatment for over a year and have not experienced a cardiovascular event (75% non-adherence), low levels of non-adherence for those who have recently had a cardiovascular event (25% non-adherence), and intermediate levels of non-adherence for those who are in between these two groups of patient (40%, 50%, and 60% non-adherence).
For permanent discontinuation of treatment before one year of follow-up and randomisation to placebo, depending on the pre-test probability and the change observed in cholesterol concentration, the post-test probability might be sufficiently high or low to diagnose, or rule out, complete non-adherence (fig 3)⇑. Patients with a pre-test probability of 75% and a change in low density lipoprotein cholesterol of −1.5, −1.0, −0.50, 0, and 0.50 mmol/L have post-test probabilities of complete non-adherence of around 14%, 40%, 81%, 95%, and 97.5%, respectively. Patients with a pre-test probability of 25% and a change in low density lipoprotein cholesterol of −1.5, −1.0, −0.50, 0, and 0.50 mmol/L have post-test probabilities of complete non-adherence of around 2%, 7%, 32%, 67%, and 76%, respectively.
Figure 3 also shows that for those taking less than 80% of prescribed pills, without permanent discontinuation, depending on the pre-test probability and the change observed in cholesterol, the post-test probability might sometimes be sufficiently high to diagnose partial non-adherence. No matter what the pre-test probability and change in cholesterol concentration observed, however, the post-test probability is unlikely ever to be sufficiently low to rule out partial non-adherence. Patients with a pre-test probability of 75% and a change in low density lipoprotein cholesterol of −1.5, −1.0, −0.50, 0, and 0.50 mmol/L have post-test probabilities of partial non-adherence of around 64%, 71%, 82%, 89%, and 93%, respectively. Patients with a pre-test probability of 25% and a change in low density lipoprotein cholesterol of −1.5, −1.0, −0.50, 0, and 0.50 mmol/L have post-test probabilities of partial non-adherence of around 16%, 21%, 34%, 48%, and 58%, respectively.
This analysis of data from the LIPID trial shows substantial overlap between the changes in cholesterol concentration seen in patients receiving placebo compared with those receiving a statin, and even greater overlap between good and poor compliers. These findings suggest that monitoring cholesterol concentration is modestly accurate in detecting patients who are completely non-adherent or non-persistent with pravastatin treatment. Cholesterol changes seem less useful in detecting partial non-adherence. In both cases, low density lipoprotein cholesterol concentration seems slightly better than total cholesterol concentration at detecting non-compliance. For typical pre-test probabilities of non-adherence the post-test probabilities indicate continuing uncertainty after lipid testing. A patient with no change in low density lipoprotein cholesterol has a post-test probability of between 67% and 95% for complete non-adherence and between 48% and 89% for partial non-adherence. Hence monitoring is more useful for diagnosing non-adherence in populations where the background prevalence is high. Even then a relatively extreme result of monitoring (no change) is required to confirm partial non-adherence, given that the average change in low density lipoprotein cholesterol for this group was a reduction of around 0.8 mmol/l. A patient with a reduction of 1.0 mmol/L has a post-test probability of between 7% and 40% for complete non-adherence and between 21% and 71% for partial non-adherence. Hence monitoring might be useful for ruling out complete non-adherence in populations where the background prevalence is low but is rarely likely to be useful for ruling out partial non-adherence. Even with a pre-test probability of 25% and a decrease in low density lipoprotein cholesterol of 1.5 mmol/L, the post-test probability of partial non-adherence is still relatively high at 16%. When monitoring patients on statins, clinicians should be aware of this inaccuracy.
Strengths and limitations of study
There are some limitations of our study. Firstly, we limited our analysis to 12 months, whereas clinicians might assess adherence for longer periods of treatment. Change in cholesterol concentration over a year after the start of treatment is likely to perform less well for detection of non-adherence. For longer term monitoring, patients’ cholesterol concentrations show random upward drift3 as a natural consequence of the ageing process. Monitoring to detect non-adherence would need to distinguish any change in cholesterol concentration as a consequence of failure to take drugs from the natural upward trend that occurs at different rates between patients. Secondly, we labelled patients as partially non-adherent if they took less than 80% of their pills (this is a common definition used in trials10 14). In clinical practice, however, definitions might be less stringent—for example, partial non-adherence might be defined as taking less than 50% or an even lower percentage. Statins are relatively tolerant to partial compliance. Most of the effect of statins is seen at low doses, with more marginal gains as the dose is increased: doses that are about 25% of the maximum dose result in roughly 50% of the effect.15 Use of a less stringent definition of partial non-adherence could mean that change in cholesterol concentration performs somewhat better than the estimates we have given here. Even so, partial non-adherence is likely to be substantially more difficult to detect than complete non-adherence by use of clinic lipid measurements. And yet it is for partial non-adherence that a diagnostic test is most needed. Unlike completely non-adherent patients, who are likely to tell their clinicians they have stopped taking the drug or simply stop turning up to follow-up clinic appointments, partially non-adherent patients might be reluctant to volunteer information on their pill taking.
Conclusions and policy implications
Despite recommendations in clinical guidelines that cholesterol concentration be monitored to detect non-adherence, there is a surprising paucity of studies that have examined the relation between change in concentration and adherence. Change in concentration three months after the start of treatment has been found to not only correlate with adherence at three months but also adherence at later time points (even after adjustment for adherence at three months). This indicates that the relation is a bidirectional one: adherence affects change in concentration and change in concentration affects adherence.14 Although an apparent drop in concentration seems to enhance adherence, an apparent rise will tend to hinder it, and for some patients monitoring cholesterol concentration might actually have a negative impact on their future adherence to treatment. This is in keeping with other research showing decreased adherence to dietary advice in patients found to have cholesterol concentrations below a certain threshold.16 17
Interviews with patients have also been found to predict adherence to treatment,18 and making patients an active part of treatment decisions increases adherence.19 20 Rather than a formal interview, a practical approach is for clinicians to simply ask the patient if she or he is having problems adhering to treatment. This question needs to be framed in a non-judgmental way that keeps the patient at ease. One suggestion is to ask “I know it must be difficult to take all your medications regularly. How often do you miss taking them?”12 Other questions that could help to identify poor adherence include asking the patient whether they are having side effects from the drugs, why they believe they are taking them, and what they believe are the benefits of treatment.12
In conclusion, while monitoring cholesterol concentration seems to be a somewhat useful method of detecting complete non-adherence to cholesterol lowering treatment, it is less accurate for detecting partial non-adherence and at most should be used as an adjunct to careful questioning about problems with adherence. Clinical practice guidelines need to be updated to reflect the limitations of cholesterol measurement as a means of detecting non-adherence and make other recommendations about how this can be achieved. Further research to identify effective ways of detecting non-adherence with cholesterol concentration might include combining multiple types of cholesterol measurement21 or multiple measurement occasions, or both.
What is already known on this topic
Adherence with lipid lowering treatment varies between patients
Guidelines recommend that clinicians monitor patients’ cholesterol concentrations to assess adherence
The capacity of lipid monitoring to detect non-adherence is unknown
What this study adds
Monitoring low density lipoprotein (or total) cholesterol concentration has modest ability to detect complete non-adherence or non-persistence with pravastatin treatment and weak ability to detect partial non-adherence
Monitoring cholesterol concentration could serve as an adjunct to careful questioning about problems with adherence
Cite this as: BMJ 2011;342:d12
Contributors: KJLB drafted and revised the manuscript and contributed to the concept, design, and analysis and interpretation of data. AK did the statistical analysis and contributed to the acquisition of data and revision of the manuscript. AH assisted in the statistical analysis and contributed to the concept, design, interpretation of data, and revision of the manuscript. LI obtained funding and contributed to the concept, design, analysis and interpretation of data, and revision of the manuscript. PG contributed to the concept, design, acquisition of data, analysis and interpretation of data, and revision of the manuscript. AK is guarantor.
Funding: The authors have received funding from the Australian National Health and Medical Research Council (Program Grant No 402764). The LIPID trial was supported by a grant from the Bristol-Myers Squibb Pharmaceutical Research Institute and conducted under the auspices of the National Heart Foundation of Australia. The funders had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three year, no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.