- Arthur A Stone, professor and vice-chair ()a,
- Saul Shiffman, chief science officerb,
- Joseph E Schwartz, associate professora,
- Joan E Broderick, assistant professora,
- Michael R Hufford, director of scientific affairsb
- a Department of Psychiatry and Behavioral Science, Stony Brook University, Stony Brook, NY, 11794-8790, USA
- b invivodata, Pittsburgh, PA, 15203, USA
- Correspondence to: A A Stone
- Accepted 24 January 2002
Doctors often ask patients to recall recent health experiences, such as pain, fatigue, and quality of life.1 Research has shown, however, that recall is unreliable and rife with inaccuracies and biases.2 Recognition of recall's shortcomings has led to the use of diaries, which are intended to capture experiences close to the time of occurrence, thus limiting recall bias and producing more accurate data.3
The rationale for using diaries would be undermined if patients failed to complete diaries according to protocol. In this study we used a newly developed paper diary that could objectively record when patients made diary entries in order to compare patients' reported and actual compliance with diary keeping. For comparison, we also used an electronic diary designed to enhance compliance in order to assess what compliance rates might be achieved.
Methods and results
We recruited 80 adults with chronic pain (pain for ≥3 hours a day and rated ≥4 on a 10 point scale) and assigned 40 to keeping a paper diary and 40 to an electronic diary. On satisfying the eligibility criteria, each patient was assigned to the next training session for which he or she was available, regardless of which diary it was for. We conducted one training session for each diary each week, with each training session for the paper diary matched by time and day of the week with an electronic diary training session. Participants were paid $150 and gave their informed consent; patients given the paper diary were not told that compliance would be recorded electronically.
The paper diary comprised diary cards bound into a DayRunner Organizer binder. The cards contained 20 questions drawn from several common pain instruments and included fields to record time and date of completion. The diary binders were unobtrusively fitted with photosensors that detected light and recorded when the binder was opened and closed; these were extensively tested and validated. The electronic diary was a Palm computer with software for data collection in clinical trials and presented identical pain questions via a touch screen and recorded time and date of entries. This system (invivodata) incorporated several features to maximise compliance, including auditory prompts, and has demonstrated good compliance.4
Patients were instructed to complete daily entries at 10 am, 4 pm, and 8 pm within 15 minutes of the target times. With the electronic diary, entries could not be initiated outside the designated 30 minute windows. We considered paper diary entries to be compliant if they were made within the 30 minute windows. A more liberal secondary outcome allowed a 90 minute window around the target times. Reported compliance was based on the time and date that patients recorded on their paper diary cards. Actual compliance was based on the electronic record (from the record of diary binder openings for paper diaries). Paper diary entries were deemed compliant if the binder was opened or closed at any point during the target time window. We also assessed “hoarding” with the paper diary, defined as days when the diary binder was not opened but for which diary cards were completed.
After three days' familiarisation, the participants began 21 days of diary keeping with weekly feedback. Participants completed an average of 20.5 days, and the table shows compliance rates. With the paper diary, reported compliance was 90%, but actual compliance was 11% (20% with the wider 90 minute window). With the electronic diary, actual compliance was 94%. Hoarding was common with the paper diary: 32% of days contained no diary openings, yet reported compliance (30 minute window) for these days was 92%. Most of the 40 patients (75%) had at least one day of hoarding.
This study shows that concerns about compliance with paper diaries are justified.5 Although patients reported high compliance, actual compliance was low and hoarding was common. The excellent compliance achieved with the electronic diary indicates that low compliance was not due to this particular sample or to an overly burdensome protocol. Overall, these results call into question the validity of paper diary records.
We thank Jill Jackowsky, Dan Arnold, J Corey Harmon, Julie Grassell, and Doyle Carney for help with completing this study. We also thank Jackie Dunbar-Jacob, Jeffery Katz, Eva Krusinska, Dennis Turk, and Michael Weintraub for their comments on an early draft of the manuscript.
Contributors: Development of the original concept was by all of the authors. JES analysed the data. The manuscript was drafted by AAS and extensively modified by all the other authors. AAS is guarantor for the study.
Funding This study was supported by a grant from the National Cancer Institute (CA-85819; Arthur A. Stone, principal investigator) and by support in kind from invivodata.
Competing interests AAS is vice-chair of the Scientific Advisory Board of invivodata, SS is a founder of invivodata, and MRH is director of scientific affairs at invivodata, which provides electronic diary support for clinical trials.