- Nina Buscemi, research associate ()1,
- Ben Vandermeer, statistician1,
- Nicola Hooton, project coordinator1,
- Rena Pandya, project manager1,
- Lisa Tjosvold, research librarian1,
- Lisa Hartling, administrative director1,
- Sunita Vohra, director2,
- Terry P Klassen, director1,
- Glen Baker, professor and chair3
- 1 University of Alberta/Capital Health Evidence-based Practice Centre, Department of Pediatrics, University of Alberta, Edmonton, AB, Canada T6G 2J3
- 2 Complementary and Alternative Research and Education Program, Department of Pediatrics, University of Alberta
- 3 Department of Psychiatry, University of Alberta
- Correspondence to: N Buscemi
- Accepted 18 November 2005
Objective To conduct a systematic review of the efficacy and safety of exogenous melatonin in managing secondary sleep disorders and sleep disorders accompanying sleep restriction, such as jet lag and shiftwork disorder.
Data sources 13 electronic databases and reference lists of relevant reviews and included studies; Associated Professional Sleep Society abstracts (1999 to 2003).
Study selection The efficacy review included randomised controlled trials; the safety review included randomised and non-randomised controlled trials.
Quality assessment Randomised controlled trials were assessed by using the Jadad Scale and criteria by Schulz et al, and non-randomised controlled trials by the Downs and Black checklist.
Data extraction and synthesis One reviewer extracted data and another reviewer verified the data extracted. The inverse variance method was used to weight studies and the random effects model was used to analyse data.
Main results Six randomised controlled trials with 97 participants showed no evidence that melatonin had an effect on sleep onset latency in people with secondary sleep disorders (weighted mean difference −13.2 (95% confidence interval −27.3 to 0.9) min). Nine randomised controlled trials with 427 participants showed no evidence that melatonin had an effect on sleep onset latency in people who had sleep disorders accompanying sleep restriction (−1.0 (−2.3 to 0.3) min). 17 randomised controlled trials with 651 participants showed no evidence of adverse effects of melatonin with short term use (three months or less).
Conclusions There is no evidence that melatonin is effective in treating secondary sleep disorders or sleep disorders accompanying sleep restriction, such as jet lag and shiftwork disorder. There is evidence that melatonin is safe with short term use.
Sleep disorders affect approximately 20% of the American population.1 A sleep disorder exists whenever a lower quality of sleep leads to impaired functioning or excessive sleepiness.2 Sleep disorders place a burden on society due to their negative impact on quality of life, safety, productivity, and healthcare utilisation.
One category of sleep disorders is secondary sleep disorders, sleep problems that are associated with medical, neurological, or substance misuse disorders. Another category of sleep disorders arises from sleep restriction: inadequate sleep results from imposed or self imposed lifestyle and work schedules, such as air travel and shift work.1
Complementary and alternative medicine has been used increasingly to manage sleep disorders. One of the most popular treatments of this type is melatonin, a hormone that is secreted by the pineal gland and is linked to the circadian rhythm.3
We conducted a systematic review of the efficacy and safety of exogenous melatonin in managing secondary sleep disorders and sleep disorders accompanying sleep restriction, such as jet lag and shiftwork disorder. Our findings can help to guide clinicians and patients in treatment decisions regarding the use of exogenous melatonin in the management of these conditions.
A health sciences librarian conducted a comprehensive search to identify relevant English-language studies. We searched 13 electronic databases (table 1; see bmj.com for search terms). The reference lists of relevant reviews, as well as a random sample of included studies, were reviewed to identify other potentially relevant studies. We hand searched abstracts of meetings of the Associated Professional Sleep Society from 1999 to 2003. Finally, we searched Medline and Embase again in early 2004 to identify recently published studies.
The full text of all articles deemed potentially relevant was retrieved and reviewed independently by two reviewers. To assess the efficacy of exogenous melatonin, we included randomised controlled trials that involved human participants who had a secondary sleep disorder or a sleep disorder accompanying sleep restriction; compared melatonin to placebo; and reported on one or more of: sleep onset latency (amount of time between lying down to sleep and onset of sleep), sleep efficiency (amount of time spent asleep as a percentage of total time spent in bed), sleep quality (perceived quality of sleep), wakefulness after sleep onset (amount of time spent awake in bed after first attainment of sleep), total sleep time (total time spent asleep while in bed), or percentage of time in rapid eye movement (REM) sleep.
To assess the safety of exogenous melatonin, we included randomised and non-randomised trials meeting the first two criteria above and reporting on adverse events. A study population was considered to have a secondary sleep disorder if the participants, as a group, were defined by a specific chronic medical or psychiatric disorder and this disorder was likely to be the cause of the sleep disorder. A study population was considered to have been exposed to sleep restriction if participants had been exposed to transmeridian air travel, shiftwork, or other forms of sleep schedule alteration. Disagreements regarding inclusion of studies were resolved through discussion.
Two reviewers assessed study quality independently. For the efficacy review, randomised controlled trials were assessed for methodological quality with the validated Jadad scale.4 In addition, concealment of treatment allocation was assessed using the criteria of Schulz et al.5 Allocation concealment was considered to be adequate if group allocation was accomplished by using such methods as central randomisation, numbered or coded containers, drugs prepared by a pharmacy, or serially numbered, opaque, sealed envelopes. For the safety review, which relied on evidence from randomised and non-randomised trials, the Downs and Black checklist was used.6 Disagreements regarding quality assessment were resolved through discussion.
Data were extracted by using a standardised data extraction form that captured details of study design, population, intervention, and outcomes. A trained reviewer extracted data and a second reviewer verified the extracted data. Disagreements were resolved through discussion.
We listed our outcomes in order of importance, with sleep onset latency as most important (primary outcome), followed by sleep efficiency, sleep quality, wakefulness after sleep onset, total sleep time, and percentage of time in REM sleep. Continuous outcomes were combined, using a weighted mean difference, with the exception of sleep quality, for which studies were combined by using a standardised mean difference. Dichotomous outcomes were combined by using a risk difference. The inverse variance method was used to weight the studies.7 All meta-analyses used a random effects model. A point estimate with corresponding 95% confidence interval was computed for each outcome, using the generic inverse variance function in RevMan 4.2.5 (Update Software, 2004).
In most cases, we were able to calculate the efficacy estimate for each study exactly, but occasionally estimates had to be made by extracting from graphs or using medians. Standard errors of the differences were calculated from available data (individual patient data or exact P values) whenever possible. For studies with a crossover design, we used the methods of Elbourne et al to compute standard errors of differences,8 and a correlation of 0.5 was imputed when it could not be calculated from available data.
All pooled estimates were assessed for heterogeneity, using the I2 statistic.9 For our primary outcome, we planned to explore heterogeneity in subgroup and sensitivity analyses. We also conducted a post hoc sensitivity analysis. Deeks' χ2 statistic10 was used to test for significant heterogeneity reduction in partitioned subgroups (age, comorbidity, type of sleep disorder, dosage, treatment duration, outcome measurement method, study design, study quality, and allocation concealment).
We tested for publication bias visually using the funnel plot and quantitatively using the rank correlation test,11 the graphical test,12 and the trim and fill method.13 Publication bias graphs and calculations were produced with STATA 7.0 (Stata Corporation, 2001).
Figure 1 shows the flow of studies through the selection process.
Secondary sleep disorders
Table 2 describes the nine trials (279 participants) included in the efficacy analysis for secondary sleep disorders.14–25 The median quality score, based on the Jadad scale, was 4 out of 5 (interquartile range 2-4). Concealment of allocation was unclear in all studies except one,23 which had adequate allocation concealment. Only five studies described a funding source; for all of these studies, funding was received from public sponsors.18–23
Sleep onset latency
Table 3 outlines the means and standard deviations for sleep onset latency for placebo and melatonin groups for the six trials providing data on this outcome.15 18 19 21–23 The studies produced a combined estimate that favoured melatonin but was not significant (weighted mean difference −13.2 (95% confidence interval −27.3 to 0.9) min) (fig 2). Heterogeneity among the studies was substantial (I2 = 79.2%) due primarily to one study23 that had a very small standard deviation and an estimate that favoured placebo, whereas the other studies had point estimates that favoured melatonin.
The results for planned subgroup and sensitivity analyses are shown in table 4. In the only two categories for which the confidence intervals across subgroups did not overlap, a study by Shamir et al seemed to be highly influential.23 Subgroups that omitted this study (actigraphy and questionnaire methods of measuring sleep outcomes and unclear allocation concealment) showed a significant result in favour of melatonin with minimal heterogeneity, while the point estimate for this study showed a significant effect in favour of placebo.
We conducted a post hoc sensitivity analysis excluding the study by Shamir et al from the primary analysis. When the study was included in the analysis, the point estimate was −13.2 (−27.3 to 0.9) min; when it was excluded, the point estimate was −17.4 (−26.4 to −8.4) min. Although the point estimate did not change substantially, the confidence interval narrowed, rendering the result significant.
Not enough studies examined sleep onset latency for publication bias to be tested on the basis of this outcome.
Other efficacy outcomes
Six trials reporting data for sleep efficiency showed a significant effect that favoured melatonin (weighted mean difference 1.9% (0.5 to 3.3); I2=0%)18 20–24; however, the effect seems not to be clinically important. The results for other efficacy outcomes are shown in table 4.
Seven studies were included in the safety analysis15–20 25; one was non-randomised and six were randomised (table 2). The studies included 164 participants. The quality of these studies was good (median quality index 21 (out of 29); range 20-22). The most commonly reported adverse events were headaches, dizziness, nausea, and drowsiness. The occurrence of these outcomes was similar for melatonin and placebo (table 4).
Table 5 describes the nine trials included in the efficacy analysis for sleep restriction.26 29–32 35–38 The trials encompassed 427 participants. The median quality score was 4 out of 5 (interquartile range 3-4). Concealment of allocation was unclear in all studies except three,29 32 38 which had adequate allocation concealment. None of the studies described a funding source.
Sleep onset latency
Table 6 outlines the mean and standard deviations for sleep onset latency for placebo and melatonin groups for the nine trials that provided data on this outcome.26 29 32 35–37 The studies produced a combined estimate that favoured melatonin but was not significant (weighted mean difference −1.0 (−2.3 to 0.3) min; I2 = 4.0%) (fig 3).
The results for planned subgroup and sensitivity analyses are in table 7. The subgroups did not differ significantly in any of the categories (all confidence intervals were overlapping, and in all but two cases (< 1 mg dose and parallel study design), results were non-significant).
Given that the study by Folkard et al29 was allotted a high proportion of weight in the primary analysis but had a small sample size, we conducted a post hoc sensitivity analysis excluding this study. When the Folkard study was excluded from the analysis, there was almost no change in the point estimate and the confidence interval widened slightly: (−1.03 (−3.59 to 1.53) min when excluded v −0.97 (−2.26 to 0.33) min).
The funnel plot for sleep onset latency showed no obvious signs of asymmetry. There were also no indications of publication bias with Begg's test (P = 0.35; n = 9); Egger's test (P = 0.48); and Duval's trim and fill method (no new studies added).
Other efficacy outcomes
For sleep efficiency, the combined estimate from five trials26 30–32 37 showed no significant difference between melatonin and placebo (weighted mean difference 0.5% (−0.6 to 1.6); I2 = 20.9%). The results for other efficacy outcomes are in table 7.
Of the 10 studies included in the safety analysis,27 28 30–36 38 all studies but one28 were randomised controlled trials (table 5). The studies included 487 participants. The methodological quality of these studies was good (median quality index 21 (out of 29); range 20-22). The most commonly reported adverse events were headache, dizziness, nausea, and drowsiness. The occurrence of these outcomes did not differ significantly for melatonin versus placebo (table 7).
This review of the effects of exogenous melatonin on people with secondary sleep disorders or sleep disorders accompanying sleep restriction showed that melatonin does not have a significant effect on sleep onset latency in either disorder or on sleep efficiency in people with sleep disorders accompanying sleep restriction. Although the increase in sleep efficiency in people with secondary sleep disorders was statistically significant with melatonin, the effect was small—1.9%—an increase of less than 10 minutes in the amount of time spent asleep for eight hours spent in bed. On the basis of advice from clinical sleep experts, we considered this effect to be clinically unimportant, due to its small magnitude.
Factors affecting heterogeneity
The effect of melatonin on sleep onset latency in studies of people with secondary sleep disorders was associated with substantial heterogeneity, which seemed to be highly influenced by the study by Shamir et al.23 This study was unique in that polysomnography was used to assess sleep outcomes and the method of concealing treatment allocation was reported and adequate. Although the estimation of sleep variables differs according to the assessment tool used to measure them,39 the heterogeneity in results across studies is unlikely to be due to variation in assessment tool, as any differences between methods would have been cancelled out when absolute differences in the effect of treatment and placebo were obtained.
Regarding the effect of allocation concealment on effect estimates, failure to conceal treatment allocation adequately is associated with larger effect estimates.5 40 Allocation concealment may have been inadequate in the studies for which the adequacy of allocation concealment was unclear, which would tend to result in overestimation of treatment effect. Also, the heterogeneity across studies may have been due to publication or reporting bias, such that small studies with negative results were not published and therefore under-represented in the analysis; as this category included only nine studies, we could not verify this bias.
Other factors may have contributed to heterogeneity in results across studies of secondary sleep disorders. Formulations of melatonin vary in quality. In studies that reported details of the intervention, the rate of release of melatonin varied from slow to fast, a range of doses was used, and the duration of administration varied from days to weeks. Indeed, our results show that dosage and duration of melatonin administration explain a considerable amount of heterogeneity across studies.
Two other systematic reviews examining the use of melatonin for jet lag concluded that melatonin is effective in alleviating the symptoms of jet lag.41 42 These reviews examined the effect of melatonin on both the daytime fatigue and the sleep disturbance aspects of jet lag. Our review shows that melatonin does not affect either sleep onset latency or sleep efficiency in people with jet lag or people with shiftwork disorder. Our results do not provide evidence that melatonin is effective in alleviating sleep disturbance in jet lag, but we did not determine the effect of melatonin on measures of daytime fatigue.
The observations of this review are based mostly on studies with relatively short durations, so the efficacy and safety of melatonin reported here may reflect only its short term effects. Secondly, several studies did not report adequately on details of the intervention, such as content, quality, and formulation of the melatonin product under study, nor on methods of allocation concealment or source of funding, which casts doubt on the methodological quality of these studies, despite a good median Jadad score or Downs and Black quality index. Thirdly, non-English language reports were excluded from the review; however, we did not find strong evidence of publication bias, so it is unlikely that the inclusion of these reports would have altered our findings substantially.
What is already known on this topic
Sleep disorders are a widespread problem and place a burden on society through their negative impact on quality of life, safety, productivity, and healthcare utilisation
Complementary and alternative therapies, such as melatonin, have been used increasingly to manage sleep disorders
What this study adds
There is no evidence that melatonin is effective in treating secondary sleep disorders or sleep disorders accompanying sleep restriction, such as jet lag or shiftwork disorder
There is evidence that melatonin is safe with short term use, but additional studies are needed to determine its long term safety
Search terms used are on bmj.com
We thank the National Centre for Complementary and Alternative Medicine, National Institutes of Health for sponsoring this research, through the Agency for Healthcare Research and Quality. We are grateful to members of our technical expert panel for providing input on the direction and scope of the review. We are especially grateful to Manisha Witmans for her input on the manuscript.
Contributors NB planned, oversaw, and participated in all steps of the systematic review process and in writing and editing the manuscript. BV performed all statistical analyses and participated in writing and editing the manuscript. NH participated in most steps of the systematic review process and in writing and editing the manuscript. RP participated in all steps of the systematic review process and reviewed the manuscript. LT conducted the literature search, provided technological expertise for the inclusion process, and participated in editing the manuscript. LH participated in writing the proposal, provided methodological expertise, and participated in writing and editing the manuscript. SV participated in writing the proposal, provided methodological and content expertise, and participated in editing the manuscript. TK participated in writing the proposal, provided methodological expertise, and provided feedback on the manuscript. GB participated in writing the proposal, provided content expertise, and participated in writing and editing the manuscript. Michelle Tubman, Mia Lang, Maria Ospina, Victor Juorio, and Ellen Crumley were involved in study selection, quality assessment, and data extraction or entry. TK is guarantor.
Funding This study was conducted under contract to the Agency for Healthcare Research and Quality (contract No 290-02-0023), Rockville, MD, and support from the National Center for Complementary and Alternative Medicine, National Institutes of Health, Bethesda, MD. SV is supported by Agency for Healthcare Research and Quality (AHRQ), USA; Canadian Institutes of Health Research; Change Foundation; Department of Pediatrics; National Health Products Directorate, Health Canada; Ontario Mental Health Foundation; Stollery Children's Hospital and Foundation; The Hospital for Sick Children Foundation; and the University of Alberta. GB is supported by AHRQ; Canadian Institutes of Health Research; Canada Research Chairs Program; Stanley Foundation; University of Alberta Hospital Foundation; Bebensee Schizophrenia Research Fund; Davey Endowment; and Zyprexa Research Foundation. The authors of this article are responsible for its contents, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality, the National Center for Complementary and Alternative Medicine or the US Department of Health and Human Services.
Competing interests None declared.
Ethical approval Not required.