Critical appraisal of the meta-analysis by Wandel et al. - clinical aspects (part 1)
Submitted 15 October 2010.
Wandel et al. (1) were very concerned about the quality (e.g.
concealment of allocation) of the trials they included in their analysis,
but very little about the validity of the results obtained in these trials
and the clinical baseline characteristics of the patients: In one trial
pain was assessed after only 4 weeks (2), too early for a SYSADOA to
develop its maximal effect; in 4 trials (3-6) pain at baseline was less
than 4.0 cm, in one trial only 2.6 cm (3), not enough to detect a decrease
of 0.9 cm considering that pain asymptotically approaches on average a
value around 3.4 cm after one year or later (7-9).
Most disputable, however, is how they made use of GAIT (7,10,11).
This trial included a celecoxib arm in order to assess its sensitivity to
detect a symptomatic effect and to exclude false negative results. In GAIT
celecoxib was superior to placebo only in the primary outcome and the
OMERACT-OARSI response criterion after 6 months (10), but not anymore
after 2 years (7). Its effect size at 6 months was only 0.13 (10), well
below the 0.37 pre-specified by Wandel et al. as a minimal clinically
important difference and also below the effect size of 0.44 found for
coxibs (12). Although this trial proved not to be sufficiently sensitive
to detect the minimal clinically important difference pre-specified by
Wandel et al. they included it in their analysis. - With respect to the
analysis of the radiological data GAIT also does not meet the pre-
specified criterion of 100 patients per arm - only between 59 and 80
patients per arm were assessed on a modified intention to treat basis
(11), not enough to detect such an effect according to the power
calculations by Michel et al. (3) and Kahan et al. (8) and the results
found later in their trials. Some, therefore, may ask themselves whether
it was correct to include GAIT in the analysis. But without GAIT, no
network meta-analysis.
In the 10 trials selected by Wandel et al. the placebo effects
expressed as relative change from baseline varied between +17 % (increase
of pain, in (6) at 24 months) and -64 % (decrease of pain, in (7) at 24
months), pain levels at baseline expressed on a VAS (0-10.0 cm) varied
between 2.6 (3) and 6.2 cm (13), and the gender distributions differed by
as many as 34 percent points. Effects on pain and function observed in
osteoarthritis trials are known to be influenced by the level of pain at
baseline, the radiologic stage of disease, the gender distribution, and in
particular also by the placebo effect. The placebo effect becomes a major
determinant of the observed effect when the later one is expressed in form
of an effect size. According to Zhang et al. (14) the placebo effect is
strongly influenced by the disease activity at baseline. Cooper et al.
(15) write in a publication (also cited by Wandel et al.) that "the
assumptions of a mixed treatment comparison analysis are that (i) study-
specific treatment effects are drawn from a common population
(exchangeable) and that (ii) heterogeneity is constant between the
different comparisons." Considering the very different placebo effects and
pain levels at baseline some may doubt that these assumptions are
fulfilled. Unfortunately, Wandel et al. only tested for heterogeneity with
respect to methodological aspects of the trials, but not with respect to
the clinical baseline characteristics or the placebo effects.
Wandel et al. calculated the effect sizes regarding the radiological
effect of chondroitin in the trials by Kahan et al. (8) and Michel et al.
(3) using a median pooled SD of 1.2 mm although the standard deviations in
these two trials had been available and are 0.6 and 0.5 mm, respectively.
Using a standard pooled SD for the calculation of effect sizes for the
radiological effect is not appropriate because different measurement
techniques with different measurement errors were used in the 6 trials
examining this effect. In this way the authors lessened the radiological
effect of chondroitin by as much as a factor 2.0 or 2.4, respectively.
We have outlined earlier that the calculation of effect sizes is not
suitable for the estimation of the radiological effect (16). Instead a
failure criterion should be defined and the radiological effect should be
expressed in terms of relative risk reduction. For example, in the trial
by Kahan et al. (8) a narrowing of the joint space by 0.25 mm - based on
the minimal clinically important difference of 0.37 SD units pre-specified
by Wandel et al. - can be regarded as a minimal clinically important
disease progression (SD = 0.6 mm, 0.37 x 0.6 mm = 0.22 mm). In this trial
the percentage of patients with radiographic progression (decrease in
minimum joint space width of > 0.25 mm) was significantly reduced in
the chondroitin group with respect to the placebo group (28% versus 41%; p
<0.0005; relative risk reduction 33% [95% CI 16-46%]). The number of
patients needed to treat was 8 (95% CI 5-17). Wandel et al. had always
judged this trial as being of high methodological quality, and it is the
only high quality trial with chondroitin included in the meta-analysis
that allows for an estimation of the symptomatic effect. Effect sizes for
pain reduction in this trial at 3, 6, and 9 months were 0.19, 0.22, and
019, respectively, i.e. higher than the 0.14 of paracetamol, the current
pharmaceutical first line therapy, and close to the effect size of 0.29
found for NSAIDs (12). There is probably no other pharmaceutical treatment
that offers such an effect that persists over 6 months with virtually no
side effects, apart maybe from glucosamine sulphate or hyaluronic acid.
Furthermore, at 6 months 29% of the patients with chondroitin experienced
a clinically relevant decrease of pain of > 80% from baseline, whereas
only 18% of the patients under placebo had such a benefit (p <0.001).
Andreas G. Helg, PhD (Dr. sc. nat. ETH)
IBSA Institut Biochimique SA, CH-6915 Pambio-Noranco, Switzerland
1. Wandel S, Juni P, Tendal B, Nuesch E, Villiger PM, Welton NJ, et
al. Effects of glucosamine, chondroitin, or placebo in patients with
osteoarthritis of hip or knee: network meta-analysis. BMJ 2010;341:c4675.
2. Noack W, Fischer M, Forster KK, Rovati LC, Setnikar I. Glucosamine
sulfate in osteoarthritis of the knee. Osteoarthritis Cart 1994;2:51-9.
3. Michel BA, Stucki G, Frey D, De Vathaire F, Vignon E, Bruehlmann
P, et al. Chondroitins 4 and 6 sulfate in osteoarthritis of the knee: a
randomized, controlled trial. Arthritis Rheum 2005;52(3):779-86.
4. Reginster JY, Deroisy R, Rovati LC, Lee RL, Lejeune E, Bruyere O,
et al. Long-term effects of glucosamine sulphate on osteoarthritis
progression: a randomised, placebo-controlled clinical trial. Lancet
2001;357(9252):251-6.
5. Pavelka K, Gatterova J, Olejarova M, Machacek S, Giacovelli G,
Rovati LC. Glucosamine sulphate use and delay of progression of knee
osteoarthritis: a 3-year, randomized, placebo-controlled, double-blind
study. Arch Intern Med 2002;162:2113-23.
6. Rozendaal RM, Koes BW, van Osch GJVM, Uitterlinden EJ, Garling EH,
Willemsen SP, et al. Effect of glucosamine sulphate on hip osteoarthritis.
Ann Intern Med 2008;148:268-77.
7. Sawitzke AD, Shi H, Finco MF, Dunlop DD, Harris CL, Singer NG, et
al. Clinical efficacy and safety of glucosamine, chondroitin sulphate,
their combination, celecoxib or placebo taken to treat osteoarthritis of
the knee: 2-year results from GAIT. Ann Rheum Dis 2010;69(8):1459-64.
8. Kahan A, Uebelhart D, De Vathaire F, Delmas PD, Reginster JY. Long
-term effects of chondroitins 4 and 6 sulfate on osteoarthritis: the study
on osteoarthritis progression prevention, a two-year, randomized, double-
blind, placebo-controlled trial. Arthritis Rheum 2009;60:524-33.
9. Scott DL, Berry H, Capell H, Coppock J, Daymond T, Doyle DV, et
al. The long-term effects of non-steroidal anti-inflammatory drugs in
osteoarthritis of the knee: a randomized placebo-controlled trial.
Rheumatology (Oxford) 2000;39(10):1095-101.
10. Clegg DO, Reda DJ, Harris CL, Klein MA, O'Dell JR, Hooper MM, et
al. Glucosamine, chondroitin sulfate, and the two in combination for
painful knee osteoarthritis. N Engl J Med 2006;354(8):796-808.
11. Sawitzke AD, Shi H, Finco MF, Dunlop DD, Bingham CO 3rd, Harris
CL, et al. The effect of glucosamine and/or chondroitin sulfate on the
progression of knee osteoarthritis: a report from the
glucosamine/chondroitin arthritis intervention trial. Arthritis Rheum
2008;58(10):3183-91.
12. Zhang W, Nuki G, Moskowitz RW, Abramson S, Altman RD, Arden NK,
et al. OARSI recommendations for the management of hip and knee
osteoarthritis Part III: changes in evidence following systematic
cumulative update of research published through January 2009.
Osteoarthritis Cart 2010;18:476-99.
13. Mazieres B, Hucher M, Zaim M, Garnero P. Effect of chondroitin
sulphate in symptomatic knee osteoarthritis: a multicentre, randomised,
double-blind, placebo-controlled study. Ann Rheum Dis 2007;66(5):639-45.
14. Zhang W, Robertson J, Jones AC, Dieppe PA, Doherty M. The placebo
effect and its determinants in osteoarthritis: meta-analysis of randomised
controlled trials. Ann Rheum Dis 2008;67:1716-23.
15. Cooper NJ, Sutton AJ, Lu G, Khunti K. Mixed comparison of stroke
prevention treatments in individuals with nonrheumatic atrial
fibrillation. Arch Intern Med 2006;166:1269-75.
Competing interests:
The author is employed by IBSA Institut Biochimique SA, a manufacturer and distributor of a prescription drug with chondroitin sulphate.
Rapid Response:
Critical appraisal of the meta-analysis by Wandel et al. - clinical aspects (part 1)
Submitted 15 October 2010.
Wandel et al. (1) were very concerned about the quality (e.g.
concealment of allocation) of the trials they included in their analysis,
but very little about the validity of the results obtained in these trials
and the clinical baseline characteristics of the patients: In one trial
pain was assessed after only 4 weeks (2), too early for a SYSADOA to
develop its maximal effect; in 4 trials (3-6) pain at baseline was less
than 4.0 cm, in one trial only 2.6 cm (3), not enough to detect a decrease
of 0.9 cm considering that pain asymptotically approaches on average a
value around 3.4 cm after one year or later (7-9).
Most disputable, however, is how they made use of GAIT (7,10,11).
This trial included a celecoxib arm in order to assess its sensitivity to
detect a symptomatic effect and to exclude false negative results. In GAIT
celecoxib was superior to placebo only in the primary outcome and the
OMERACT-OARSI response criterion after 6 months (10), but not anymore
after 2 years (7). Its effect size at 6 months was only 0.13 (10), well
below the 0.37 pre-specified by Wandel et al. as a minimal clinically
important difference and also below the effect size of 0.44 found for
coxibs (12). Although this trial proved not to be sufficiently sensitive
to detect the minimal clinically important difference pre-specified by
Wandel et al. they included it in their analysis. - With respect to the
analysis of the radiological data GAIT also does not meet the pre-
specified criterion of 100 patients per arm - only between 59 and 80
patients per arm were assessed on a modified intention to treat basis
(11), not enough to detect such an effect according to the power
calculations by Michel et al. (3) and Kahan et al. (8) and the results
found later in their trials. Some, therefore, may ask themselves whether
it was correct to include GAIT in the analysis. But without GAIT, no
network meta-analysis.
In the 10 trials selected by Wandel et al. the placebo effects
expressed as relative change from baseline varied between +17 % (increase
of pain, in (6) at 24 months) and -64 % (decrease of pain, in (7) at 24
months), pain levels at baseline expressed on a VAS (0-10.0 cm) varied
between 2.6 (3) and 6.2 cm (13), and the gender distributions differed by
as many as 34 percent points. Effects on pain and function observed in
osteoarthritis trials are known to be influenced by the level of pain at
baseline, the radiologic stage of disease, the gender distribution, and in
particular also by the placebo effect. The placebo effect becomes a major
determinant of the observed effect when the later one is expressed in form
of an effect size. According to Zhang et al. (14) the placebo effect is
strongly influenced by the disease activity at baseline. Cooper et al.
(15) write in a publication (also cited by Wandel et al.) that "the
assumptions of a mixed treatment comparison analysis are that (i) study-
specific treatment effects are drawn from a common population
(exchangeable) and that (ii) heterogeneity is constant between the
different comparisons." Considering the very different placebo effects and
pain levels at baseline some may doubt that these assumptions are
fulfilled. Unfortunately, Wandel et al. only tested for heterogeneity with
respect to methodological aspects of the trials, but not with respect to
the clinical baseline characteristics or the placebo effects.
Wandel et al. calculated the effect sizes regarding the radiological
effect of chondroitin in the trials by Kahan et al. (8) and Michel et al.
(3) using a median pooled SD of 1.2 mm although the standard deviations in
these two trials had been available and are 0.6 and 0.5 mm, respectively.
Using a standard pooled SD for the calculation of effect sizes for the
radiological effect is not appropriate because different measurement
techniques with different measurement errors were used in the 6 trials
examining this effect. In this way the authors lessened the radiological
effect of chondroitin by as much as a factor 2.0 or 2.4, respectively.
We have outlined earlier that the calculation of effect sizes is not
suitable for the estimation of the radiological effect (16). Instead a
failure criterion should be defined and the radiological effect should be
expressed in terms of relative risk reduction. For example, in the trial
by Kahan et al. (8) a narrowing of the joint space by 0.25 mm - based on
the minimal clinically important difference of 0.37 SD units pre-specified
by Wandel et al. - can be regarded as a minimal clinically important
disease progression (SD = 0.6 mm, 0.37 x 0.6 mm = 0.22 mm). In this trial
the percentage of patients with radiographic progression (decrease in
minimum joint space width of > 0.25 mm) was significantly reduced in
the chondroitin group with respect to the placebo group (28% versus 41%; p
<0.0005; relative risk reduction 33% [95% CI 16-46%]). The number of
patients needed to treat was 8 (95% CI 5-17). Wandel et al. had always
judged this trial as being of high methodological quality, and it is the
only high quality trial with chondroitin included in the meta-analysis
that allows for an estimation of the symptomatic effect. Effect sizes for
pain reduction in this trial at 3, 6, and 9 months were 0.19, 0.22, and
019, respectively, i.e. higher than the 0.14 of paracetamol, the current
pharmaceutical first line therapy, and close to the effect size of 0.29
found for NSAIDs (12). There is probably no other pharmaceutical treatment
that offers such an effect that persists over 6 months with virtually no
side effects, apart maybe from glucosamine sulphate or hyaluronic acid.
Furthermore, at 6 months 29% of the patients with chondroitin experienced
a clinically relevant decrease of pain of > 80% from baseline, whereas
only 18% of the patients under placebo had such a benefit (p <0.001).
Andreas G. Helg, PhD (Dr. sc. nat. ETH)
IBSA Institut Biochimique SA, CH-6915 Pambio-Noranco, Switzerland
1. Wandel S, Juni P, Tendal B, Nuesch E, Villiger PM, Welton NJ, et
al. Effects of glucosamine, chondroitin, or placebo in patients with
osteoarthritis of hip or knee: network meta-analysis. BMJ 2010;341:c4675.
2. Noack W, Fischer M, Forster KK, Rovati LC, Setnikar I. Glucosamine
sulfate in osteoarthritis of the knee. Osteoarthritis Cart 1994;2:51-9.
3. Michel BA, Stucki G, Frey D, De Vathaire F, Vignon E, Bruehlmann
P, et al. Chondroitins 4 and 6 sulfate in osteoarthritis of the knee: a
randomized, controlled trial. Arthritis Rheum 2005;52(3):779-86.
4. Reginster JY, Deroisy R, Rovati LC, Lee RL, Lejeune E, Bruyere O,
et al. Long-term effects of glucosamine sulphate on osteoarthritis
progression: a randomised, placebo-controlled clinical trial. Lancet
2001;357(9252):251-6.
5. Pavelka K, Gatterova J, Olejarova M, Machacek S, Giacovelli G,
Rovati LC. Glucosamine sulphate use and delay of progression of knee
osteoarthritis: a 3-year, randomized, placebo-controlled, double-blind
study. Arch Intern Med 2002;162:2113-23.
6. Rozendaal RM, Koes BW, van Osch GJVM, Uitterlinden EJ, Garling EH,
Willemsen SP, et al. Effect of glucosamine sulphate on hip osteoarthritis.
Ann Intern Med 2008;148:268-77.
7. Sawitzke AD, Shi H, Finco MF, Dunlop DD, Harris CL, Singer NG, et
al. Clinical efficacy and safety of glucosamine, chondroitin sulphate,
their combination, celecoxib or placebo taken to treat osteoarthritis of
the knee: 2-year results from GAIT. Ann Rheum Dis 2010;69(8):1459-64.
8. Kahan A, Uebelhart D, De Vathaire F, Delmas PD, Reginster JY. Long
-term effects of chondroitins 4 and 6 sulfate on osteoarthritis: the study
on osteoarthritis progression prevention, a two-year, randomized, double-
blind, placebo-controlled trial. Arthritis Rheum 2009;60:524-33.
9. Scott DL, Berry H, Capell H, Coppock J, Daymond T, Doyle DV, et
al. The long-term effects of non-steroidal anti-inflammatory drugs in
osteoarthritis of the knee: a randomized placebo-controlled trial.
Rheumatology (Oxford) 2000;39(10):1095-101.
10. Clegg DO, Reda DJ, Harris CL, Klein MA, O'Dell JR, Hooper MM, et
al. Glucosamine, chondroitin sulfate, and the two in combination for
painful knee osteoarthritis. N Engl J Med 2006;354(8):796-808.
11. Sawitzke AD, Shi H, Finco MF, Dunlop DD, Bingham CO 3rd, Harris
CL, et al. The effect of glucosamine and/or chondroitin sulfate on the
progression of knee osteoarthritis: a report from the
glucosamine/chondroitin arthritis intervention trial. Arthritis Rheum
2008;58(10):3183-91.
12. Zhang W, Nuki G, Moskowitz RW, Abramson S, Altman RD, Arden NK,
et al. OARSI recommendations for the management of hip and knee
osteoarthritis Part III: changes in evidence following systematic
cumulative update of research published through January 2009.
Osteoarthritis Cart 2010;18:476-99.
13. Mazieres B, Hucher M, Zaim M, Garnero P. Effect of chondroitin
sulphate in symptomatic knee osteoarthritis: a multicentre, randomised,
double-blind, placebo-controlled study. Ann Rheum Dis 2007;66(5):639-45.
14. Zhang W, Robertson J, Jones AC, Dieppe PA, Doherty M. The placebo
effect and its determinants in osteoarthritis: meta-analysis of randomised
controlled trials. Ann Rheum Dis 2008;67:1716-23.
15. Cooper NJ, Sutton AJ, Lu G, Khunti K. Mixed comparison of stroke
prevention treatments in individuals with nonrheumatic atrial
fibrillation. Arch Intern Med 2006;166:1269-75.
16. Helg AG, De Vathaire F. How solid are the results of the meta-
analysis by Reichenbach et al. and its conclusions? Ann Intern Med 2009.
www.annals.org/content/146/8/580.abstract/reply#annintmed_el_44993.
Competing interests: The author is employed by IBSA Institut Biochimique SA, a manufacturer and distributor of a prescription drug with chondroitin sulphate.