BMJ 1995;310:446 (18 February)

Statistics notes

Calculating correlation coefficients with repeated observations: Part 1--correlation within subjects

J Martin Bland, reader in medical statistics,^a Douglas G Altman, head ^b

^a Department of Public Health Sciences, St George's Hospital Medical School, London SW17 0RE, ^b Medical Statistics Laboratory, Imperial Cancer Research Fund, PO Box 123, London WC2A 3PX

Correspondence to: Dr Bland.

In an earlier Statistics Note¹ we commented on the analysis of paired data where there is more than one observation per subject, as shown in table I. We pointed out that it could be highly misleading to analyse such data by combining repeated observations from several subjects and then calculating the correlation coefficient as if the data were a simple sample. This note is a response to several letters about the appropriate analysis for such data.

TABLE I--Repeated measurements of intramural pH and PaCO2 for eight subjects2 -------------------------------------------------- Subject pH PaCO2 Subject pH PaCO2 -------------------------------------------------- 1 6.68 3.97 5 7.30 4.32 1 6.53 4.12 5 7.37 3.23 1 6.43 4.09 5 7.27 4.46 1 6.33 3.97 5 7.28 4.72 2 6.85 5.27 5 7.32 4.75 2 7.06 5.37 5 7.32 4.99 2 7.13 5.41 6 7.38 4.78 2 7.17 5.44 6 7.30 4.73 3 7.40 5.67 6 7.29 5.12 3 7.42 3.64 6 7.33 4.93 3 7.41 4.32 6 7.31 5.03 3 7.37 4.73 6 7.33 4.93 3 7.34 4.96 7 6.86 6.85 3 7.35 5.04 7 6.94 6.44 3 7.28 5.22 7 6.92 6.52 3 7.30 4.82 8 7.19 5.28 3 7.34 5.07 8 7.29 4.56 4 7.36 5.67 8 7.21 4.34 4 7.33 5.10 8 7.25 4.32 4 7.29 5.53 8 7.20 4.41 4 7.30 4.75 8 7.19 3.69 4 7.35 5.51 8 6.77 6.09 5 7.35 4.28 8 6.82 5.58 5 7.30 4.44

The choice of analysis for the data in table I depends on the question we want to answer. If we want to know whether subjects with high values of intramural pH also tend to have high values of PaCO2 we are interested in whether the average pH for a subject is related to the subject's average PaCO2. We can use the correlation between the subject means, which we shall describe in a subsequent note. If we want to know whether an increase in pH within the individual was associated with an increase in PaCO2 we want to remove the differences between subjects and look only at changes within.

To look at variation within the subject we can use multiple regression. We make one of our variables, pH or PaCO2, the outcome variable and the other variable and the subject the predictor variables. Subject is treated as a categorical factor using dummy variables^{3 4} and so has seven degrees of freedom. We use the analysis of variance table^{3 4} for the regression (table II), which shows how the variability in pH can be partitioned into components due to different sources. This method is also known as analysis of covariance and is equivalent to fitting parallel lines through each subject's data (see figure). The residual sum of squares in table II represents the variation about these lines. We remove the variation due to subjects (and any other nuisance variables which might be present) and express the variation in pH due to PaCO2 as a proportion of what's left: (Sum of squares for PaCO2)/(Sum of squares for PaCO2 + residual sum of squares) The magnitude of the correlation coefficient within subjects is the square root of this proportion. For table II this is: (square root) 0.1153/0.1153+0.3337 = 0.51 The sign of the correlation coefficient is given by the sign of the regression coefficient for PaCO2. Here the regression slope is -0.108, so the correlation coefficient within subjects is -0.51. The P value is found either from the F test in the associated analysis of variance table, or from the t test for the regression slope. It doesn't matter which variable we regress on which; we get the same correlation coefficient and P value either way.

TABLE II--Analysis of variance for the data in table I ------------------------------------------------------------------- Source of Degrees of Sum of Mean Variance variation freedom squares square ratio (F) Probability ------------------------------------------------------------------- Subjects 7 2.9661 0.4237 48.3 <0.0001 PaCO2 1 0.1153 0.1153 13.1 0.0008 Residual 38 0.3337 0.0088 ------------------------------------------------------------------- Total 46 3.3139 0.0720

View larger version (12K): [in this window] [in a new window]   pH against PaCO2 for eight subjects, with parallel lines fitted for each subject

If we incorrectly calculate the correlation coefficient ignoring the fact that we have 47 observations on only 8 subjects, we get -0.07, P=0.7. Hence the correct analysis within subjects reveals a relation which the incorrect analysis misses.

1. Bland JM, Altman DG. Correlation, regression, and repeated data. BMJ 1994;308:896. [Free Full Text]
2. Boyd O, Mackay CJ, Lamb G, Bland JM, Grounds RM, Bennett ED. Comparison of clinical information gained from routine blood-gas analysis and from gastric tonometry for intramural pH. Lancet 1993;341:142-6. [Medline]
3. Altman DG. Practical statistics for medical research. London: Chapman and Hall, 1991.
4. Armitage P, Berry G. Statistical methods in medical research. 3rd ed. Oxford: Blackwell, 1994.

This article has been cited by other articles:

• Jenkins, C., Monaghan, M., Shirali, G., Guraraja, R., Marwick, T. H. (2008). An intensive interactive course for 3D echocardiography: is 'crop till you drop' an effective learning strategy?. Eur J Echocardiogr 9: 373-380 [Abstract] [Full text]  
• Karmisholt, J., Laurberg, P. (2008). Serum TSH and serum thyroid peroxidase antibody fluctuate in parallel and high urinary iodine excretion predicts subsequent thyroid failure in a 1-year study of patients with untreated subclinical hypothyroidism. Eur J Endocrinol 158: 209-215 [Abstract] [Full text]  
• Notomi, Y., Popovic, Z. B., Yamada, H., Wallick, D. W., Martin, M. G., Oryszak, S. J., Shiota, T., Greenberg, N. L., Thomas, J. D. (2008). Ventricular untwisting: a temporal link between left ventricular relaxation and suction. Am. J. Physiol. Heart Circ. Physiol. 294: H505-H513 [Abstract] [Full text]  
• Feigin, A., Kaplitt, M. G., Tang, C., Lin, T., Mattis, P., Dhawan, V., During, M. J., Eidelberg, D. (2007). Modulation of metabolic brain networks after subthalamic gene therapy for Parkinson's disease. Proc. Natl. Acad. Sci. USA 104: 19559-19564 [Abstract] [Full text]  
• Russo, R. J, Silva, P. D, Yeager, M. (2007). Coronary artery overexpansion increases neointimal hyperplasia after stent placement in a porcine model. Heart 93: 1609-1615 [Abstract] [Full text]  
• Pilichiewicz, A. N., Papadopoulos, P., Brennan, I. M., Little, T. J., Meyer, J. H., Wishart, J. M., Horowitz, M., Feinle-Bisset, C. (2007). Load-dependent effects of duodenal lipid on antropyloroduodenal motility, plasma CCK and PYY, and energy intake in healthy men. Am. J. Physiol. Regul. Integr. Comp. Physiol. 293: R2170-R2178 [Abstract] [Full text]  
• Feigin, A., Tang, C., Ma, Y., Mattis, P., Zgaljardic, D., Guttman, M., Paulsen, J. S., Dhawan, V., Eidelberg, D. (2007). Thalamic metabolism and symptom onset in preclinical Huntington's disease. Brain 130: 2858-2867 [Abstract] [Full text]  
• Pilichiewicz, A. N., Chaikomin, R., Brennan, I. M., Wishart, J. M., Rayner, C. K., Jones, K. L., Smout, A. J. P. M., Horowitz, M., Feinle-Bisset, C. (2007). Load-dependent effects of duodenal glucose on glycemia, gastrointestinal hormones, antropyloroduodenal motility, and energy intake in healthy men. Am. J. Physiol. Endocrinol. Metab. 293: E743-E753 [Abstract] [Full text]  
• Huang, C., Tang, C., Feigin, A., Lesser, M., Ma, Y., Pourfar, M., Dhawan, V., Eidelberg, D. (2007). Changes in network activity with the progression of Parkinson's disease. Brain 130: 1834-1846 [Abstract] [Full text]  
• Shushakov, V., Stubbe, C., Peuckert, A., Endeward, V., Maassen, N. (2007). Human, Environmental & Exercise: The relationships between plasma potassium, muscle excitability and fatigue during voluntary exercise in humans. Exp Physiol 92: 705-715 [Abstract] [Full text]  
• Jenkins, C., Chan, J., Bricknell, K., Strudwick, M., Marwick, T. H. (2007). Reproducibility of Right Ventricular Volumes and Ejection Fraction Using Real-time Three-Dimensional Echocardiography: Comparison With Cardiac MRI. Chest 131: 1844-1851 [Abstract] [Full text]  
• Dufour, S. P., Doutreleau, S., Lonsdorfer-Wolf, E., Lampert, E., Hirth, C., Piquard, F., Lonsdorfer, J., Geny, B., Mettauer, B., Richard, R. (2007). Deciphering the metabolic and mechanical contributions to the exercise-induced circulatory response: insights from eccentric cycling. Am. J. Physiol. Regul. Integr. Comp. Physiol. 292: R1641-R1648 [Abstract] [Full text]  
• Esbjornsson, M., Bulow, J., Norman, B., Simonsen, L., Nowak, J., Rooyackers, O., Kaijser, L., Jansson, E. (2006). Adipose tissue extracts plasma ammonia after sprint exercise in women and men. J. Appl. Physiol. 101: 1576-1580 [Abstract] [Full text]  
• Wachters-Hagedoorn, R. E., Priebe, M. G., Heimweg, J. A. J., Heiner, A. M., Englyst, K. N., Holst, Jens. J., Stellaard, F., Vonk, R. J. (2006). The Rate of Intestinal Glucose Absorption Is Correlated with Plasma Glucose-Dependent Insulinotropic Polypeptide Concentrations in Healthy Men. J. Nutr. 136: 1511-1516 [Abstract] [Full text]  
• da Graca, R. L., Hassinger, D. C., Flynn, P. A., Sison, C. P., Nesin, M., Auld, P. A.M. (2006). Longitudinal changes of brain-type natriuretic Peptide in preterm neonates.. Pediatrics 117: 2183-2189 [Abstract] [Full text]  
• Nygren, A., Thoren, A., Houltz, E., Ricksten, S.-E. (2006). Autoregulation of human jejunal mucosal perfusion during cardiopulmonary bypass.. Anesth. Analg. 102: 1617-1622 [Abstract] [Full text]  
• Cysique, L. A.J., Maruff, P., Brew, B. J. (2006). Variable benefit in neuropsychological function in HIV-infected HAART-treated patients. Neurology 66: 1447-1450 [Abstract] [Full text]  
• Jensen, E. C., Bennet, L., Hunter, C. J., Power, G. C., Gunn, A. J. (2006). Post-hypoxic hypoperfusion is associated with suppression of cerebral metabolism and increased tissue oxygenation in near-term fetal sheep. J. Physiol. 572: 131-139 [Abstract] [Full text]  
• Bryant, D., Havey, T. C., Roberts, R., Guyatt, G. (2006). How Many Patients? How Many Limbs? Analysis of Patients or Limbs in the Orthopaedic Literature: A Systematic Review. JBJS 88: 41-45 [Abstract] [Full text]  
• Bennet, L., Westgate, J. A., Liu, Y.-C., Wassink, G., Gunn, A. J. (2005). Fetal acidosis and hypotension during repeated umbilical cord occlusions are associated with enhanced chemoreflex responses in near-term fetal sheep. J. Appl. Physiol. 99: 1477-1482 [Abstract] [Full text]  
• Little, T. J., Feltrin, K. L., Horowitz, M., Smout, A. J. P. M., Rades, T., Meyer, J. H., Pilichiewicz, A. N., Wishart, J., Feinle-Bisset, C. (2005). Dose-related effects of lauric acid on antropyloroduodenal motility, gastrointestinal hormone release, appetite, and energy intake in healthy men. Am. J. Physiol. Regul. Integr. Comp. Physiol. 289: R1090-R1098 [Abstract] [Full text]  
• Yotti, R., Bermejo, J., Desco, M. M., Antoranz, J. C., Rojo-Alvarez, J. L., Cortina, C., Allue, C., Rodriguez-Abella, H., Moreno, M., Garcia-Fernandez, M. A. (2005). Doppler-Derived Ejection Intraventricular Pressure Gradients Provide a Reliable Assessment of Left Ventricular Systolic Chamber Function. Circulation 112: 1771-1779 [Abstract] [Full text]  
• Roelfsema, V., Gunn, A. J, Fraser, M., Quaedackers, J. S, Bennet, L. (2005). Cortisol and ACTH responses to severe asphyxia in preterm fetal sheep. Exp Physiol 90: 545-555 [Abstract] [Full text]  
• Notomi, Y., Lysyansky, P., Setser, R. M., Shiota, T., Popovic, Z. B., Martin-Miklovic, M. G., Weaver, J. A., Oryszak, S. J., Greenberg, N. L., White, R. D., Thomas, J. D. (2005). Measurement of Ventricular Torsion by Two-Dimensional Ultrasound Speckle Tracking Imaging. J Am Coll Cardiol 45: 2034-2041 [Abstract] [Full text]  
• Pittas, A. G., Hariharan, R., Stark, P. C., Hajduk, C. L., Greenberg, A. S., Roberts, S. B. (2005). Interstitial Glucose Level Is a Significant Predictor of Energy Intake in Free-Living Women with Healthy Body Weight. J. Nutr. 135: 1070-1074 [Abstract] [Full text]  
• Notomi, Y., Setser, R. M., Shiota, T., Martin-Miklovic, M. G., Weaver, J. A., Popovic, Z. B., Yamada, H., Greenberg, N. L., White, R. D., Thomas, J. D. (2005). Assessment of Left Ventricular Torsional Deformation by Doppler Tissue Imaging: Validation Study With Tagged Magnetic Resonance Imaging. Circulation 111: 1141-1147 [Abstract] [Full text]  
• Ho, W. M., Yang, N. C., Wong, K. C., Hwang, K. L. (2005). A Real-Time Method for Estimating the Concentrations of Isoflurane in Mixed Venous Blood by a Derived Fick's Equation. Anesth. Analg. 100: 38-45 [Abstract] [Full text]  
• Nirmalan, M., Niranjan, M., Willard, T., Edwards, J. D., Little, R. A., Dark, P. M. (2004). Estimation of errors in determining intrathoracic blood volume using thermal dilution in pigs with acute lung injury and haemorrhage. Br J Anaesth 93: 546-551 [Abstract] [Full text]  
• Sturm, K., Parker, B., Wishart, J., Feinle-Bisset, C., Jones, K. L, Chapman, I., Horowitz, M. (2004). Energy intake and appetite are related to antral area in healthy young and older subjects. Am. J. Clin. Nutr. 80: 656-667 [Abstract] [Full text]  
• Feltrin, K. L., Little, T. J., Meyer, J. H., Horowitz, M., Smout, A. J. P. M., Wishart, J., Pilichiewicz, A. N., Rades, T., Chapman, I. M., Feinle-Bisset, C. (2004). Effects of intraduodenal fatty acids on appetite, antropyloroduodenal motility, and plasma CCK and GLP-1 in humans vary with their chain length. Am. J. Physiol. Regul. Integr. Comp. Physiol. 287: R524-R533 [Abstract] [Full text]  
• Fadel, P. J., Keller, D. M., Watanabe, H., Raven, P. B., Thomas, G. D. (2004). Noninvasive assessment of sympathetic vasoconstriction in human and rodent skeletal muscle using near-infrared spectroscopy and Doppler ultrasound. J. Appl. Physiol. 96: 1323-1330 [Abstract] [Full text]  
• Victorino, J. A., Borges, J. B., Okamoto, V. N., Matos, G. F. J., Tucci, M. R., Caramez, M. P. R., Tanaka, H., Sipmann, F. S., Santos, D. C. B., Barbas, C. S. V., Carvalho, C. R. R., Amato, M. B. P. (2004). Imbalances in Regional Lung Ventilation: A Validation Study on Electrical Impedance Tomography. Am. J. Respir. Crit. Care Med. 169: 791-800 [Abstract] [Full text]  
• Sanchez-Moreno, C., Dorfman, S. E., Lichtenstein, A. H., Martin, A. (2004). Dietary Fat Type Affects Vitamins C and E and Biomarkers of Oxidative Status in Peripheral and Brain Tissues of Golden Syrian Hamsters. J. Nutr. 134: 655-660 [Abstract] [Full text]  
• Quaedackers, J. S., Roelfsema, V., Hunter, C. J., Heineman, E., Gunn, A. J., Bennet, L. (2004). Polyuria and impaired renal blood flow after asphyxia in preterm fetal sheep. Am. J. Physiol. Regul. Integr. Comp. Physiol. 286: R576-R583 [Abstract] [Full text]  
• Sanchez-Moreno, C., Cano, M P., de Ancos, B., Plaza, L., Olmedilla, B., Granado, F., Martin, A. (2003). Effect of orange juice intake on vitamin C concentrations and biomarkers of antioxidant status in humans. Am. J. Clin. Nutr. 78: 454-460 [Abstract] [Full text]  
• Bruner, L H, Carr, G J, Harbell, J W, Curren, R D (2002). An investigation of new toxicity test method performance in validation studies: 2. comparison of three measures of toxicity test performance. Hum Exp Toxicol 21: 313-323 [Abstract]  
• Chambers, D C, Ayres, J G (2001). Effect of nebulised L- and D-arginine on exhaled nitric oxide in steroid naive asthma. Thorax 56: 602-606 [Abstract] [Full text]  
• Ahmed, M. L., Ong, K. K. L., Watts, A. P., Morrell, D. J., Preece, M. A., Dunger, D. B. (2001). Elevated Leptin Levels Are Associated with Excess Gains in Fat Mass in Girls, But Not Boys, with Type 1 Diabetes: Longitudinal Study during Adolescence. J. Clin. Endocrinol. Metab. 86: 1188-1193 [Abstract] [Full text]  
• Schultz, C. J., Neil, H. A. W., Dalton, R. N., Bahu, T. K., Dunger, D. B. (2001). Blood Pressure Does Not Rise Before the Onset of Microalbuminuria in Children Followed From Diagnosis of Type 1 Diabetes. Diabetes Care 24: 555-560 [Abstract] [Full text]  
• Subhedar, N V, Shaw, N J (2000). Changes in pulmonary arterial pressure in preterm infants with chronic lung disease. Arch. Dis. Child. Fetal Neonatal Ed. 82: 243F-247 [Abstract] [Full text]  
• Lovell, A. T., Marshall, A. C., Elwell, C. E., Smith, M., Goldstone, J. C. (2000). Changes in Cerebral Blood Volume with Changes in Position in Awake and Anesthetized Subjects. Anesth. Analg. 90: 372-372 [Abstract] [Full text]  
• CUTTITTA, G., CIBELLA, F., VISCONTI, A., SCICHILONE, N., BELLIA, V., BONSIGNORE, G. (2000). Spontaneous Gastroesophageal Reflux and Airway Patency during the Night in Adult Asthmatics. Am. J. Respir. Crit. Care Med. 161: 177-181 [Abstract] [Full text]  
• Booth, S. L, O'Brien-Morse, M. E, Dallal, G. E, Davidson, K. W, Gundberg, C. M (1999). Response of vitamin K status to different intakes and sources of phylloquinone-rich foods: comparison of younger and older adults. Am. J. Clin. Nutr. 70: 368-377 [Abstract] [Full text]  
• De Marinis, L., Mancini, A., Valle, D., Bianchi, A., Milardi, D., Proto, A., Lanzone, A., Tacchino, R. (1999). Plasma Leptin Levels after Biliopancreatic Diversion: Dissociation with Body Mass Index. J. Clin. Endocrinol. Metab. 84: 2386-2389 [Abstract] [Full text]  
• Lunn, P. G., Erinoso, H. O., Northrop-Clewes, C. A., Boyce, S. A. (1999). Giardia intestinalis Is Unlikely To Be a Major Cause of the Poor Growth of Rural Gambian Infants. J. Nutr. 129: 872-877 [Abstract] [Full text]  
• Ahmed, M. L., Ong, K. K. L., Morrell, D. J., Cox, L., Drayer, N., Perry, L., Preece, M. A., Dunger, D. B. (1999). Longitudinal Study of Leptin Concentrations during Puberty: Sex Differences and Relationship to Changes in Body Composition. J. Clin. Endocrinol. Metab. 84: 899-905 [Abstract] [Full text]  
• Ludwig, D. S., Majzoub, J. A., Al-Zahrani, A., Dallal, G. E., Blanco, I., Roberts, S. B. (1999). High Glycemic Index Foods, Overeating, and Obesity. Pediatrics 103: 26e-26 [Abstract] [Full text]  
• Subhedar, N V, Shaw, N J (1997). Changes in oxygenation and pulmonary haemodynamics in preterm infants treated with inhaled nitric oxide. Arch. Dis. Child. Fetal Neonatal Ed. 77: 191F-197 [Abstract] [Full text]  
• Altman, D. G, Bland, J M. (1997). Statistics Notes: Units of analysis. BMJ 314: 1874-1874 [Full text]  
• Bland, J M., Altman, D. G (1995). Statistics notes: Calculating correlation coefficients with repeated observations: Part 2--correlation between subjects. BMJ 310: 633-633 [Full text]