Abstract
Objectives
Web-based studies nowadays raise a major interest as they can improve all steps involved in observational studies. Our objective was to compare the web-based version of the NutriNet-Santé self-administered socio-demographic and economic questionnaire with the traditional paper version.
Methods
Both versions of the questionnaire were sent to 170 volunteers and filled in by 147 of them (either paper first, n = 76, or web-based first, n = 71). Agreement between versions was assessed by intraclass correlations (ICC) and kappas.
Results
Agreement between both versions was high, with ICC and kappas ranging between 0.81–1.00 and 0.76–1.00, respectively, similarly across groups of administration order, age, gender and self-estimated web knowledge in general. The web-based version was the one preferred by 93.7% of the subjects and enabled to avoid 553 missing values (2.00% of the total entries), 24 inconsistent data (0.09%), 8 aberrant data (0.03%), 472 data entry errors (0.85%) and to save 2,800 € (US $4,072) when sent to 170 subjects.
Conclusion
The web-based socio-demographic and economic questionnaire provided information of similar-to-superior quality compared to the traditional paper version, with substantial logistic and cost advantages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
http://media.etude-nutrinet-sante.fr/questionnaires/questionnaires_nutrinet_papier.zip: (NutriNet-Santé questionnaires [in French]) (2009)
Bertrais S, Preziosi P, Mennen L, Galan P, Hercberg S, Oppert JM (2004) Sociodemographic and geographic correlates of meeting current recommendations for physical activity in middle-aged French adults: the Supplementation en Vitamines et Mineraux Antioxydants (SUVIMAX) Study. Am J Public Health 94:1560–1566
Brigham J, Lessov-Schlaggar CN, Javitz HS, Krasnow RE, McElroy M, Swan GE (2009) Test–retest reliability of web-based retrospective self-report of tobacco exposure and risk. J Med Internet Res 11:35
Clark AM, DesMeules M, Luo W, Duncan AS, Wielgosz A (2009) Socioeconomic status and cardiovascular disease: risks and implications for care. Nat Rev Cardiol 6:712–722
Drewnowski A (2009) Obesity, diets, and social inequalities. Nutr Rev 67(Suppl 1):36–39
Ekman A, Litton JE (2007) New times, new needs; e-epidemiology. Eur J Epidemiol 22:285–292
European Interactive Advertising Association (2005) http://www.eiaa.net/news/eiaa-articles-details.asp?lang=1&id=79
Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43:543–549
Etude NutriNet-Santé Website https://www.etude-nutrinet-sante.fr (2009)
Graham AL, Papandonatos GD (2008) Reliability of internet-versus telephone-administered questionnaires in a diverse sample of smokers. J Med Internet Res 10:8
Greene J, Speizer H, Wiitala W (2008) Telephone and web: mixed-mode challenge. Health Serv Res 43:230–248
Hammersley R (1994) A digest of memory phenomena for addiction research. Addiction 89:283–293
Hercberg S, Preziosi P, Briancon S, Galan P, Triol I, Malvy D, Roussel AM, Favier A (1998) A primary prevention trial using nutritional doses of antioxidant vitamins and minerals in cardiovascular diseases and cancers in a general population: the SU.VI.MAX study—design, methods, and participant characteristics. SUpplementation en VItamines et Mineraux AntioXydants. Control Clin Trials 19:336–351
Internet au quotidien: un Français sur quatre; Insee Première No. 1076 (2006) http://www.insee.fr/fr/themes/document.asp?ref_id=ip1076®_id=0
Joinson A (1999) Social desirability, anonymity, and Internet-based questionnaires. Behav Res Methods Instrum Comput 31:433–438
L’audience de l’Internet en France Médiamétrie (2010) http://www.mediametrie.fr/internet/
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174
McCabe SE, Boyd CJ, Couper MP, Crawford S, D’Arcy H (2002) Mode effects for collecting alcohol and other drug use data: web and U.S. mail. J Stud Alcohol 63:755–761
McLaren L (2007) Socioeconomic status and obesity. Epidemiol Rev 29:29–48
Pealer LN, Weiler RM, Pigg RM Jr, Miller D, Dorman SM (2001) The feasibility of a web-based surveillance system to collect health risk behavior data from college students. Health Educ Behav 28:547–559
Profil sociodemographique des internautes Ipsos Media (2007) http://www.ipsos.fr/canalipsos/poll/8351.asp
Rankin KM, Rauscher GH, McCarthy B, Erdal S, Lada P, Il’yasova D, Davis F (2008) Comparing the reliability of responses to telephone-administered versus self-administered Web-based surveys in a case–control study of adult malignant brain cancer. Cancer Epidemiol Biomarkers Prev 17:2639–2646
Schatzkin A, Subar AF, Moore S, Park Y, Potischman N, Thompson FE, Leitzmann M, Hollenbeck A, Morrissey KG, Kipnis V (2009) Observational epidemiologic studies of nutrition and cancer: the next generation (with better observation). Cancer Epidemiol Biomarkers Prev 18:1026–1032
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428
Stanton CA, Papandonatos G, Lloyd-Richardson EE, Niaura R (2007) Consistency of self-reported smoking over a 6-year interval from adolescence to young adulthood. Addiction 102:1831–1839
Touvier M, Mejean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, Hercberg S (2010) Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol 25(5):287–296
Wang YC, Lee CM, Lew-Ting CY, Hsiao CK, Chen DR, Chen WJ (2005) Survey of substance use among high school students in Taipei: web-based questionnaire versus paper-and-pencil questionnaire. J Adolesc Health 37:289–295
Ward E, Jemal A, Cokkinides V, Singh GK, Cardinez C, Ghafoor A, Thun M (2004) Cancer disparities by race/ethnicity and socioeconomic status. CA Cancer J Clin 54:78–93
Acknowledgments
We thank all scientists, dieticians, technicians and assistants who helped to carry out the NutriNet-Santé study and the dedicated and conscientious volunteers who participated in this test study. We especially thank Laudine Brico (data manager) for performing the statistical analysis and Gwenael Monot (computer scientist), who coordinates the computing aspects of the NutriNet-Santé study. We thank Voluntis (a healthcare software company) for developing the NutriNet-Santé web-based interface according to our guidelines. This work was supported by the French Ministry of Health (DGS), the Institut de Veille Sanitaire (InVS), the Institut National de Prévention et d’Education pour la Santé (INPES), the Fondation pour la Recherche Médicale (FRM), the Institut National de la Santé et de la Recherche Médicale (Inserm), the Institut de Recherche en Santé Publique (IRESP), the Institut National de la Rechereche Agronomique (Inra), the Conservatoire National des Arts et Métiers (CNAM) and the Paris 13 University.
Conflict of interest
None of the authors had any conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vergnaud, AC., Touvier, M., Méjean, C. et al. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study. Int J Public Health 56, 407–417 (2011). https://doi.org/10.1007/s00038-011-0257-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00038-011-0257-5