Review of prevalence data in, and evaluation of methods for cross cultural adaptation of, UK surveys on tobacco and alcohol in ethnic minority groupsBMJ 2004; 328 doi: https://doi.org/10.1136/bmj.328.7431.76 (Published 09 January 2004) Cite this as: BMJ 2004;328:76
- Raj Bhopal, professor of public health ()1,
- Amanda Vettini, research associate1,
- Sonja Hunt, honorary research fellow1,
- Sushmita Wiebe, research fellow1,
- Lisa Hanna, PhD student1,
- Amanda Amos, senior lecturer1
- 1Public Health Sciences Section, Division of Community Health Sciences, Medical School, University of Edinburgh, Edinburgh EH8 9AG
- Correspondence to: R Bhopal
- Accepted 16 October 2003
Objective To assess the adequacy of cross cultural adaptations of survey questions on self reported tobacco and alcohol consumption in the United Kingdom.
Design Assessment of consistency of data between studies identified through literature review. Studies evaluated with 12 guidelines developed from the research literature on achieving cross cultural comparability.
Results The literature review identified 18 key studies, five of them on national samples. Survey instruments were obtained for 15 of these. The comparison of prevalence data in national surveys showed some important discrepancies, greater for tobacco than for alcohol. For example, prevalence of cigarette smoking in Bangladeshi women was 6% in a national survey in 1994 and 1% in a national survey in 1999; in Chinese men it was 31% in a survey in 1993-4 and 17% in one in 1999; in African-Caribbean men it was 29% in a 1992 survey and 42% in one in 1993-4. The most guidelines met by any study was three, although one study partly met a fourth. Two studies met no guidelines. Only four studies consulted with ethnic minority communities in developing the questionnaire, none checked each language version with all others, and two stated the questionnaire had not been validated.
Conclusions Surveys have not followed best practice in relation to measurement of risk factors in cross cultural settings. There is inconsistency in the prevalence data on smoking provided by different major national UK studies. Users of such data should be aware of their limitations. Research is needed to help achieve linguistic equivalence of survey questions in cross cultural research.
Cancers and cardiovascular disease are dominant causes of death in Britain's ethnic minority groups.1 The prevention of such disorders requires accurate information about health related behaviour such as the amount and pattern of consumption of tobacco and alcohol. Such information is usually acquired by self completed questionnaires or schedules administered by interviewers, sometimes validated by biochemical and other tests.
Patterns of tobacco and alcohol consumption in ethnic minority groups in the United Kingdom show substantial differences from those of white populations of European origin.2–6 For example, in the health survey for England in 1999 Bangladeshi men self reported a smoking prevalence of 44% compared with 27% for the general population of men, with Chinese men reporting 17%.2 For alcohol consumption even bigger differences seem to exist. Nazroo found that only 4% of Pakistani men reported drinking more than once a week compared with 69% of white men.5 Are these differences accurate or a result of study artefact?
Most survey instruments on tobacco and alcohol consumption by ethnic minorities were developed for English speaking people and translated into other languages. To compare data across language groups the items on the questionnaire, the instructions given, and the responses obtained should be conceptually and functionally equivalent in each language.7–10 If reliability and validity for each language varies, comparisons across groups may be invalid. Translation is a vital step in the process.
Highly educated translators from professional backgrounds may use the “high” form of a language in translation, when the “low” or colloquial form is more appropriate. The best epidemiological studies have used back translation, whereby the original instrument is translated, a second translator translates this back into English, and the two versions are compared. This may, however, be insufficient. People who speak different languages may interpret concepts, words, or phrases in different ways, and cultural differences may render some questions offensive, irrelevant, or inappropriate. The social, cultural, and religious taboos and norms of particular ethnic minority groups may affect the self reporting of tobacco and alcohol consumption. For example, Sikhs are prohibited by their religion from smoking, and Muslims are prohibited from drinking alcohol. It is socially unacceptable for women from several ethnic minority groups to smoke or to drink. These factors could lead to misreporting among these groups.
There are two main ways to develop a cross cultural instrument: firstly, design different language versions in parallel to produce linguistically equivalent items and, secondly, adapt a single language version for use in other languages. We have examined the adaptation of single language versions, the approach most commonly used. Adaptation of questionnaires requires an extensive process, which has been described by several authors (box 1).10–17 In practice it may not be possible to achieve this ideal—for example, comparing every language version with every other requires people who are bilingual in the languages concerned, but investigators may not be able to recruit someone who is familiar with both Cantonese and Bengali. Even in questionnaires designed for English speaking samples, there may be dissent as to the meaning, interpretation, and appropriateness of some items.18
The quality of data obtained from surveys of non-English speakers may be compromised by inadequacies in the translation procedures, failure to compare questionnaire content across languages, failure to consider the cultural appropriateness of items for use with English speakers, and lack of standardisation in terminology, sampling, and the grouping of samples.
We established whether previous studies that measured prevalence of tobacco and alcohol use in ethnic minority groups in the United Kingdom applied guidelines for cross cultural research (box 2) and looked for evidence of inconsistency in the empirical findings that might indicate problems.
We identified publications of investigations on prevalence of tobacco or alcohol consumption, or both, in minority ethnic groups in the United Kingdom. Although our focus was on national studies because of their influence on policy, we also included local studies that measured the prevalence of consumption of tobacco, alcohol, or smokeless tobacco. We did not include studies in which tobacco or alcohol consumption was a small component of a health or lifestyle questionnaire.
Box 1: Principles for adapting a questionnaire for use in different languages
Panels of bilingual people translate questionnaires into the target language(s);as there are several ways to translate, the panel members negotiate a “best fit”
As bilingual people may not be representative of the target population because of education, age, and, in some cases, sex, the people who are monolingual in the target language(s) assess meaning and acceptability of the translations and the appropriateness of the response options and instructions and modifications are made
Field testing of the resultant questionnaire is done to check face and content validity; further changes are made
Where there has been translation into more than one language, each language version is compared with every other to ensure comparability; this may lead to further adjustment of the items
Tests of criterion and construct validity, reliability, and responsiveness are carried out in the target language(s)
Box 2: Summary of 12 guidelines for maximising cross cultural validity of questionnaire
Questionnaire source—whether professional, lay, or both
Piloting of questionnaire
Validity testing of original questionnaire
Reliability testing of original questionnaire
Responsiveness testing of original questionnaire
Discussion of translations of questionnaire by bilingual people
Discussions of translations with community members who speak only one language
Comparison of original questionnaire with each translation
Comparison of each translation with every other
Validity testing of translated questionnaire(s)
Reliability testing of translated questionnaire(s)
Responsiveness testing of translated questionnaire(s)
Databases searched were BIOMED (Medline; CINAHL); Web of Science (Science Citation Index, Social Sciences Citation Index, Arts and Humanities Citation Index); PsychINFO; and Embase and the Cochrane Library. We used keyword searches, cited reference searches, and hand searching of relevant academic journals. Key words used included “drinking”, “consumption”; “tobacco,” “cessation”; “bidi”; “biri”; “paan”; “smokeless tobacco”; “betel quid”; “race”; “black”; “questionnaire”; “adaptation”; “translation”. To maximise the scope of our search strategy we used truncated terms such as “alcohol*”; “cigarette*”; “ethnic*”; “minorit*”; “cross-cultural*”; “valid*”; “India*”; “Pakistan*”; “Bangladesh*”; “Chin*”; “Afric*”.
We identified 43 publications. Publications were grouped according to relevance: key studies providing detailed prevalence data (national and local); relevant studies providing some data but not focusing on tobacco and alcohol; and background studies that were discussions of this issue. We analysed 18 key studies and for 15 of these obtained the questionnaires from the authors, who were informed about the work. Three questionnaires were unobtainable because the researcher(s) had disposed of the questionnaire, the researcher had left the organisation and contact details were unavailable, or the survey instruments could not be retrieved. The investigators' translations were sought. Five questionnaires were from national surveys. Fourteen of the 15 studies included data on tobacco, 11 on alcohol, and six on smokeless tobacco products. The studies on smokeless tobacco products are not tabulated here but are included in the evaluation of methods (table available from authors).
We summarised tobacco and alcohol consumption by ethnic group and sex. Consistency across the studies should increase confidence in the translations and methods used to collect the data while large differences would cast doubt.
Procedures for the translation and adaptation of questionnaires and interview schedules from English into other languages have been evolving since the 1960s. There are well established guidelines in the literature on linguistics and patient-assessed outcomes in cross-cultural research.8 9 11 12 14 15 19 20 We analysed the survey instrument(s) for each key study and publication according to these guidelines (box 2). Our analysis was sent to the authors for checking and for additional information. Nine authors responded to requests to check our analysis of their study. Data from these nine authors, however, supported our analysis, with few changes being made. Mostly, we have preserved the terminology relating to ethnic groups used by the authors—for example, use of the term “white” or “European.” However, we use the term South Asian to refer to mainly Indian, Pakistan, and Bangladeshi populations, and European to mean predominantly white populations. General populations in the United Kingdom are usually more than 90% white. Other terms are as in general use.
Tables 1 and 2 show the reported prevalence of tobacco consumption by ethnic group in 14 studies.2–6 2129 We were able to compare the five national studies because they had similar aims, but the local studies were too different to permit comparisons. The results from the national studies showed important discrepancies (table 1). For example, in Bangladeshi women in the health survey for England the prevalence of cigarette smoking was 1% compared with 6% in the survey of black and minority ethnic groups in 1994.4 The prevalence of smoking in men of Chinese origin in the health survey for England 1999 was 17% compared with 31% in the fourth national survey of ethnic minorities (1993-4). The prevalence for African-Caribbean men was 29% in the 1992 survey of black and ethnic minority groups in England6 and 42% in the fourth national survey of ethnic minorities.5
Cotinine, a derivative of nicotine, can indicate whether a person smokes (at levels on or above 15 mg/ml) or has had recent exposure to tobacco smoke through passive smoking.2 In the health survey for England 1999 when the prevalence of smoking was adjusted for cotinine it was substantially higher for men from most ethnic minorities except for the Chinese (table 1). For example, 53% of Bangladeshi men reported that they used any tobacco product but the cotinine adjusted rate was 59%; in Bangladeshi women the equivalent figures were 27% and 38%. This level of discrepancy was not seen in the general population—for example, in women the figures were 27% and 29%, respectively. These figures show more inaccuracy of self reported data in most ethnic minority groups compared with populations of European origin.
Table 3 shows considerable consistency between ethnic groups in the national studies for reported drinking among men. For women, however, the results were mixed. There was consistency between studies for Pakistani, Bangladeshi, African-Caribbean, Chinese, and European women. However, in Indian women differences existed—for example, in the fourth national survey of ethnic minorities 18% of Indian women reported drinking alcohol compared with 35% in the health survey for England 1999. Table 4 summarises data on alcohol consumption from local studies.
Using the 12 guidelines in box 2 we appraised publications and information obtained from nine of the 15 authors and the study by Pearson et al reporting only on smokeless tobacco.31 The health survey for England 1999 met three of the 12 guidelines and partially met one other.2 Three studies met three guidelines.5 6 26 Two of these studies were national,5 6 and one was a local study in Newcastle.26 Most studies fulfilled one or two of the guidelines. The findings are presented in categories—administrative aspects of the survey, source of questions, testing of questionnaires, piloting, and translation methods.
Table 5 shows that all national surveys included an interview. Most questionnaires were in dual language format, with the question written in English and the translation underneath. The health survey for England 1999 differed in that its translated questionnaires were exclusively in the target language with the interviewers' coding instructions in English. Five of the local studies did not translate their questionnaires (in writing), two being targeted at schoolchildren who could be assumed to speak English.
Ideally, survey questions should be based on consultation with lay people.32 Only four studies explicitly consulted with the community to design the English questionnaire (table 6). The surveys of black and ethnic minority groups in England in 1992 and 1994 used focus groups and individual interviews with ethnic minority people to develop the questionnaire.4 6
Only two studies reported that the questionnaire had been validated (table 6). However, they gave no details of the validation process and these could not be obtained from the authors. Information on reliability and responsiveness of the English questionnaires was not provided. Fourteen of the 15 studies carried out some piloting of research instruments (table 6). This varied from piloting the questionnaire in English only, to piloting it in different language versions in particular geographical areas to test appropriateness of wording and acceptability.3 4 5
Two of the studies explicitly engaged in a group translation process.26 29 All of the others used a single translator (table 7). The 1992 survey of black and ethnic minority groups in England involved consultations with the community to investigate sensitivity of questions and cultural taboos but did not seek their opinion on the accuracy, simplicity, and conceptual equivalence of the translation.6 During piloting the health survey for England 1999 sought the views of monolingual people on how understandable the questions were. Interviewees were asked to express any concerns to the interviewer that they may have had with the translated question.2
None of the studies compared each language version of the questionnaire with every other language to check for linguistic and conceptual equivalence. Most of the studies (such as the fourth national survey of ethnic minorities5 and the health of minority groups 19992) compared translations to the original English version. Some, but not all, studies used the written back translation method to do this.6 262930 None of the studies retested the translated questions for validity, reliability, or responsiveness.
We assessed the consistency of findings and processes for maximising cross cultural validity of survey instruments on tobacco and alcohol use in epidemiological studies, particularly those translated from the original English into the languages of ethnic minorities in Britain. We assessed studies by comparison of reported prevalence figures, analysis of the process of development of the survey instruments (both reported here), and comments from Bengali speakers on the versions translated into Bengali (reported elsewhere33). All three methods showed problems in the translation of the questionnaires and their cultural appropriateness.
There were important discrepancies between self reported prevalence of tobacco in the five national surveys. This may be because the fieldwork for the studies was done during different time frames, the earliest in 1991 and the most recent in 1999; the sampling frames were different; the questions were not standard across studies; and the mode of administration of the questionnaires was not the same. However, most of these explanations apply equally to the comparison, predominantly white populations, in which the consistency was much greater. Comparisons between cotinine adjusted and self reported prevalences of tobacco use were greatest in ethnic minority groups, indicating that self reporting was less valid in these populations. For some ethnic minority populations the anomalies in prevalence of tobacco use are so striking that the above explanations seem inadequate. One potential source of discrepancy is that for non-English questionnaires the items had been translated. This does not, however, explain the discrepancy in data on African-Caribbean men where the questionnaires were in English. Inadequate cross-cultural adaptation is an important potential explanation for these discrepancies.
Limitations of the study and strengths
Several limitations should be taken into account in the interpretation of the findings. Tables 6 and 7 may be incomplete as not all of the authors provided the required detail, despite our repeated attempts to obtain this. As a result of communicating with nine authors, however, our key amendments were on whether the translation had been compared with the original or with every other, or both. In terms of meeting guidelines authors made few other amendments. This suggests that the information contained in publications and reports, though brief, gives a reasonable indication of the methods used for translation and design. Our 12 guidelines are those most salient to epidemiology and were agreed before we examined the papers. Although our study was limited to tobacco and alcohol, it gives direction to more general future research.
Implications of the study
We have questioned the quality of data obtained from surveys with non-English speaking ethnic minorities. New questionnaires should be developed and guided, as far as possible, by established guidelines for cross cultural research. To our knowledge, studies such as this one are rare, and we have not been able to identify any that are directly comparable. Bowden and Fox-Rushby recently reported a review of the process of translation and adaptation of generic health related quality of life measures internationally.34 Their review used guidelines based on similar sources to assess whether authors of original articles had followed recommended processes. Their conclusions were similar to ours—that is, those using and developing health related measures had paid insufficient heed to cross cultural equivalence, particularly conceptual. Even such stringent guidelines may not lead to full cross cultural equivalence, and more work is needed including further development and validation.35
What is already known on this topic
There are major ethnic variations in key risk factors, including consumption of tobacco and alcohol
Collecting data on risk factors by ethnic group in multi-ethnic settings is necessary but difficult
Translation into appropriate languages and back translation are necessary but insufficient steps to safeguard cross cultural validity
What this study adds
Discrepancies in prevalence data between even the national surveys are substantial and call for greater attention to the validity of study methods
The scientific literature offers guidelines on how to conduct surveys in cross cultural settings, but most of these have not been implemented in UK surveys on tobacco and alcohol
There is an urgent need to improve the cross cultural validity of survey methods, particularly in multilingual, multi-ethnic societies
Future research and policy application
Our findings have implications for the design of future epidemiological research that depends on self report from ethnic minority groups, especially in older people, new immigrants, and refugees living in the United Kingdom. The health survey for England in 2004 will once again focus on ethnic minority groups and will present an opportunity to enhance the processes by which cross cultural equivalence can be maximised. Policy makers, health planners, health promoters, and health carers should use existing data to underpin their efforts to reduce the prevalence of risk factors in UK ethnic minority groups.1 36 It is vital, however, that they do so with awareness of the data's strengths and limitations, some of which are considered in this paper. The principles and issues raised here, and discussed in greater detail elsewhere,37 are likely to be relevant to all multi-ethnic societies.
We thank Rory Williams and Hannah Bradby for advice on relevant publications and studies; the librarians at the Health Education Board for Scotland (HEBS) library in Edinburgh and the Medical Research Council (MRC) library in Glasgow for assistance; Marshall Dozier and Irene McGowan at the Erskine Medical Library in Edinburgh for training on database searching; the authors of the publications and reports who gave up considerable time and effort to provide and check information about their studies and questionnaires; Hazel King for secretarial services; and Sarfraz Mohammed for computing support.
Contributors RB planned the study with input from all other authors and wrote the paper. AV undertook the literature reviews and fieldwork and drafted the project report. SH, SW, LH, and AA helped RB to supervise the project. All authors provided critical comment on the drafts of the paper. RB, AV, and SH are guarantors.
Funding Scottish Cancer Group of the Scottish Executive.
Competing interests None declared.