Bias and conclusions

Judith Lumley, Jeanne Daly
2002 Australian and New Zealand Journal of Public Health  
Any issue of a public health journal provides food for thought on issues of sampling, and this one is no exception. Three papers involve relatively unusual samples -houses and child occupants, classrooms and child occupants, shops and basic foods -but the largest category of papers is about populations. These populations are defined by locality (Busselton, Western Australia; Auckland, New Zealand), by occupation (New Zealand fire fighters), by health problem and locality (diabetes and New South
more » ... betes and New South Wales), by reason for health service contact and locality (pregnancy and Christchurch, birth and Victoria) and by age and locality (New South Wales, Perth). Readers need to know whether and how the people taking part in the study reflect the population being studied, and whether there are any systematic differences between them. If there are systematic differences, how do they modify the conclusions that the authors -or the readers -might draw? Do these results apply more generally to other groups of people? These are methodological considerations that need to be taken into account in all research articles. Methodological issues III: bias, samples and conclusions Four papers in this issue illuminate these questions about sample selection and conclusions. The population of interest to Jill Cockburn and colleagues was 'community-dwelling adults in NSW'. The selection of the study group involved a random selection of households from the electronic NSW telephone directory, followed by a letter of information about the study and then a telephone contact. The next stage was to select one adult in the household -the person with the next birthday -to be interviewed, over the phone. This process resulted in a response fraction of 61.4%, adjusted for those able to be contacted to 75.4%. Only those who were 40 and over were interviewed about bowel cancer and bowel cancer screening, the focus of this article, and their characteristics were then compared with NSW Census data. On almost all criteria the study group was representative of the NSW population, the only exception being a higher participation in the study by women (60% compared with 52% in the Census) and a lower participation by men (39% compared with 48% in the Census). Bess Fowler and colleagues sought to contact a random sample of men aged 65 to 83 years living in the metropolitan area of Perth, using the Western Australia State Electoral Roll to invite them to a vascular screening clinic. They refer to an earlier paper describing recruitment. The file of potentially eligible men was linked to a file of nursing home addresses to exclude men 'likely to have significant physical or cognitive impairments' and unlikely to benefit from the screening and subsequent surgical intervention. One letter of invitation and one reminder resulted in 62.4% of men attending for screening. The account of the non-participants is very helpful: 2,278/7,371 were ineligible through death (364), prolonged absence from Perth (236), having already had surgery for the condition (397) or having the condition (28), being housebound or too ill to attend (686), change of address and untraceable (567). The adjusted response fraction was 70.5%. Those excluded by the use of the electoral roll were residents who were not Australian citizens, something which is likely have a different degree of importance in different parts of Australia, but the participation rate of those invited was very high. The details about non-participants show the importance of major ill-health and increasing age as factors limiting participation. The population of interest to Stephanie Brown and colleagues was women who gave birth in a defined time period (one week or two weeks) in Victoria, excluding those who had a stillbirth or a neonatal death. These women were mailed a questionnaire about their health six to nine months after birth and asked about their views and experiences of maternity care from early pregnancy onward. The questionnaires were mailed by maternity hospitals and homebirth practitioners to all their clients who had given birth in the defined time and returned to the researchers. This ensured anonymity and confidentiality, but it did cause some errors in the mailing, such as questionnaires being sent to women whose births occurred outside the study dates. The adjusted response fraction, after excluding the group outside the study dates, a few duplicates and those unable to be delivered at the mailing address held by the hospital, was 67%. One strength of this study is the availability of state-wide data from the Victorian Perinatal Data Collection Unit for the defined time period, which makes it possible to characterise differences between respondents and non-respondents with respect to both social and demographic factors and obstetric and reproductive factors. Obstetric and reproductive factors have little effect on responses but there are consistent social differences, predictable in postal surveys, that limit the conclusions which can be drawn about younger women, single women and women born outside Australia in countries where English is not the first language. These are presented and discussed in the paper. The people of the Shire of Busselton have been active participants in mapping population health since 1966. Dallas English and colleagues describe a study that began there in 1978. The population attending to be measured, tested and to complete a comprehensive questionnaire on lifestyle comprised 4,006 people, 74% of the 5,415 registered on the electoral roll. Among this population of attendees, 3,230 were 25 to 79 years old and completed the questions about current smoking. They form the study cohort followed up until 1994 through records of hospital separations and death certificates. The question of how similar the study group of 3,230 is to the 5,415 does not apply here since the research questions are about the relationship between smoking measured in 1978 and subsequent hospital admission and death. This study is notable for its major success in verifying the vital status of all participants by linkage to the electoral roll, telephone directories, direct contact and through relatives, with only 2% lost to follow-up.
doi:10.1111/j.1467-842x.2002.tb00151.x fatcat:kgorwb5nvnbwff47clasps3lnq