Background
Participants in research may differ systematically from the population of interest. For example, participants included in an influenza vaccine trial may be healthy young adults, whereas those who are most likely to receive the intervention in practice may be elderly and have many comorbidities, and are therefore not representative. Similarly, in observational studies, conclusions from the research population may not apply to real-world people, as the observed effect may be exaggerated or it is not possible to assume an effect in those not included in the study.
Selection bias can arise in studies because groups of participants may differ in ways other than the interventions or exposures under investigation. When this is the case, the results of the study are biased by confounding.
Example
A study of the prevalence of Parkinson’s disease (PD) completed a door to door survey of an entire US county. They used a two-stage screening technique, first administering a comprehensive questionnaire and then referring those subjects with signs or symptoms suggestive of PD for a neurological evaluation. Over 97% of the households in the county participated. Some 15% of those screening positive in the first screen refused follow-up. Extensive efforts provided ‘valuable information on almost all the refusals which was reviewed by a neurologist to establish a diagnosis’. The authors presented convincing evidence that they had succeeded in obtaining a complete enumeration of PD cases in the county.
Of the approximately 24,000 residents on prevalence day, 1 January 1978, PD was diagnosed in 31 participants. Thirteen of those 31 had never been seen for medical care. In this survey, if another approach to the ascertainment of cases had used only the medical care system, all of those who had not received care (over 40%) would not have been identified. Furthermore, there would have been no definitive way of characterizing the bias introduced if only those identified via health records were used.
Another example is the effect of HRT on coronary heart disease (CHD) in women. Several studies showed that HRT reduced coronary heart disease (CHD), but subsequent RCTs showed that HRT might increase the risk of CHD disease. The Women in the observational studies on HRT were more health conscious, more physically active, and had higher socioeconomic status than those not on HRT. This self-selection of women (selection bias) led to confounding and a “healthy-user bias”.
In a double-blind placebo-controlled study of the use of phenobarbital for prevention of recurrent febrile seizures, adherence was not as good as hoped. The Kaplan-Meier curves for remaining free from seizures were not statistically significantly different from each other, contrary to expectation. The authors used several definitions of adherence and reanalysed the results ‘as treated’. The results based on one definition of adherence showed that adherent subjects in both the phenobarbital and the placebo groups had a higher risk of recurrence than those who were non-adherent. The results were thus conflicting and demonstrated selection bias due to attrition.
Prospective cohort studies of dietary and lifestyle factors exhibit a “healthy participant effect”, reporting lower mortality rates among participants than among the general population. This suggests that people who are interested in healthy lifestyles, and therefore have more healthy behaviours, such as low smoking rates, are more likely to sign up to take part in a prospective study than those with less healthy lifestyles. This can also be considered a sampling bias
A study of cigarette smoking and dementia found potential selection bias in the elderly. Selection bias due to censoring by death was one explanation for the lower relative rate of dementia in smokers with increasing age.
Impact
Selection bias can have varying effects, and the magnitude of its impact and the direction of the effect is often hard to determine. (Odgaard-Jensen J et al.)
As examples, a meta-epidemiological study of the Impact of Selection Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions found significantly larger treatment effect estimates in trials that had inadequate/unknown sequence generation (difference in ES = 0.13; 95% CI: 0.01 to 0.25). A further study to determine survival in preterm infant cohort studies found that the presence of selection bias overestimated survival by as much as 100%.
Preventive steps
To assess the probable degree of selection bias, authors should include the following information at different stages of the trial or study:
– Numbers of participants screened as well as randomised/included.
– How intervention/exposure groups compared at baseline.
– To what extent potential participants were re-screened.
– Exactly what procedures were put in place to prevent prediction of future allocations and knowledge of previous allocations.
– What the restrictions were on randomisation, e.g. block sizes.
– Any evidence of unblinding.
– How missing data from participants lost to follow-up were handled.
Randomisation of participants in intervention studies aims to provide the fairest method of comparing the effect of an intervention with a control, and preventing selection biases is part of this aim. However, it may not be perfectly achieved. Berger and colleagues have made clear arguments for the crucial role of adequate allocation concealment and randomisation procedures to prevent selection biases.
Because anything that happens after randomisation can affect the chance that a study participant has the outcome of interest, it is essential that all patients (even those who fail to take their medicine or accidentally or intentionally receive the wrong treatment) are analysed in the groups to which they were allocated. The intention-to-treat analysis includes data from all the participants randomly assigned to the treatment comparison groups, whether or not they received the treatment to which they were assigned, even if they never started the treatment, or switched to a different one during the study. This prevents bias caused by disruption of the baseline equivalence established by random allocation.
In Observational studies selection bias is difficult to address with analytical methods, but methods for dealing with missing data are available, including last observation (or baseline value) carried forward, mixed models, imputation, and sensitivity analysis using ‘worst case’ scenarios (assuming that those with no information all got worse) and ‘best case’ scenarios (assuming that all got better). Analysing data only from participants remaining in the study is called complete case analysis.
Certain external measures can sometimes be used to calibrate the data from a study, an example being standardised mortality rates. Moreover, inverse probability weighting can be used under certain assumptions.
To improve generalisability of study findings the selection of the population should be broad and reported in the recruitment/inclusion criteria.
