If in a case-control study, cases are selected during one period and controls are selected during another period, then the relationships observed between exposures and outcomes of interest might be affected. Changes in disease or diagnostic definitions, exposures over time and treatments could all contribute to non-contemporaneous bias.
Case-control studies can use historical controls in their design. For practical reasons, this can be a useful approach since it avoids the need to collect new information for the control group. However, this risks introducing non-contemporaneous control bias as over time there may have been changing factors affecting controls and their experiences, obscure true relationships or exaggerate relationships.
Historical and contemporaneous cohort studies can be used to examine rare exposures and multiple effects of a single exposure over time.
An intervention study using historical controls assessed the effect of a psychosocial intervention programme for parents of premature infants (Gonya et al. 2014). The intervention aimed to improve the coping capacity of parents to reduce the length of hospital stay and reduce readmission rates. The intervention was already implemented. Therefore the researchers used a pragmatic evaluation by selecting a historical comparison group (they obtained data on outcomes for infants before the introduction of the intervention).
However, Other factors (non-contemporaneous control bias) could have changed between the two-time points, such as the level of ill health of the infants, improvements in medical care, new treatments that influenced the results.
(Papageorgiou et al 2017) investigated how the use of historical controls versus contemporaneous controls affected the effect estimates from clinical trials of orthodontic interventions: they found that use of historical controls produced smaller effect estimates (across 13 meta-analyses the standardised mean difference was reduced by 0.31 (95% CI = 0.10 to 0.53). This effect did not differ if the interventional group was studied prospectively or retrospectively and by the study’s sample size.
Comparison of effects from randomised controls and historical controls in clinical trials found six therapies for which 50 randomised controls and 56 historical controls were reported. Studies with historical controls were nearly four times more likely to report positive results (Sacks 1982) 44/56 (79%) historical controls found the therapy better than the control regimen, but only 10/50 (20%) randomised controls. Largely caused by differences in outcome for the control groups. Historical controls generally did worse than the randomised control groups. Adjustment of outcomes for prognostic factors in the groups did not change the results.
To avoid this bias, concurrent controls should be used where possible. When evaluating the results from studies using non-contemporaneous controls caution is warranted.
If the study does consider characteristics then it needs to ensure that the people in the study have the same changing characteristics as the population of interest.