Non-contemporaneous control bias

Differences in the timing of selection of case and controls within in a study influence exposures and outcomes resulting in biased estimates.

Background

If, in a case-control study, cases are selected during one period and controls are selected during another period, then the relationships observed between exposures and outcomes of interest may be affected.  Changes in disease or diagnostic definitions, exposures over time, and treatments, could all contribute to non-contemporaneous bias.

Case-control studies sometimes use historical controls.  For practical reasons, this can be a useful approach since it avoids the need to collect new information for the control group. However, this risks introducing non-contemporaneous control bias as, over time, factors affecting controls and their experiences whic may have changed, obscure or exaggerate relationships

Historical and contemporaneous cohort studies can be used to examine rare exposures and multiple effects of a single exposure over time.

Example

An intervention study using historical controls assessed the effect of a  psychosocial intervention programme for parents of premature infants (Gonya et al. 2014). The intervention aimed to improve the coping capacity of parents,  reduce the length of hospital stay and readmission rates. The intervention had already been implemented.

 

The researchers, therefore, used a pragmatic evaluation by selecting a historical comparison group (they obtained data on outcomes for infants before the introduction of the intervention). However, Other factors could have changed between the two-time points, such as the degree of ill health of the infants, improvements in medical care and new treatments.

Impact

(Papageorgiou et al 2017) investigated how the use of historical controls rather than contemporaneous controls affected the effect estimates derived from clinical trials of orthodontic interventions: they found that use of historical controls produced smaller estimates of effect  (across 13 meta-analyses the standardised mean difference was reduced by 0.31 (95% CI = 0.10 to 0.53). This effect did not differ if the interventional group was studied prospectively or retrospectively, or by the study’s sample size.

Comparison of effects from randomised controls and historical controls in clinical trials found six therapies for which 50 randomised controls and 56 historical controls were reported. Studies with historical controls were nearly four times more likely to report positive results (Sacks 1982)  44/56 (79%) historical controls found the therapy better than the control regimen, but only 10/50 (20%) with randomised controls, largely caused by differences in outcome for the control groups. Historical controls generally did worse than the randomised control groups. Adjustment of outcomes for prognostic factors in the groups did not change the results.

Preventive steps

To avoid this bias, concurrent controls should be used where possible. Caution is warranted when evaluating the results of studies using non-contemporaneous controls .