Differential Reference Bias

When not all participants receive the same reference test in a diagnostic accuracy study.

Background

A differential reference bias is a type of a verification bias. In diagnostic studies, a differential reference bias occurs when study participants receive different reference tests. The presence of a differential reference bias can mean that the mixing of data ignores potential differences in the accuracy of the reference tests. As a result, combining outcome data can misrepresent the “true” pattern of disease presence or absence.

Many reference tests are invasive, expensive, or carry a procedural risk (e.g. angiography, biopsy, surgery), and therefore, patients and clinicians may be less likely to pursue the gold standard test in all cases.

Example

In one study of an elbow extension test to rule out elbow fracture, participants with a positive index test received the reference test of radiography. Those who did not undergo radiography (for example because the index test was negative) received a structured follow-up assessment by telephone.

Impact

There is consistent evidence of an association with accuracy when differential reference bias is present, but the direction of the effect varies. Some reviews have reported an overestimation of test accuracy, while others have noted an effect on specificity but not sensitivity.

In general, it is difficult to predict the magnitude and direction of the effect of differential reference bias on the results as its presence can lead to a test perceived as being more or less accurate.

Studies, where the reference standard was an expensive and/or invasive test, are particularly prone to differential reference bias. Although these designs may have been used for ethical, funding or practical/clinical reasons, they introduce the potential for this kind of verification bias.

Preventive steps

Differential reference bias is avoided when all study participants receive the same reference test. In diagnostic studies where this is not possible, some statistical methods have been developed to correct for the effect of differential reference bias. These statistical treatments attempt to correct for the inconsistency of measures produced by reporting of the outcomes (positive and negative tests) but should be used with caution.