Partial reference bias

When only a proportion of the study group receive both the index and reference standard test when investigating for diagnostic accuracy.


A partial reference bias is a type of a verification bias (also referred to as a work-up bias, or a referral bias). A partial reference bias results in missing data and a potential misrepresentation of the accuracy for a new test against a reference standard test.

Various mechanisms or reasons can be offered as to why a study group would not receive both tests. For example, many reference tests are invasive, expensive, or carry a procedural risk (e.g. angiography, biopsy, surgery), and therefore, patients and clinicians may be less likely to pursue further tests, especially if a preliminary test is negative.


In a study, designed to evaluate the accuracy of positron emission tomography (PET) to determine the presence of malignancy in patients with lung nodules, their standard reference test was histological examination, commonly known as a biopsy. As only positive PET exams could receive the biopsy test, in effect, only the patients who were positive on the PET exam were tested using the reference standard.


If a diagnostic study is at high risk of partial reference bias, the results could overestimate sensitivity and specificity, and in some cases, suggest a decrease in specificity of the index test.

In general, it is difficult to predict the magnitude and direction of the effect of verification bias on the results as its presence can lead to a test being more or less accurate.  

Studies, where the reference standard was an expensive and/or invasive test, are particularly prone to partial reference bias. Although these designs may have been used for ethical or funding restrictions, they introduce the potential for verification bias.

Preventive steps

Ideally, in a diagnostic accuracy study, all patients should receive the same reference test.  

However, obtaining a reference test in every patient may not be ethical, practical, or cost effective, which can lead to partial reference bias. One way to reduce verification bias in clinical studies is to perform the reference test in a random sample of study participants. Some statistical methods have been developed to correct for partial reference bias, but these should be used with caution.


Cronin AM, Vickers AJ. Statistical methods to correct for verification bias in diagnostic studies are inadequate when there are few false negatives: a simulation study BMC Med Res Methodol. 2008 Nov 11;8:75. doi: 10.1186/1471-2288-8-75.

de Groot JA, et al. Verification problems in diagnostic accuracy studies: consequences and solutions. BMJ. 2011 Aug 2;343:d4770.

de Groot JA, et al. Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies. Am J Epidemiol. 2012 Apr 15;175(8):847-53.

Karch A et al. Partial verification bias and incorporation bias affected accuracy estimates of diagnostic studies for biomarkers that were part of an existing composite gold standard. J Clin Epidemiol. 2016 Oct;78:73-82. doi:10.1016/j.jclinepi.2016.03.022 Epub 2016 Apr 21.

Lee J, et al. Accuracy of F-18 fluorodeoxyglucose positron emission tomography for the evaluation of malignancy in patients presenting with new lung abnormalities: a retrospective review. Chest 2001 Dec; 120(6): 1791–1797.

Rutjes AW et al. Evidence of bias and variation in diagnostic accuracy studies.CMAJ. 2006 Feb 14;174(4):469-67.

Whiting PF, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-536

Whiting PF et al. A systematic review classifies sources of bias and variation in diagnostic test accuracy studies. J Clin Epidemiol. 2013 Oct;66(10):1093-104.

PubMed feed