Observer bias

The process of observing and recording information which includes systematic discrepancies from the truth.

Background

Observer bias is a type of detection bias that can affect assessment in observational and interventional studies. Parta’s Dictionary of Epidemiology gives the following definition: “Systematic difference between a true value and the value actually observed due to observer variation” and continues to describe observer variation.

Many healthcare observations are open to systematic variation. For example, in the assessment of medical images, one observer might record an abnormality but another might not. Different observers might tend to round up or round down a measurement scale. Colour change tests can be interpreted differently by different observers. Where subjective judgement is part of the observation, there is great potential for variability between observers, and some of these differences might be systematic and lead to bias. Observation of objective data, such as death, is at much lower risk of observer bias.

Biases in recording objective data may result from inadequate training in the use of measurement devices or data sources or unchecked bad habits. By recording subjective data, predispositions of the observer are likely to underpin observer biases. Observers might be somewhat conscious of their own biases about a study or may be unaware of factors influencing their decisions when recording study information.

Randomized controlled trials are designed to provide the fairest test of an intervention. However, if any part of the data collection process involves observation, observer bias can affect the measurement in the study.

Example

Observer bias has been repeatedly been documented in studies of blood pressure. Clinicians measuring participants blood pressure using mercury sphygmomanometers have been found to round up, or down, readings to the nearest whole number. Observer bias may also occur if the researcher has a preconceived idea of what the blood pressure ought to be, leading to arbitrary adjustments of the readings.

Impact

Hróbjartsson and colleagues produced three systematic reviews estimating the size of the impact of observer bias, by comparing estimates from studies in which outcome assessors were blinded to the intervention with those in which outcome assessors were not blinded. Three types of RCTs were investigated: those with binary outcomes; RCTs with measurement scale outcomes and RCTs with time-to-event outcomes.

The studies included a range of conditions from angina to wound treatment to psychiatric disorders. For RCTs with binary outcomes, non-blinded outcome assessors generated odds ratios that, on average, were exaggerated by 36%. For clinical trials that used measurement scale outcomes, non-blinded outcome assessment exaggerated effect size by 68% (6). For RCTs using time-to-event outcomes, non-blinded assessment overstated the hazard ratio by approximately 27%.

Preventive steps

A key method is to ensure that outcome assessors are blinded to the exposure status of study participants. This can apply to randomised controlled trials, in which an individual has been allocated a particular intervention, and also to observational studies, which track the progress of study participants with different exposures. Achieving blinding might mean separating access for data on exposures from data on outcomes; in a blinded trial the allocation should remain unknown throughout the study (unless it must be revealed for safety reasons).

Strategies can also include adequate training for observers in how to record findings, identifying any potential conflicts before recordings commence and clearly defining the methods, tools and time frames for collecting data.

Another preventive aspect includes training study observers to become aware of their prejudices and habits, in order to improve accuracy. One study on blood pressure looked at training procedures designed to reduce observer bias and how long they lasted. The study showed nurses had biases in either under or over reporting blood pressure; training did reduce the between-nurse variation but differences remained and were not changed by training at various time points.

Whilst observer bias can be reduced, it is likely that observer bias will always remain, and researchers should be aware of this when analysing and evaluating data.

Sources

Beevers G et al. ABC of hypertension. Blood pressure measurement. Part I-sphygmomanometry: factors common to all techniques. BMJ 2001;322:981–5

Bruce NG et al. Observer bias in blood pressure studies. J Hypertens.1988;6(5):375-80

Hróbjartsson A et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 2012;344:e1119

Hróbjartsson A et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ 2013;185:E201–11

Hróbjartsson A et al. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol 2014;43:937–48

Porta M et al. A dictionary of epidemiology. 6th edition. New York: Oxford University Press: 2014

Stewart MJ et al. Measurement of blood pressure in the technological age. Br Med Bull. 1994 Apr;50(2):420-42


PubMed feed

These sources are retrieved dynamically from PubMed

View more →