Positive results bias

The tendency to submit, accept and publish positive results rather than non-significant or negative results.

Background

Positive results bias occurs because a considerable amount of research evidence goes unpublished, which contains more negative or null results than positive ones. This leads to spurious claims and overestimation of the results of systematic reviews and can also be considered unethical. Non-publication of results can also lead to research wastage as researchers may unnecessarily repeat studies because the results are unpublished.

Example

A review of empirical studies and assessment of 300 systematic review found that trials with positive outcomes are twice as likely to be published, and published faster, compared with trials with negative outcomes (Song et al.). This review also found that there was ‘convincing evidence that outcome reporting bias exists and has an impact on the pooled summary in systematic reviews. 

It is also possible that industry is more likely to design studies with a high likelihood of a positive outcome, for example, by selecting a comparison population likely to yield results favouring the product (Djulbegovic et al. 2000).  

Impact

A Cochrane review analysed the relationship between publication and the significance and direction of results (Hopewell et al. 2009). Trials with positive results were more likely to have been published than those with negative or null results: relative risk 1.78 (95% CI 1.58 to 1.95). The review also demonstrated that trials with negative findings, when published, were slower to be published: 6 to 8 years after completion compared with 4 to 5 years for trials with positive results.

A New England Journal of Medicine paper examined the publication rate of trials relating to antidepressants. (Turner et al. 2008) 36 of 37 (97%) antidepressant trials with positive results were published, compared with 3 of 22 (14%) antidepressant trials with negative results.

Preventive steps

Registration of clinical trials before inception is one way to reduce publication bias, as theoretically it is then possible to hold the trialists accountable for publication of the trial results regardless of the findings.  The AllTrials Campaign has called for registration and reporting of all trials.

There has been a focus on pharmaceutical intervention studies, but medical devices and all forms of intervention should be subject to the same trial reporting considerations. Moreover, in an era where large-scale observational studies are common, there is a need to be aware of the positive reporting bias concerns across all research designs.

Systematic reviews should attempt to find and include all relevant studies that are difficult to find and should report statistical methods to deal with the presence of unpublished results.

Awareness of publication bias amongst the research and publishing communities should help reduce the issue of negative results being rejected by journals. The lack of reporting of negative research studies in mainstream media suggests that solutions to positive results bias and more broadly, publication bias, need to reach further than scientific journals.

Sources

Hopewell S et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1.

Sackett DL. Bias in analytic research. J Chron Dis 1979; 32: 51-63

Song F, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8).

Turner EH et al. (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.

What does all trials registered and reported mean? http://www.alltrials.net/find-out-more/all-trials/

What doctors don’t know about the drugs they prescribe – video https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe


PubMed feed

These sources are retrieved dynamically from PubMed:

View more →