Kay Dickersin: Making all the evidence count

 

Remembering a pioneer whose work transformed our understanding of bias in medical research.

We were saddened to learn of the death of Kay Dickersin, a pioneering epidemiologist whose work changed how we understand, investigate and prevent bias in medical research.

Kay helped establish something that now seems fundamental to evidence-based medicine: the evidence we can see is not necessarily representative of all the evidence that exists.

Imagine ten clinical trials investigating whether a treatment works: five produce encouraging results; five find little or no benefit. Now imagine that only the positive trials are published. A clinician reviewing the literature might reasonably conclude that the treatment works. Even a carefully conducted systematic review could reach a misleading conclusion, simply because important evidence was missing.

This is publication bias. And Kay Dickersin was one of the scientists who did most to bring it to the attention of the medical research community.

Kay’s influential 1990 paper, The existence of publication bias and risk factors for its occurrence, helped establish that selective publication was not merely a theoretical concern. Studies reporting positive or statistically significant findings were more likely to be published than those reporting negative or inconclusive results.

Her work exposed a fundamental vulnerability in evidence-based medicine: even the most rigorous methods of evidence synthesis cannot produce reliable conclusions when important studies are missing. She pursued this problem with remarkable determination, documenting its consequences and challenging the research community to address them.

Kay was ahead of her time, not only in recognising how medical evidence could be distorted, but in anticipating what was needed to make it more complete and trustworthy. Her contribution extended well beyond publication bias. Long before clinical trial registration became an established expectation, she was helping to develop the tools and infrastructure needed to identify studies and make their findings accessible.

She helped establish the Oxford Database of Perinatal Trials, one of the earliest specialist clinical trial registries. She went on to become a founding member of the Cochrane Collaboration and established the US Cochrane Centre in 1994, helping advance systematic reviews as a foundation for evidence-informed healthcare. 

Her foresight was also evident in her early efforts to identify and share the resources that others needed. In a chapter of the 1994 Handbook of Research Synthesis, she documented 26 clinical trial registries that researchers could use to identify clinical studies – at a time when trial registration was far from routine.

But Kay’s contribution extended beyond identifying missing studies. Together with colleagues, she investigated the many ways research findings can be selectively reported or disseminated. These include outcome reporting bias, where only selected outcomes are reported; time-lag bias, where positive findings may be published more quickly; and language bias, where the language of publication affects which evidence is identified.

In 2011, Kay and Iain Chalmers published Recognizing, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the WHO. Their account examined the long history of efforts to recognise and address incomplete reporting, including how selective citation can distort the apparent evidence base, a problem addressed in our entry on citation bias.

One of Kay’s greatest contributions was working to change the systems that allowed these problems to persist. She was a leading advocate for prospective clinical trial registration, which enables the identification of missing studies, comparison of published outcomes with those originally planned, and accountability for making findings available.

Kay also helped to make it clear that people participate in clinical trials to contribute to knowledge that may benefit future patients. When results remain unpublished, that contribution can be wasted. Clinicians may make decisions using incomplete evidence, patients may receive ineffective or harmful treatments, and researchers may unnecessarily repeat studies.

Kay and her many collaborators, including fellow EBM luminaries like Chalmers and Drummond Rennie, argued forcefully that research findings, including benefits and harms of treatments, should be made publicly available, regardless of whether the results were positive, negative, or inconclusive.

Even if you were unfamiliar with Kay’s broader work, if you’ve ever done a systematic review or even glanced over the Cochrane Handbook, you’ve likely encountered her influence through her contribution to developing the Highly Sensitive Search Strategy for identifying randomised controlled trials. 

As editors and authors of the Catalogue of Bias, we see Kay’s influence in many of the problems we seek to explain. Our definition of publication bias draws directly on her work, while entries on selective outcome reporting, time lag, language, and citation biases address related distortions of the evidence base.

Kay Dickersin helped change not only how we understand bias in medical research, but also what we expect of those who conduct and report it. Evidence-based medicine requires more than critically appraising the studies we can find. It requires knowing what evidence is missing, understanding why it is missing, and doing something about it.

We are grateful for Kay’s extraordinary contribution to medical research and evidence-based healthcare. Her work will continue to inform ours.

Written by David Nunan and Carl Heneghan (Editors, Catalogue of Bias), and Nicholas DeVito (Catalogue of Bias author and co-author of the Publication bias entry), on behalf of all authors of the Catalogue of Bias.