In the ‘About’ page of our Catalogue we state the following:
Sackett recognised the importance of bias in research. His 1979 paper “Bias in Analytic Research”, published in the Journal of Chronic Diseases, reported the first draft of a ‘catalog of biases which may distort the design, execution, analysis, and interpretation of research.’ Sackett catalogued 35 biases that arise in sampling and measurement, in the context of clinical trials, and listed 56 biases potentially affecting case-control and cohort studies.
We need to make a correction to the above statement. Whilst Dave did indeed present 35 biases that formed the beginning of a catalogue, he did not start it himself. The James Lind Library provides the backstory to Dave’s first presentation of a “catalog of biases” at a meeting chaired by Michel Ibrahim which discussed and debated the conflicting views about the validity of case-control study designs. Dave’s contribution was to compile a catalogue of the biases that might need to be taken into account in evaluating observational data. Befitting to his character, he acknowledges the person who started the catalogue in the footnote on page 1 of his 1979 paper:
‘The catalog was initiated by a clinical epidemiology graduate student, JoAnne Chiavetta: it was benefitted from the contributions of a number of colleagues (especially John C Sinclair) and other publications (especially references [5] and [6]).”
It turns out then that the catalogue was initiated by JoAnne Chiavetta, a graduate student in clinical epidemiology studying at McMaster University where Dave was departmental chair. Credit must go to JoAnne for providing some of the material that Dave presented about the biases affecting clinical epidemiology, without which we would not have been inspired to fulfil Dave’s wish.
How fitting that JoAnne’s surname means ‘little key’. Through the magic of the internet, we found the little key (see picture). JoAnne was surprised and honoured when we contacted her and was only too glad to tell us her story and how the catalogue idea came about. And here is that story.
JoAnne Chiavetta
The origin of the Catalogue of Bias by JoAnne Chiavetta
My interest in research design and methods began when I was a research assistant for about 2 years at the University of Toronto and was assigned to identify NIH (National Institute of Health) requests for proposals and develop designs for submission, along with working on specific research questions for investigators and grant proposals. This was an invaluable experience in providing an overview of what information was available and still unavailable, what possibly required supplementation with new information, and where expansion of already existing information would be useful. Being able to outline what might be needed if a research question is of value and how new information might make a difference was an essential component of creating successful proposals.
In 1979, I was a graduate student in clinical epidemiology and biostatistics at McMaster University in Hamilton, Ontario, Canada. During my time there, I came across many inconsistencies in research methodology across the literature. Popular study designs (such as the cohort study and the case-control study) were known to have particular strengths and weaknesses, implying that some research questions were more suited to one type of study than another. Beyond selecting an appropriate and feasible study design, I noticed a number of ways in which the composition of the study sample could lead to biased results. The logistical difficulties of enrolling participants in epidemiological studies meant that investigators were often forced to rely on convenience samples, and so may have been subject to sampling bias and confounding effects directly impacting their conclusions.
This observation led me to begin outlining the various sampling biases potentially affecting clinical epidemiological studies. Inspired by the rigorous critical thinking taught by Dave Sackett (who was my departmental chair at the time), that initial list eventually became part of his seminal article on research biases. Dave was kind enough to credit me in the publication of the original catalog of biases, a testament to his collaborative nature in an era when graduate students were typically overlooked in publications.
The catalog set out biases in a systematic way that allowed investigators to examine studies for potential sources of bias, consider countermeasures to mitigate or prevent the occurrence of known biases, and also more readily identify previously unknown biases. Cultivating an actively expanding online version of the catalog is an excellent endeavour to increase knowledge about biases, making them easier to recognize and avoid.
My hope for the catalog is that it will assist researchers in producing results of the highest possible quality, maximizing the utility of their work and ensuring that future research has a solid foundation to build upon. It is certainly an exciting time for epidemiological research with the development of increasingly sophisticated biostatistical tools, the wide availability of computerized clinical datasets, and the wonderful collaborative initiatives such as this one amongst scientists worldwide.
JoAnne Chiavetta, March 2019
