Background

One-sided reference bias occurs when a study author cites only publications that demonstrate one side of the picture of available evidence. This bias may arise when researchers cite publications that support their preconceptions or hypotheses, ignoring evidence that does not support their view. This can happen in any study report, but a particular problem arises when this occurs in literature reviews, which are supposed to represent a comprehensive collection of all relevant information, along with description and appraisal of quality and content. The result can be a misrepresentation of the current totality of evidence and can lead to spurious claims or needless additional research.

Example

‘Retrieving literature by scanning reference lists may produce a biased sample of articles, and reference bias may also render the conclusions of an article less reliable.’ Gøtzsche, 1987.  Reference bias in reports of drug trials. BMJ 1987; 295: 654-656

Impact

An analysis of one-sided reference bias examined reference lists of articles reporting on clinical trials in rheumatoid arthritis. From 111 articles reviewed, 22 showed a negative selection of references and 44 a positive selection (while 35 were not amenable to analysis because all the references gave the same outcome).

Analysis of reference bias in narrative review articles that discussed house dust mites interventions in asthmatics found that the majority of reviews had reference bias. The most quoted trial had only seven participants per group, and the most quoted nonrandomized study included just ten participants per group but claimed very positive results.

An analysis of a citation network for articles that reported the hypotheses that beta-amyloid is a protein accumulated in the brain in Alzheimer’s disease led to unfounded claims due to the presence of reference bias. (Greenberg SA. 2009) The author concluded that one-sided use of citations can be used to ‘generate information cascades resulting in unfounded authority of claims.’

Preventive steps

Researchers need to begin with a systematic, unbiased, search of the literature pertaining to their area of interest. Authors should present their work in the context of all available information on the topic, which should include referring to articles that support and do not support the findings of their study if they exist. Literature review authors should follow a pre-published protocol and should report the searches used to find studies, the databases searched and how articles were selected for inclusion in the review.