Biasing of studies towards results desired by the investigators—investigator bias—arises from many sources, from outright data fabrication to subtle and even unconscious bias in design and analysis choices. Investigator bias has had an important impact in some areas of clinical practice, and can be a major source of uncertainty about study effects. Among the sources of investigator bias, empirical studies have suggested that estimates of effect are often associated with funding source. Disclosure of financial ties thus supplies predictive information about study results. Although this predictability does not by itself say which results are more or less biased, it does stand as a potentially important source of study variation, and hence is needed for full uncertainty assessments. The key problem is then fair use of disclosure data by the evaluator. Fair use will require a clear understanding that predictive power at the group level should never be used for indictment let alone claim of impropriety (as has occurred). Evaluators need to be on the alert for their own biases, and if they wish to use these biases they should be given the form of a subjective prior distribution.

Many have argued that full-scale uncertainty analysis is needed before public-health or medical recommendations are made.1^–4 These arguments have largely concerned formal analysis of uncertainty arising from the “holy trinity” of validity threats: uncontrolled confounding, selection bias and measurement error; model misspecification is sometimes added to the list. Occasionally, attention is drawn to more subtle (but potentially large) biases due to sparse data, regression to the mean, data dredging and other instances of method failure.5 6

Ongoing news7^–13 suggests it is time to debate incorporation of another potentially major source of uncertainty into literature assessments: investigator bias, the biasing of study results towards results …