Diagnostic tests are traditionally characterized by simple measures of efficacy such as the sensitivity and the specificity. These measures, though widely recognized and easy to understand, are subject to definitional arbitrariness. Moreover, studies constructed to estimate the sensitivity and specificity are susceptible to a variety of biases. In this paper the various potential problems are described with reference to examples from the diagnostic literature. These difficulties have implications for the design of diagnostic test evaluations, and the choice of suitable measures of test efficacy.