LIMITATIONS OF “NOSOLOGICAL INDICES”

Indeed, apart from interpretative problems, “nosological indices” such as sensitivity and specificity are problematic as they will vary with the spectrum and selection of patients.^2–4 However, also “diagnostic indices” (predictive accuracy) are not constants. Taking account of all relevant covariables is, if at all useful, generally impossible because many of these are not clear, unmeasurable, or too particularistic for a useful general evaluation. So, we must not overemphasise the value of diagnostic indices. And where we use them, we must be very specific as to objective, context, and setting. We should also be keen on setting specific external validation when we want to generalise observed index values for clinical purposes. Acknowledging all this, it cannot be denied that for clinical and epidemiological purposes there is often a need to use “nosological indices” to communicate general characteristics of a test. Also Feinstein did not escape from speaking about characteristics required for a “rule out” test (high sensitivity) compared with a “rule in” test (high specificity). The question what index is “the best” is too general, and therefore a non-issue. While in principle all indices (and combinatory roles, using multivariable analysis) can be calculated from data on the association between test outcome and disease, which index is actually used depends on objective and context. For example, knowing about sensitivity and specificity is important for the selection of the most appropriate screening test, while a high predictive value is important when evaluating the diagnostic value of a pre-mastectomy cancer diagnosis.

Bayes's theorem, ROC curves with varying cut off values, likelihood ratios, and logistic regression can be unattractive for many clinicians and also these entities are not insensitive for spectrum and selection.⁵ But for various purposes they cannot be missed, for example, in data analysis, computer algorithms, and in finding optimal cut off values for screening tests. At the same time, many challenges in the context of data analysis remain to be adequately dealt with (such as: assessing additional diagnostic value of a test rather than accuracy in itself⁶; differences between multivariable analysis for diagnostic and aetiological purposes (with major emphasis on subgroup analysis for the former); meta-analysis of research data and making them tailormade at the same time; and bridging the gap between clinical reasoning in practice and products of data analysis.

DESIGN ISSUES MORE IMPORTANT

We should not forget that mathematical indices are just ways to summarise collected research data. For the quality of research, defining the research question, and methodological challenges in study design, are far more important. The high degree of difficulty of diagnostic research studies is reflected by various overviews over the past decades, repeatedly showing important flaws in a high proportion of studies,^7–9 and indicating only slow improvement.

Diagnostic research should be improved and refined with respect to strategy (where phase I to phase IV studies should be subsequently designed^1,10), spectrum and selection effects, diagnostic and prognostic reference standards, and the assessment of the clinical impact of testing. Better methods to warrant and evaluate external clinical validity are required. Furthermore, proposed innovations must always be compared with achievements of experienced clinicians, before they are recommended as better. We need more understanding of the “doctor's black box” of diagnostic decision making, jumping between observations on groups and caring for individuals, including reassurance. This will facilitate more efficient diagnostic reasoning, and developing custom-made support systems. Efficiency in evaluation of the impact of diagnostic procedures can be gained if new data on specific aspects (for example, a diagnostic test) can be inserted in the mozaic of available evidence on a clinical problem, rather than studying the whole problem again whenever one element has changed. For this purpose, flexible scenario models of current clinical knowledge are needed. This can be strongly supported by ongoing prospective systematic review and meta-analysis of diagnostic studies and developing clinical databases.

DIA-PROGNOSTIC SIGNIFICANCE

As Feinstein points out, significance for prognosis and medical decision making is the justification of diagnosis. This insight is already often implemented and evaluated, for example in the process of staging tumours or in the clinical classification of heart failure. But we must recognise that (evaluation of) prognostic significance needs much more attention, especially where test technology advances without concomitant therapeutic improvements. A well known example is detailed, three dimensional MRI observations of the brain of which clinical relevance is not (yet) clear.

Innovation of biomedical knowledge and understanding pathophysiological processes is decisive for the development of tests with better “dia-prognostic” impact. DNA diagnostics is in fact prognostics as it touches the basic functional level of our biomedical nature. DNA testing will not only be supportive of genetic counselling and reproductive applications, but also for diagnostic and prognostic purposes. Moreover, DNA testing is expected to improve the targeting and dosing of interventions. Much work in this field is being done, for example, in cardiovascular medicine, oncology, and psychiatry. However, it will take much time before these promises will have impact in daily patient care. Considerable efforts are still needed not only in the laboratory but also in clinical epidemiological reseach¹¹ including long term follow up to clinically validate diagnostic and prognostic predictions.

INDIVIDUALISATION AND DEALING WITH (UN)CERTAINTY

In view of the ambition to develop a more tailormade, perhaps even individualised, “dia-prognostic” process, study population oriented validations will get increasingly under pressure. In this context, n=1 research, focused on individual patients, represents great methodological challenges. In addition, the moment of testing and the occurrence of a clinical problem will not necessarily be always related. The DNA profile with its predictive potentials is there from the beginning. Ethical questions on when to test, how to deal with privacy of genetic information, and the right of (not) knowing have to be addressed. Doctors and patients, traditionally fighting to reduce uncertainty, must also learn to cope with possibly reaching the goal of getting certainty.

FUNDING AND FORMAL STANDARDS

In contrast with therapeutic research, and despite the clinical fact that accurate dia-prognosis is the key to effective management, the funding possibilities for dia-prognostic research are generally poor. Feinstein's plea for improvement is hitting the target. Such improvement would be stimulated by adopting formal standards for evaluation of dia-prognostic procedures, to control acceptance, maintenance, and substitution in the healthcare market. This also requires high quality and transparency of evaluation reports. The recent initiative taken by the Amsterdam Academic Medical Centre to reach international agreement on Standards for Reporting Diagnostic Accuracy (STARD) deserves therefore full support from the scientific and healthcare community.