Results

Since 1988, 2577 publications in Medline are tagged by “ROC curve”, but only seven of them were in Russian (0.0027) while of all diagnosis related publications 0.076 were in Russian. Of all diagnosis related English language publications 0.033 (standard error of proportion m=0.0007) used ROC analysis while of similar Russian publications only 0.001(m=0.0004) used ROC analysis. The increasing frequency of English language studies using ROC analysis (Medline data) is displayed in figure 1. Russian studies do not show definite time trend.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Frequency of ROC analysis use in diagnosis related publications from English language journals and from Russian language journals, Medline data (%±m). Because the numbers are small, data are pooled for adjacent years (for Russian publications 1988–1990, 1991–1995, 1996–1997; for English language publications 1988–1989, 1990, 1991–1994, 1995, 1996, 1997).

The hand search in the journal Radiologyreveals more frequent application of ROC analysis than the whole Medline: 5.7% (m=2.3) in 1991 (6 of 106) and 7.3% (m=2.8) in 1993 (6 of 82) of all diagnosis related articles. In 1991 and in 1993 the hand search of two Moscow-based radiology journals revealed not a single article that used ROC analysis among 120 articles about diagnostic tests. In addition, in 1993, not one abstract of the last all USSR congress of medical radiologists reflected use of ROC analysis.

Figure 2 shows the proportion of papers presenting results of longitudinal studies or clinical trials. The frequency of such methodologically advanced research in Russian publications is lower than that in the international medical literature. The increase of the frequency of advanced research in the international literature is seen also in Russian publications, but the Russian literature lags far behind the international numbers.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Frequency of selected advanced types of research in English language and in Russian language publications, related to human subjects, Medline data (%±m). Frequencies are averaged for two adjacent years (1978 and 1979, 1980 and 1981, etc.)

The database of the Central Medical Library of the Russian Ministry of Health contains 67 dissertations for degrees of candidate or doctor of medical science published in 1995 and 1996. None were devoted to problems of non-infectious diseases. All journal titles related to epidemiology are journals devoted to the epidemiology of infectious diseases and microbiology and virology.

The curriculum recommended by Federal Ministry of Health has the epidemiology course and the recommended programme of the course is devoted only to infectious diseases epidemiology.

Discussion

There are major differences in the types of methodology used in medical research in Russia as compared with those applied by the international community. ROC analysis is all but ignored in the Russian literature devoted to diagnostic problems despite the fact that this type of analysis became standard in the early 1980s, especially in radiology.23 Clinical trials and studies of longitudinal design are rare in USSR/Russian medical research.

There are several reasons for the underutilisation of advanced methodology in Russian medical research. Firstly, the limited resources of Soviet/Russian medical science are an important contribution: clinical trials and longitudinal studies are costly study designs.

Secondly, the state of epidemiology as a science in Russia is antiquated. While around the world epidemiology is taught as “science of the art of medicine”,24 epidemiology in Russia is concerned with the spread of infectious diseases. It is detrimental to the methodology of all medical research.

The emphasis on infectious diseases stems from the personal preferences of Russian medical leaders, as is the case in many areas of Russian life.16 The President of the Russian Academy of Medical Sciences, who also headed the powerful State Research and Development Institute of Epidemiology, strongly resisted change (“Epidemiology is a Russian Science”).25 26 He claimed that, because infectious disease epidemiology has an “object of research”, it is the only legitimate, scientific epidemiology. Despite the traditional limitation, there is growing use of the “risk factor” approach to analyse the environment and for health surveillance and disease prevention.27 28 The argument about whether to transform Russian epidemiology continues, but still, according to State classification, epidemiology as a speciality (code 14.00.30) includes only infectious diseases. It is reflected by dissertations published in epidemiology, as was find when checking their content.

Thirdly, there are no schools of epidemiology in Russia, and not one university has a department of clinical epidemiology and biostatistics as found in the West. In Western universities courses of epidemiology and biostatistics are united and devoted to teaching main epidemiological conceptions, study design and statistical analysis of the data. In departments of epidemiology of Russian medical universities students are taught the epidemiology of infectious diseases. As the Ministry of Health recommends in the standard curriculum, elements of statistics are taught in classes of social medicine and in short courses of mathematics and “informatics”. Statistics are taught only as a means of evaluating disease prevalence and outcomes in state health statistics. It is a longstanding tradition of teaching social medicine.29 A substantial part of the curriculum is devoted to hygiene, especially to microbiological aspects of food hygiene, analysis of the nutritional content of products, and work place safety.11 The teaching of non-infectious disease epidemiology is scarce and limited to epidemiology of some diseases at clinical departments. Few research teams exist in Russian cancer epidemiology2 and in other fields of specialty medicine.

From the point of history, teaching statistics to students of biology and medicine in universities almost stopped in the 1930s as a part of exorcising genetics and its instruments from Soviet science.30 31 Until the 1950s use of statistical techniques in research was ideologically suspicious.31 The restoration of teaching statistics was insufficient and ineffective in undergraduate medical education as well as in postgraduate education.32 Inefficiency of teaching is reflected in the narrow spectrum of statistical methods used, as shown for Russian psychiatric publications,33 34 and in the biased use of statistical methods.31 This recent study of use of statistical methods in Russian medical research concluded that the use of statistics is commonly a kind of ritual of camouflage for the sake of adding ‘importance’ to the report or dissertation.31Our data and some other reports show that the problem has deeper roots—it is in the study designs rather than in the statistical treatment of the data.33 34

Fourthly, during the Soviet era, individual subscriptions to international journals were not permitted, and in libraries they were available only in central specialised research institutes and in a few libraries in Moscow and some other cities. In all of the USSR, there was only one copy of the Journal of Clinical Epidemiology, and a few copies of theAmerican Journal of Epidemiology and theAmerican Journal of Public Health available to the public, as indexed in All-USSR Catalogue of Foreign Periodicals.

The situation today actually has worsened. Russia has still not a single journal devoted to biostatistics or to epidemiological research. The existing “epidemiological” journals, in actuality, are journals of infectious diseases. Despite the appearance of elements of a market economy, medical textbooks are printed under government grants, perpetuating the domination of ministerial policy. There are no contemporary epidemiological textbooks for medical or public health specialists in Russia. Texts on clinical trials35 and on methods of evaluating diagnostic tests exist.36 37However, since the publication of the classic handbook of epidemiology in 1965,38 the first book on epidemiology was published only in 1998.39

Available books on statistics for physicians provide the basics of statistical analysis, but do not discuss research design, possible bias, and other elements of clinical epidemiology. It is probable that the lack of relevant education and available literature contributes importantly to the poor quality of research in Russia as documented here. International experts helping reforms of Russian medicine suggested that ineffective use of informational technologies and ineffective decision making is probably more harmful for health care than insufficient funding.40

Fifthly, the special service—Sanitary Epidemiological Service—exists in Russia. It plays a significant part in the surveillance of infectious diseases. This service employs thousands of specialists in hospitals, districts of large and small towns, and in rural provinces. The infectious diseases are the target of the efforts of the service. The non-infectious disease epidemiology is still not the major object of the attention of the Epidemiological Service. Only the large regional offices of this Service do hygienic (ecological) testing in the residential houses (house hygiene) as well as in industrial enterprises (industrial hygiene). All health statistics in Russia are of low quality, which makes it difficult to estimate trends in the nation's health41 as well as epidemiological research of cancer and other diseases.2 ICD is not in use everywhere in Russia despite declarations of USSR Ministry of Health since the 1960s and current efforts of Russian health authorities.

Another problem is the absence in Russia of accurate equivalents of the main terms used in epidemiology, such as incidence, prevalence, longitudinal study, case-control study, etc. In a textbook of epidemiology currently in use in many universities,42 for example, screening is described as “mass tests of the population for collecting the data which specialists need” (page 399). Our examination shows that in the electronic catalogue of the Central Medical Library of the Ministry of Health in Moscow terms used in the description of modern study designs are incorrectly translated and poorly recognisable. For example, “clinical trials” is translated as “klinicheskie issledovaniia”. Translated back to English, this means “clinical research”. “Randomised controlled trials” is translated as “slepogo otbora metod kontroliruemii”; the English meaning is “blind selection method, controlled”. A further example appears in the Ethical Code of the Russian Medical Association,43 which mentions “Mass surveys of epidemiological character”, but no other types of epidemiological studies. These examples indicate that epidemiology not only is defined officially as an infectious disease discipline, but also that the field is imperfectly understood in Russian medicine.

Because there are no specialists in epidemiology, Russian physicians receive no training in epidemiological methods. The difficulty of accessing the literature and the misrepresentation of epidemiology in publications contributes to the low proportion of longitudinal studies and the high proportion of uncontrolled studies reported here. In the absence of funding for costly types of research such as clinical trials and cohort studies, Russian scientists tend to produce descriptive and retrospective studies. When Russian scientists do controlled clinical trials, reports from these trials are almost exclusively positive,20 which means that, not only that negative results are published less frequently (publication bias), but probably the results of trials are biased. Widespread wrong designs of studies and prevalent bias in statistical analyses led some scholars to the conclusion that there is a methodological crisis in Russian medical research.31 All these results and discussion are true for other countries of the former USSR.

The review of cancer epidemiology led Boyle17 to the conclusion that “epidemiology is alive and well in central and Eastern Europe, where it has managed to continue to function under extremely difficult conditions”. Now, after this evaluation of the epidemiology in USSR and Russia using the systematic analysis of the design of research, teaching and practice of epidemiology, I regret that epidemiology in Russia is more dead than alive. There is, however, a foundation for the growth and development of this important field. The recognition of gaps and inadequacies is perhaps the first essential step toward reaching that goal.