OBJECTIVE To examine the current state of epidemiology in Russia.
DESIGN The structure of clinical research and statistical methods was used to shed light on the epidemiology in Russia. The frequencies of specific study designs were evaluated using Medline data for 1970–1997. To determine the proportion of advanced design clinical studies the frequency of cohort, prospective, follow up, or longitudinal studies, and controlled trials was evaluated. All diagnosis related studies were found to determine the usage of advanced statistical technique (ROC analysis). The adequacy of Medline information was checked by hand search of journals. All dissertations in epidemiology defended in Russia in 1995 and 1996 were evaluated for their methodology. The curriculum recommended by Ministry of Health to Medical Universities was evaluated. Available literature and library indexing of epidemiological terms examined.
MAIN RESULTS Russian medical research uses less frequently advanced study designs and methods of data analysis. Medical students are taught epidemiology as a science of spread of infectious diseases. There is no department of epidemiology in Russian universities where epidemiology is taught in the modern sense and no epidemiological and biostatistical periodicals available in Russia.
CONCLUSION Epidemiology in Russia remains in an archaic state of science of the spread of infectious diseases and it is detrimental to methodology of medical research in Russia.
- Soviet Union
- study design
- comparative studies
Statistics from Altmetric.com
Data on the health of the Soviet people were elusive because of the political nature of documentation in medicine as in other aspects of Soviet life. It was difficult to separate actual successes from failures or falsified data.1-3 A related problem involved the inappropriate generalisation of successes in specific areas, such as aerospace medicine, which were mistakenly extrapolated to medical science in general.4
In the 1970s and 1980s, Western physicians visiting the USSR found not only outdated technology, but also unethical medical practices and a bureaucratic state system that regulated both medical care and the physician's professional life.5-8 As Soviet/Russian society became more open, the impressions of visiting doctors who studied Russian medicine changed.9-11 The most prevalent current impressions are that medicine in the former USSR suffers severely from insufficient funding.12-14 Evaluations of the quality of Russian medicine usually were negative,8 12 14-17 despite changes in nearly every aspect of physicians' training11 12 15 and increased exposure to modern treatment options.8 12 14 18 With few exceptions, evaluations of the Russian scientific literature also were negative,11 14 18 19 but they did not develop systematically the state of medical research.
Except for the personal impressions and limited case studies noted above, the state of medical science in Russia remains unknown. One recent bibliometric study found that clinical trials from Russia bring only the “positive” results, which means that research practice and publications are probably biased.20 The hypothesis of our study was that research study designs can shed some light on the state of medical science in Russia, especially on the state of epidemiology, which influence practice of all medical research. This report compares methodologies used in Soviet/Russian medical research with those of studies from other countries. It attempts to describe the current state of epidemiology in Russia as a science, field of research, and teaching discipline, and to discuss the influence of the development of epidemiology and biostatistics on Russian medical research.
Three aspects of the epidemiological theory and practice were evaluated: the extent to which selected advanced study designs (longitudinal studies and controlled trials) and advanced biostatistical techniques (receiver operating characteristic curve analysis (ROC) analysis as an example) were applied; the content of Russian dissertations in epidemiology for 1995 and 1996; and the content of Russian epidemiological journals and the teaching practice in universities.
Publications were retrieved from Medline. English and Russian language publications were compared. Russian language publications come from the USSR and countries of the former USSR. After 1991, they are predominately from Russia. It is not possible to separate origin of articles by country because most medical journals of the former USSR do not yet provide authors' affiliations or addresses. Of all human related articles located by Russian language or address, only 3.7% have addresses in 1997. The number of studies from Russia published in foreign languages is not significant: 7.5% of all human related publications located by Russian language or authors' address in 1995 and in 1997. Location by language is sensitive enough and gives a less biased statistic than selection of Russian origin publication by address: in 1997 of 361 non-Russian language publications all 361 were publications in English and 19 of them used US Government support. It means that Russian research that is published abroad comprises very special publications, many of which are a result of international collaboration.
There is another problem related to the comparison of the English language publications with publications in other languages. The articles in English are probably indexed better in Medline than articles in foreign languages, but there is no proof of this hypothesis. On the other hand, the collection of English language journals indexed in Medline is more comprehensive than the collection of Russian (or Chinese) journals. This last aspect of selection bias leads to probable underestimation of the differences between English language and Russian language journals.
To determine the frequency of use of selected type of advanced biostatistical method, for years 1988–1997 all diagnosis related studies (tagged in MeSH with ‘diagn*’) were examined. Studies applying the most sophisticated method of analysis of diagnosis related data—ROC analysis—were selected by searching for ‘ROC curve’ in MeSH. This method of analysis was developed during the second world war and became the standard of quality of description of diagnostic test effectiveness.21 The term was introduced in Medline in 1988.
To check the adequacy of Medline information, all 1991 and 1993 issues of Meditsinskaia Radiologiia (Medical Radiology) and Vestnik Rentgenologii i Radiologii (Archives of Radiology)—two major USSR/Russian radiological journals—and the journal Radiologywere checked by hand, page by page. The hand search was undertaken in addition to Medline search because the translation of titles and abstracts and subsequent database tagging of journal articles is sometimes misleading. Radiological journals were selected because ROC analysis was used in radiology from the beginning and more frequently. Articles related to diagnostic problems were selected, and the frequency of research using ROC analysis were evaluated.
To determine the frequency of advanced study design human subject studies for 1978–1997 were reviewed. All studies tagged in the MeSH field as cohort, prospective, follow up, or longitudinal were collected as longitudinal. Clinical trials were sought using publication types clinical trials, controlled clinical trials, or randomised controlled trials. These two groups of research designs were used as “advanced” because of their increasing use during the past 50 years and because results of the research of these types are used as most reliable “evidence” in the interpretation of aetiology, prognosis, and interventions.22
Dissertations in epidemiology that were defended in Russia in 1995 and 1996 were evaluated for subject. They were found using the electronic database of Central Medical Library of the Ministry of Health in Moscow. The catalogue of this library was examined for Russian equivalents of terms intensively used in description of modern study designs (flavours of longitudinal studies and clinical trials). All Russian medical, public health and biological journals of 1998 were checked using the collection of the same library. All titles related to epidemiology were checked for their content. The curriculum recommended by Ministry of Health to Medical Faculties of Universities and the programme of epidemiology, recommended by Ministry of Health were examined.
All hand searching, Medline analysis, catalogue and curriculum examinations were done by the author.
Bibliometric analysis shows that Russian clinical research uses advanced study design and modern statistical techniques less frequently than in English language publications.
Epidemiology in Russia remains in an archaic state of the science of spread of infectious diseases.
Russian physicians and other specialists are not taught epidemiology in the modern sense.
The absence of teaching epidemiology and shortage of literature harms the methodological quality of Russian medical research.
Since 1988, 2577 publications in Medline are tagged by “ROC curve”, but only seven of them were in Russian (0.0027) while of all diagnosis related publications 0.076 were in Russian. Of all diagnosis related English language publications 0.033 (standard error of proportion m=0.0007) used ROC analysis while of similar Russian publications only 0.001(m=0.0004) used ROC analysis. The increasing frequency of English language studies using ROC analysis (Medline data) is displayed in figure 1. Russian studies do not show definite time trend.
The hand search in the journal Radiologyreveals more frequent application of ROC analysis than the whole Medline: 5.7% (m=2.3) in 1991 (6 of 106) and 7.3% (m=2.8) in 1993 (6 of 82) of all diagnosis related articles. In 1991 and in 1993 the hand search of two Moscow-based radiology journals revealed not a single article that used ROC analysis among 120 articles about diagnostic tests. In addition, in 1993, not one abstract of the last all USSR congress of medical radiologists reflected use of ROC analysis.
Figure 2 shows the proportion of papers presenting results of longitudinal studies or clinical trials. The frequency of such methodologically advanced research in Russian publications is lower than that in the international medical literature. The increase of the frequency of advanced research in the international literature is seen also in Russian publications, but the Russian literature lags far behind the international numbers.
The database of the Central Medical Library of the Russian Ministry of Health contains 67 dissertations for degrees of candidate or doctor of medical science published in 1995 and 1996. None were devoted to problems of non-infectious diseases. All journal titles related to epidemiology are journals devoted to the epidemiology of infectious diseases and microbiology and virology.
The curriculum recommended by Federal Ministry of Health has the epidemiology course and the recommended programme of the course is devoted only to infectious diseases epidemiology.
There are major differences in the types of methodology used in medical research in Russia as compared with those applied by the international community. ROC analysis is all but ignored in the Russian literature devoted to diagnostic problems despite the fact that this type of analysis became standard in the early 1980s, especially in radiology.23 Clinical trials and studies of longitudinal design are rare in USSR/Russian medical research.
There are several reasons for the underutilisation of advanced methodology in Russian medical research. Firstly, the limited resources of Soviet/Russian medical science are an important contribution: clinical trials and longitudinal studies are costly study designs.
Secondly, the state of epidemiology as a science in Russia is antiquated. While around the world epidemiology is taught as “science of the art of medicine”,24 epidemiology in Russia is concerned with the spread of infectious diseases. It is detrimental to the methodology of all medical research.
The emphasis on infectious diseases stems from the personal preferences of Russian medical leaders, as is the case in many areas of Russian life.16 The President of the Russian Academy of Medical Sciences, who also headed the powerful State Research and Development Institute of Epidemiology, strongly resisted change (“Epidemiology is a Russian Science”).25 26 He claimed that, because infectious disease epidemiology has an “object of research”, it is the only legitimate, scientific epidemiology. Despite the traditional limitation, there is growing use of the “risk factor” approach to analyse the environment and for health surveillance and disease prevention.27 28 The argument about whether to transform Russian epidemiology continues, but still, according to State classification, epidemiology as a speciality (code 14.00.30) includes only infectious diseases. It is reflected by dissertations published in epidemiology, as was find when checking their content.
Thirdly, there are no schools of epidemiology in Russia, and not one university has a department of clinical epidemiology and biostatistics as found in the West. In Western universities courses of epidemiology and biostatistics are united and devoted to teaching main epidemiological conceptions, study design and statistical analysis of the data. In departments of epidemiology of Russian medical universities students are taught the epidemiology of infectious diseases. As the Ministry of Health recommends in the standard curriculum, elements of statistics are taught in classes of social medicine and in short courses of mathematics and “informatics”. Statistics are taught only as a means of evaluating disease prevalence and outcomes in state health statistics. It is a longstanding tradition of teaching social medicine.29 A substantial part of the curriculum is devoted to hygiene, especially to microbiological aspects of food hygiene, analysis of the nutritional content of products, and work place safety.11 The teaching of non-infectious disease epidemiology is scarce and limited to epidemiology of some diseases at clinical departments. Few research teams exist in Russian cancer epidemiology2 and in other fields of specialty medicine.
From the point of history, teaching statistics to students of biology and medicine in universities almost stopped in the 1930s as a part of exorcising genetics and its instruments from Soviet science.30 31 Until the 1950s use of statistical techniques in research was ideologically suspicious.31 The restoration of teaching statistics was insufficient and ineffective in undergraduate medical education as well as in postgraduate education.32 Inefficiency of teaching is reflected in the narrow spectrum of statistical methods used, as shown for Russian psychiatric publications,33 34 and in the biased use of statistical methods.31 This recent study of use of statistical methods in Russian medical research concluded that the use of statistics is commonly a kind of ritual of camouflage for the sake of adding ‘importance’ to the report or dissertation.31Our data and some other reports show that the problem has deeper roots—it is in the study designs rather than in the statistical treatment of the data.33 34
Fourthly, during the Soviet era, individual subscriptions to international journals were not permitted, and in libraries they were available only in central specialised research institutes and in a few libraries in Moscow and some other cities. In all of the USSR, there was only one copy of the Journal of Clinical Epidemiology, and a few copies of theAmerican Journal of Epidemiology and theAmerican Journal of Public Health available to the public, as indexed in All-USSR Catalogue of Foreign Periodicals.
The situation today actually has worsened. Russia has still not a single journal devoted to biostatistics or to epidemiological research. The existing “epidemiological” journals, in actuality, are journals of infectious diseases. Despite the appearance of elements of a market economy, medical textbooks are printed under government grants, perpetuating the domination of ministerial policy. There are no contemporary epidemiological textbooks for medical or public health specialists in Russia. Texts on clinical trials35 and on methods of evaluating diagnostic tests exist.36 37However, since the publication of the classic handbook of epidemiology in 1965,38 the first book on epidemiology was published only in 1998.39
Available books on statistics for physicians provide the basics of statistical analysis, but do not discuss research design, possible bias, and other elements of clinical epidemiology. It is probable that the lack of relevant education and available literature contributes importantly to the poor quality of research in Russia as documented here. International experts helping reforms of Russian medicine suggested that ineffective use of informational technologies and ineffective decision making is probably more harmful for health care than insufficient funding.40
Fifthly, the special service—Sanitary Epidemiological Service—exists in Russia. It plays a significant part in the surveillance of infectious diseases. This service employs thousands of specialists in hospitals, districts of large and small towns, and in rural provinces. The infectious diseases are the target of the efforts of the service. The non-infectious disease epidemiology is still not the major object of the attention of the Epidemiological Service. Only the large regional offices of this Service do hygienic (ecological) testing in the residential houses (house hygiene) as well as in industrial enterprises (industrial hygiene). All health statistics in Russia are of low quality, which makes it difficult to estimate trends in the nation's health41 as well as epidemiological research of cancer and other diseases.2 ICD is not in use everywhere in Russia despite declarations of USSR Ministry of Health since the 1960s and current efforts of Russian health authorities.
Another problem is the absence in Russia of accurate equivalents of the main terms used in epidemiology, such as incidence, prevalence, longitudinal study, case-control study, etc. In a textbook of epidemiology currently in use in many universities,42 for example, screening is described as “mass tests of the population for collecting the data which specialists need” (page 399). Our examination shows that in the electronic catalogue of the Central Medical Library of the Ministry of Health in Moscow terms used in the description of modern study designs are incorrectly translated and poorly recognisable. For example, “clinical trials” is translated as “klinicheskie issledovaniia”. Translated back to English, this means “clinical research”. “Randomised controlled trials” is translated as “slepogo otbora metod kontroliruemii”; the English meaning is “blind selection method, controlled”. A further example appears in the Ethical Code of the Russian Medical Association,43 which mentions “Mass surveys of epidemiological character”, but no other types of epidemiological studies. These examples indicate that epidemiology not only is defined officially as an infectious disease discipline, but also that the field is imperfectly understood in Russian medicine.
Because there are no specialists in epidemiology, Russian physicians receive no training in epidemiological methods. The difficulty of accessing the literature and the misrepresentation of epidemiology in publications contributes to the low proportion of longitudinal studies and the high proportion of uncontrolled studies reported here. In the absence of funding for costly types of research such as clinical trials and cohort studies, Russian scientists tend to produce descriptive and retrospective studies. When Russian scientists do controlled clinical trials, reports from these trials are almost exclusively positive,20 which means that, not only that negative results are published less frequently (publication bias), but probably the results of trials are biased. Widespread wrong designs of studies and prevalent bias in statistical analyses led some scholars to the conclusion that there is a methodological crisis in Russian medical research.31 All these results and discussion are true for other countries of the former USSR.
The review of cancer epidemiology led Boyle17 to the conclusion that “epidemiology is alive and well in central and Eastern Europe, where it has managed to continue to function under extremely difficult conditions”. Now, after this evaluation of the epidemiology in USSR and Russia using the systematic analysis of the design of research, teaching and practice of epidemiology, I regret that epidemiology in Russia is more dead than alive. There is, however, a foundation for the growth and development of this important field. The recognition of gaps and inadequacies is perhaps the first essential step toward reaching that goal.
The author aknowledges the help of Dr Barrie R Cassileth in reviewing and commenting on the manuscript.
Conflicts of interest: none.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.