Original Article
Hospital chart review provided more accurate comorbidity information than data from a general practitioner survey or an administrative database

https://doi.org/10.1016/j.jclinepi.2004.03.016Get rights and content

Abstract

Background and objective

The accuracy of comorbidity data within the Western Australian Data Linkage System was evaluated by means of comparison with hospital charts and a general practitioner (GP) survey.

Methods

Patients (n = 2,037) with a hospital admission from 1991 to 1996 were selected. Linked data were extracted for 100 comorbidities, categorized into 16 diagnostic chapters, for each hospital admission within a 5-year period. Clinical chart review and a GP survey were performed. Comorbidity occurrence in each data source and false-positive and false-negative diagnoses were ascertained.

Results

Administrative data contained 45.5% of comorbidity recorded in hospital charts and underascertained secondary conditions for all 16 diagnostic chapters. False-positive diagnoses were low for most conditions (range: 0–1.5%); however, a high occurrence of false negatives existed for all comorbidity chapters (range: 16.3–91.3%). GP-identified comorbidity was 20.0% greater than that found using administrative data but, with the exceptions of injury–poisoning and cutaneous–subcutaneous diseases, was less (42.0%) than that observed from hospital charts.

Conclusion

Our results indicate that when accurate comorbidity data are crucial to health outcome research, hospital chart review (as opposed to using administrative data) may be required. Furthermore, surveying GPs, at least in Australia, appears an unsatisfactory alternative to hospital charts for obtaining retrospective comorbidity information.

Introduction

The linkage of medical records provides a potentially powerful tool for disease surveillance, evaluating health care outcomes and monitoring health service utilization [1], [2], [3], [4], [5]. Currently, there are at least six comprehensive population-based medical record linkage systems operating around the world that routinely link administrative health data [3], [6]. One of these is the Western Australian (WA) Data Linkage System, which includes links between seven core data sets: birth registrations, death registrations, hospital morbidity data, mental health records, midwives' notifications, cancer notifications, and electoral registrations variously dating back as early as 1966 to the present [3].

With the widespread use of record linkage systems, and other health-related administrative databases, there is a need to validate the quality of data they contain. Such validation assumes further importance when considering that research findings derived from these systems often form the basis of health care reforms [2], [7]. A number of validation studies have assessed the quality of data in administrative databases, including hospital morbidity data systems (HMDSs), with results suggesting that data on primary diagnoses and procedures are seldom missing and are accurately coded in more than 95% of instances [8], [9], [10], [11]. Within the WA Data Linkage System, a validation study of end-stage renal failure patients found that the administrative data pertaining to procedures such as renal dialysis and transplant were accurate in 97% of cases [6].

Although the quality of primary diagnosis data appears adequate, the accuracy of secondary diagnoses is less certain. This may pose problems for the numerous epidemiological investigations that use linked administrative data to adjust analyses for potential confounding produced by the presence of certain comorbid conditions [12], [13]. Since the development of the Charlson Comorbidity Index [14], it and similar indices of comorbidity have routinely been used for statistical adjustment by studies focusing on morbidity outcomes and hospital readmission [1], [15], [16], [17].

Clinical chart review, although more labor intensive and costly than the use of linked databases, has been shown to be a valid source for identifying primary and secondary conditions and has been suggested to be the best known available method for obtaining patient medical diagnosis data [18], [19], [20]. Moreover, data from patient interviews have been reported to correlate highly with measures of comorbidity based on medical records [19]. It is possible also that survey of patients' family physicians (known in Australia as general practitioners, or GPs) would also provide quality information on current and previous medical conditions.

In the present study we evaluated the accuracy of comorbidity data contained in the HMDS of the WA Data Linkage System by comparison with manual hospital chart review and survey of patients' GPs.

Section snippets

Selection of study comorbidities

One hundred comorbid conditions, categorized into 16 diagnostic chapters, were selected on the basis of their frequency of occurrence and potential to effect mortality or morbidity outcomes. The conditions were classified using the ninth revision of the International Classification of Diseases with Clinical Modifications (ICD-9-CM). Comorbidity diagnostic chapters and individual conditions comprising them are given in Table 1, Table 2, respectively. Corresponding ICD-9-CM codes for each of the

Comparison of administrative data and hospital charts

The occurrence (1986–1996) of comorbid conditions grouped under the 16 diagnostic chapters as recorded in the WA Data Linkage System and in the patients' hospital charts is given in Table 1. The number of comorbid conditions in HMDS records was underascertained in all diagnostic chapters, compared with hospital charts. In total, the frequency of recorded comorbidity in the administrative database accounted for only 45.5% of that found in hospital charts (range: 8.7–83.8% of hospital charts).

Discussion

The results indicate that, with few exceptions, large discrepancies existed in recorded comorbidity between hospital charts and the HMDS of the WA Data Linkage System for hospital admissions from 1986 to 1996. Examination of the individual comorbidities indicated that 83 of the selected 100 conditions were underascertained in administrative data compared with hospital charts. In addition, false-negative comorbidity diagnosis was high for all 16 comorbidity diagnostic chapters for those patients

Conclusions

Our conclusion is that comorbidity information in the WA Data Linkage System, preceding 1996, is poorly recorded compared with that contained in hospital charts, suggesting an inadequate transfer of comorbidity data from hospital to administrative database. To gain more accurate comorbidity data for research purposes, hospital chart review is required. Further, surveying GPs is an unsatisfactory alternative for obtaining quality retrospective information relating to patient comorbidity. Future

Acknowledgments

The initial construction of the Western Australian Data Linkage System was funded by the Western Australian Lotteries Commission. This study was supported by a project grant from the National Health and Medical Research Council. We thank the hospitals involved in the study for their assistance with hospital chart review.

References (34)

  • W. D'Hoore et al.

    Practical considerations on the use of the Charlson Comorbidity Index with administrative databases

    J Clin Epidemiol

    (1996)
  • D.J. Malenka et al.

    Using administrative data to describe casemix: a comparison with the medical record

    J Clin Epidemiol

    (1994)
  • P.S. Romano et al.

    Ischemic Heart Disease Patient Outcomes Research Team. A comparison of administrative versus clinical data: coronary artery bypass surgery as an example

    J Clin Epidemiol

    (1994)
  • R.L. O'Connell et al.

    Utility of Charlson Comorbidity Index computed from routinely collected hospital discharge diagnosis codes

    Method Inf Med

    (2000)
  • J.B. Semmens et al.

    The Quality of Surgical Care Project: a model to evaluate surgical outcomes in Western Australia using population-based record linkage

    Aust N Z J Surg

    (1998)
  • F.J. Stanley et al.

    A population database for maternal and child health research in Western Australia using record linkage

    Paediatr Perinat Epidemiol

    (1994)
  • W.A. Ray

    Policy and program analysis using administrative databases

    Ann Intern Med

    (1997)
  • Cited by (102)

    • Hysterectomy with and without oophorectomy and all-cause and cause-specific mortality

      2020, American Journal of Obstetrics and Gynecology
      Citation Excerpt :

      We conducted a retrospective cohort study using deidentified linked administrative data provided by the Western Australian Data Linkage Branch (WADLB). WADLB continually monitors the quality of data,15 and validation studies of hospital morbidity data have found that primary diagnosis and procedure codes have an accuracy of more than 95%.16 The cohort included all women who were on the WA electoral roll on Oct. 1, 1988, and those who joined the electoral roll up until Dec. 31, 2014 (n=968,352).

    View all citing articles on Scopus
    View full text