Using multiple data features improved the validity of osteoporosis case ascertainment from administrative databases

Abstract

Objectives

The aim was to construct and validate algorithms for osteoporosis case ascertainment from administrative databases and to estimate the population prevalence of osteoporosis for these algorithms.

Study Design and Setting

Artificial neural networks, classification trees, and logistic regression were applied to hospital, physician, and pharmacy data from Manitoba, Canada. Discriminative performance and calibration (i.e., error) were compared for algorithms defined from different sets of diagnosis, prescription drug, comorbidity, and demographic variables. Algorithms were validated against a regional bone mineral density testing program.

Results

Discriminative performance and calibration were poorer and sensitivity was generally lower for algorithms based on diagnosis codes alone than for algorithms based on an expanded set of data features that included osteoporosis prescriptions and age. Validation measures were similar for neural networks and classification trees, but prevalence estimates were lower for the former model.

Conclusion

Multiple features of administrative data generally resulted in improved sensitivity of osteoporosis case-detection algorithm without loss of specificity. However, prevalence estimates using an expanded set of features were still slightly lower than estimates from a population-based study with primary data collection. The classification methods developed in this study can be extended to other chronic diseases for which there may be multiple markers in administrative data.

Introduction

Osteoporosis is a disease characterized by decreased bone mass and increased fracture risk. It represents a significant population health issue because of its known negative effects on quality of life and other health outcomes, and also because of its increasing prevalence due to an aging population [1], [2], [3], [4]. The operational definition of osteoporosis developed by the World Health Organization (WHO) is based on the measurement of bone mineral density (BMD), which is not easily captured on a population-wide basis [5]. Cohort studies involving primary data collection have been used to estimate osteoporosis prevalence [6], [7], but such studies are expensive and time-consuming to conduct. Population-based administrative databases are increasingly being explored for their potential to provide epidemiological information about osteoporosis [3], [4], [8], [9], [10], [11]. The advantages of using administrative data include: (a) it is relatively inexpensive to establish and maintain a population-based surveillance system using these databases, (b) longitudinal studies can be conducted using data linkage techniques, and (c) disease cases and noncases can be compared on comorbid conditions.

Osteoporosis and fracture diagnosis codes have been used in previous studies to ascertain disease cases in administrative databases. Administrative data have been used to construct case ascertainment algorithms for a number of other chronic diseases including diabetes, asthma, and inflammatory bowel disease [12], [13], [14], [15], [16], [17], [18], [19]. However, osteoporosis is known to be underdiagnosed in administrative databases [8], [9]. When diagnosis codes are the sole source of case ascertainment, this results in an underestimation of disease prevalence. Underdiagnosis is not unique to osteoporosis; incomplete capture of diagnoses in administrative data has been documented for other chronic diseases, such as hypertension [20].

One solution is to develop a case ascertainment algorithm that does not rely exclusively on diagnosis codes. In many jurisdictions, population-based administrative data repositories are expanding in size and scope. As a result, there are multiple data features that could potentially be used for case ascertainment. For example, osteoporosis drug treatments have been proposed for case ascertainment in pharmacy databases [10], [21].

Statistical and machine-learning classification models have been used in fields such as psychology, marketing, and engineering, to identify the data features that distinguish among population subgroups and to predict the probability of group membership [22], [23], [24], [25], [26], [27]. Classification is a process whereby an algorithm, or rule, that assigns observations to groups is developed from multiple data features in a training data set. Once trained, the classifier can take new, unseen observations and predict their probability of group membership.

The purpose of this study is to compare the validity of algorithms for osteoporosis case ascertainment defined from various features of hospital, physician, and pharmacy administrative data. The algorithms are constructed using classification models, and validity is assessed using data from a regional BMD testing program. The algorithms are applied to administrative data to estimate the population prevalence of osteoporosis.