Article Text

PDF

Performance profile of an outcome measure: morbidity assessment index for newborns
  1. Anila Verma1,
  2. Angela Weir2,
  3. Jane Drummond2,
  4. Bryan F Mitchell1
  1. 1Department of Obstetrics and Gynaecology, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
  2. 2Faculty of Nursing, University of Alberta
  1. Correspondence to:
 Dr A Verma
 Clinical Safety and Epidemiology, Bldg 419/1150, One Health Plaza, East Hanover, Novartis Pharmaceuticals, New Jersey, USA 07936; anila.vermapharma.novartis.com

Abstract

Objective: Few tools have been optimised for use over the entire spectrum of neonatal morbidity and standardised for use in perinatal population and community health studies. The objective of this study was to determine the performance profile of the recently developed morbidity assessment index for newborns (MAIN score). This score was designed as a discriminative index of morbidity for the entire population of babies delivered >28 weeks gestation without a major congenital anomaly.

Design and setting: MAIN score items were extracted retrospectively from the health records of 2892 consecutively born babies delivered beyond 28 weeks gestation in Edmonton area hospitals between June and December of 1999.

Main results: The mean MAIN score in the general newborn population was 70.3 (95% confidence intervals 64.2 to 76.4). With the MAIN score tool, 84.6% of newborns scored from 0 to 150 (no/minimal morbidity), 11.3% from 151 to 500 (mild), 3.1% from 501 to 800 (moderate), and 1% had >800 (severe) score. The MAIN score tool was sufficiently sensitive to detect significant effects of low gestational age, low birth weight, male sex, caesarean delivery, tertiary hospital delivery, twins/triplets, non-vertex presentation, prenatal illicit drug use, and medical complications of pregnancy.

Conclusion: The MAIN score fulfills the need for a simple, universal, yet sensitive and robust tool to provide a numerical index of early neonatal outcomes of prenatal care and adverse prenatal exposures in babies delivered beyond 28 weeks gestation. The performance of the MAIN score agrees well with the current medical awareness regarding the impact of adverse prenatal exposures on newborn morbidity.

  • BW, birth weight
  • GA, gestational age
  • MAIN, morbidity assessment index for newborns
  • newborn morbidity
  • high risk pregnancy
  • perinatal epidemiology
  • maternal child health
  • outcome measurement

Statistics from Altmetric.com

Few tools of newborn morbidity have been optimised and standardised for use over the entire spectrum, from minimal to severe, for use in perinatal population studies of early neonatal period. Traditionally, indicators such as perinatal or infant mortality, birth weight, or gestational age have been used as outcome measures of effectiveness of national health policies and interventions for pregnant women. While mortality rates still have some value in underdeveloped countries, they are of diminishing significance for countries with highly developed healthcare systems. In the absence of valid measures of newborn morbidity, low birth weight (BW), low gestational age (GA), or Apgar score are considered to be proxy measures to report morbidity at birth.1–3 However, these proxy measures are insufficiently sensitive or precise to provide the information necessary to shape current maternal-child healthcare policy. The Apgar score was originally designed as a discriminative index to identify infants in need of immediate cardiopulmonary resuscitation and performs this function well. However, it is a poor index of a newborn’s health beyond the first few hours of life.4,5 A joint statement published by the American Association of Pediatricians with the American College of Obstetricians and Gynecologists emphasises that a low Apgar score has little meaning unless it occurs in combination with profound metabolic or mixed acidaemia (pH<7) and neurological manifestations, for example, seizure, coma, or hypotonia to diagnose neurological injury at birth.6

Several specific measurement tools have been developed in the past to assess morbidity in very low birthweight (<1500 g) or low GA babies.7–9 Such babies constitute <2% of all births and morbidity in this highly selected group does not emulate the morbidity pattern of most of the newborn population. Perinatal epidemiologists and community health researchers need a standardised, valid tool for assessing general newborn health across the entire spectrum of morbidity at birth. Such measures should not only be reliable, universal, and commonly found in the study population but should also be sufficiently sensitive to warrant concern when there is a positive finding.

Recently, we developed and validated the structure of the morbidity assessment index for newborns (MAIN score). The details of the design strategy of the MAIN score have been published previously.10–12 This score was designed specifically for babies born without major congenital anomalies at ⩾28 weeks’ gestation. This population includes about 97% of all newborns. The MAIN score is based on items of routine clinical and laboratory examination of newborns. It was designed to be simple and easily completed by retrospective chart review. Its purpose is to reflect morbidity in the first week of life. The specific objective of the design of the MAIN score was to provide a tool to assist population health researchers in comparing healthcare delivery systems available for pregnant women. The MAIN score could also serve as a baseline assessment for subsequent studies of developmental outcomes in infants.

The specific aim of this study was to examine the performance profile of MAIN score in the entire newborn population in three community hospitals in the city of Edmonton, Canada. We hypothesised that, if the MAIN score is a sensitive and discriminative measure of newborn morbidity, there would be significantly higher scores in newborns from pregnancies complicated by a variety of adverse prenatal exposures and medical conditions. Therefore, after the initial analyses, the population of newborns was stratified according to factors known to influence newborn health (for example, GA, BW, singletons/multiple births, malpresentation, maternal medical complications, prenatal adverse exposures, etc). These analyses have provided evidence for validity of the MAIN score as a discriminative tool of newborn morbidity.

METHODS

The Health Research Ethics Board of University of Alberta approved this study for data collection by chart review at three Edmonton hospitals: Royal Alexandra (RAH), Grey Nuns (GNH), and Misericordia (MH). In addition to a community maternal-child health service, the RAH is the regional tertiary care facility for high risk deliveries. The GNH and MH are community hospitals. Data on the MAIN score morbidity items were obtained from the health records of 2892 newborns delivered at the three hospitals from June to December 1999, born at ⩾28 weeks’ gestation with no major life threatening anomaly at birth. The MAIN score for each recruited newborn was calculated by retrospective review of the hospital chart. Relevant maternal data (see later) were collected retrospectively from the chart of each newborn’s mother.

For designing the MAIN score, innovative item response theory (IRT)13,14 and dimensionality testing techniques15 were used for structural validation and item analysis, as published previously.10–12 Structural validation techniques helped evaluate relation between the morbidity items and the underlying construct of newborn morbidity by determining the goodness of fit of items included in the MAIN score inventory. Item analysis provided scale values or weights that were assigned to each morbidity item contained in the MAIN score inventory. The scale values are relative values that provide weighting to the item(s) reflecting its contribution to the overall morbidity score. The statistical parameters of goodness of fit, including χ2 statistics, discrimination statistics (shown in table 1), and item characteristic curves (ICC) were also obtained from the item analysis model, using BILOG-W software (BILOG, Scientific Software, Chicago, IL, 2nd edition, 1990).

Table 1

 MAIN score morbidity items with their scale values and goodness of fit parameters

The χ2 statistic reflects fitness of observed on predicted test score distributions. The discrimination statistic is an indicator of discriminative capability of an item to capture various grades of morbidity on the morbidity continuum. The discrimination statistic in conjunction with the ICCs provided evidence regarding information function of an individual item and the goodness of fit of items in the MAIN score. In brief, after a number of iterations and re-evaluations of goodness of fit of different combinations of items, an inventory containing 47 binary items representing 24 attributes of early neonatal morbidity was specified as the best fit for the MAIN score. Table 1 contains the scale values, χ2, and discrimination statistics of the morbidity items estimated using BILOG-W software for a two parameter IRT model. The MAIN score for a newborn is computed by aggregating the scale values of all items selected in the MAIN score inventory (table 1) from birth until seven days of life or discharge from the hospital, whichever is earlier. For example, a newborn having transient morbidity attributable to apnea corrected by oxygen (item 18) and respiratory rate >60/min between 3 to 24 hours of life (item 10) will receive a MAIN score of 246; whereas, a newborn with an Apgar score of 6 at five minutes (item 4) requiring resuscitation with intubation (item 2), who had a single seizure (item 24), in addition to hypotonia (item 15) and poor sucking for the first three days after birth (item 30) will receive a MAIN score of 616.

The review of charts of many newborns showed no indication or suspicion of morbidity during the early neonatal period and therefore, no assignment of morbidity according to items in the MAIN score. These infants were assigned a MAIN score of “zero”. We have assumed that such newborns were deemed healthy and in no need for elaborate testing (for example, bacterial culture or urine output assessment) at birth before discharge from the hospital.

Sample size considerations

Sample size calculations were based on the probability of occurrence of 47 morbidity items contained in the MAIN score. Measurement theories advise that to provide stable parameter estimates for the measurement tool, at least five participants per item need to be recruited.16 Considering that the prevalence of uncommon morbidity items may be as low as 0.2% (for example for peripheral nerve injury), we calculated that a sample of 2500 newborns was required for providing stable parameter estimates of morbidity items in the MAIN score. A total of 2892 eligible newborns were recruited to this study.

Newborn data collection

Data were collected for 2892 infants. Of these, 1405 (48.5%) newborns were recruited from the RAH, 875 (30.3%) from the GNH, and 606 (21%) from the MH. Depending on the complexity of morbidity, it takes from 2 to 15 minutes per chart to extract data for the MAIN score from hospital records. Customised electronic forms were prepared using Microsoft Access (version 5.0) for data entry and computation of the MAIN score. Three research assistants were trained using 20 charts each to extract morbidity data from the health records. Inter-rater reliability among the three data abstractors was tested on other sets of 20 charts for each pair—that is, a total of 40 charts. Inter-rater reliability between two raters in each pair was greater than 98%.

Maternal data collection

In all three hospitals there is a standardised nursing assessment of maternal risk indicators for both antepartum and intrapartrum periods. The antepartum risk score assessed the number and severity of any maternal conditions that may adversely affect the pregnancy outcome for either mother or newborn. The risk items included pre-pregnancy maternal disease(s), history of obstetrical complications, and maternal complications associated with the current pregnancy. A pregnancy was considered low risk if the antepartum risk score was ⩽2, moderate risk if 3–5, high risk if 6–10, and extreme risk if >10. The most common maternal complications were defined by the following criteria:

Antepartum haemorrhage: bleeding from the genital tract after 28 weeks of gestation.

Intrauterine growth restriction: ultrasound predicted BW <3rd centile using Canadian standards for gestational age.

Hypertension: chronic or pregnancy related hypertension: systolic blood pressure (BP) >140 mm Hg or diastolic BP >90 mm Hg on ⩾2 occasions, recorded at least six hours apart.

Pre-eclamptic toxaemia: pregnancy related hypertension plus proteinuria >1+ on a clean-catch urine specimen or when the total urinary protein is greater than 300 mg during a 24 hour period in a woman without a urinary tract infection.

Diabetes mellitus: insulin dependent diabetes mellitus before pregnancy.

Gestational diabetes mellitus: two or more values exceeding the recommended maternal blood glucose concentration criteria after a two hour 100 g oral glucose tolerance test.

Group B streptococcus: a positive culture growth of Streptococcus agalactiae upon routine screening at 35–36 weeks’ gestation by lower vaginal/perianal swab.

Malpresentation: any obstetrical presentation other than cephalic.

Smoking was recorded as no smoking, <10 cigarettes/day, or >10 cigarettes/day.

The assessment form also showed the use of alcohol and marijuana (termed recreational drug use) or street drug use (including misuse of cocaine, narcotics, amphetamines, inhalation of fumes, etc) There was no distinction as to the quantity or frequency of drug ingestion in drug misusers.

The intrapartrum risk score assessment included any of the above complications that onset during labour or present as labour related complications. This score also included assessment of gestational age, prolonged rupture of fetal membranes (>24 hours), maternal fever, fetal heart rate abnormalities, etc. An intrapartum score of 0 was considered no risk and a score of ⩾5 extreme risk.

Missing data

Data on the hospital of delivery and the type of delivery were missing from six maternal records; maternal smoking status was absent from 26 records; data regarding drug misuse, diabetes mellitus, and pre-eclamptic toxaemia were missing from 15 records and on presentation at delivery from eight records. The number of newborns thus available for various different analyses is shown in the respective tables. The hospital charts of all infants were available. As noted earlier, we have assumed that all abnormal examinations and tests were recorded on the chart. In the absence of checked items on the MAIN score inventory, we have considered that infant completely healthy with a MAIN score of zero. Thus, there were no missing data for calculation of the MAIN score.

Statistical analyses

Analyses were performed using the SPSS 10.0 version (IL, Chicago). To facilitate analysis and to improve the clinical application of MAIN score tool, it was elected to divide the MAIN scores into four subgroups according to their relative severity of morbidity. After consultation with the obstetrical and paediatric consultant groups from the three hospitals, morbidity scores were subgrouped empirically into: (1) none/minimal, score 0–150; (2) mild, 151–500; (3) moderate, 501–800; or (4) severe, >800.

Bivariate analytical techniques were used to test differences in the MAIN score among various specific newborn and maternal populations shown in tables 2 and 3. Linear trend analysis was performed to show the influence of increasing degree of adverse prenatal exposure and obstetrical complications on newborn morbidity.

Table 2

 Distribution of MAIN score in the overall population and in the subgroups stratified by specific maternal or newborn characteristics

Table 3

 Mean MAIN score (95% CI) in selected high risk maternal populations

RESULTS

Data from 2892 newborns (1480 males and 1412 females) delivered at or beyond 28 weeks’ gestation, with no congenital anomaly were used to compute and analyse the MAIN score profile. The average MAIN score in our population was 70.3 with 95% CI 64.2 to 76.4 (table 2). On classifying 2892 newborns into clinically relevant qualitative categories, 84.6% had none/minimal, 11.3% mild, 3.1% moderate, and 1% severe morbidity. Table 2 also shows the distribution of MAIN scores within specific subgroups of newborn and maternal populations. Both the mean morbidity score and the proportion of morbidity were significantly higher in all of the following: males compared with females; twin/triplets compared with singletons; infants delivered by caesarean section compared with vaginal delivery; malpresentation compared with cephalic presentation; tertiary compared with community hospital delivery, infant of smoker compared with non-smoker; or infant of multiple drug misusers compared with recreational/no drug misusers. As shown in table 2, with each increment in GA (except ⩾40 weeks) and BW (except ⩾4500 g), there was significant reduction in newborn morbidity. Among twins and triplets, about half (49%) had no/minimal morbidity at birth but the remaining had significantly higher morbidity over the entire spectrum of morbidity compared with singletons.

Key points

  • The MAIN score fulfils the need for a simple, universal, yet sensitive and robust tool to provide a numerical index of the influence of prenatal care and adverse prenatal exposures on early newborn morbidity. Data presented show that the MAIN score provides fine discrimination across the spectrum of newborn morbidity in the general newborn population.

  • In the public health domain, this tool could assist in community health programming by early identification of infants who may require increased use of community healthcare resources.

Table 3 provides the MAIN scores for selected high risk maternal populations. Except for group B streptococcus and gestational diabetes, the MAIN scores for infants from mothers with other maternal conditions were statistically significantly higher than the overall population. With each successive increase in the degree of adverse prenatal exposure, there was a significant trend towards increase in the MAIN score, for example, from no drug to recreational, street and multiple drug misuse (p = 0.001); from no smoking to smoking ⩽10 to >10 cigarettes/day (p = 0.009); from hypertension to PET (p = 0.001); and from gestational diabetes to pregestational diabetes (p = 0.05), as shown in tables 2 and 3.

In Figure 1 we have plotted the antepartum and intrapartum risk scores for all pregnant women against the mean MAIN scores of their infants. The MAIN scores had a significant linear positive correlation with both antepartum (Pearson’s correlation coefficient, r = 0.35, p = 0.01) and intrapartum risk scores (r = 0.38, p = 0.01).

Figure 1

 Relation between the MAIN scores and antepartum risk score category (A) and intrapartum risk score (B). There was a significant correlation between the MAIN scores and both of these obstetrical risk indicators. The numbers above the histograms show the number of infants in that group.

DISCUSSION

Sound decision making in perinatal epidemiology and policy development is dependent upon the use of readily available, generalisable, and valid outcome measures of morbidity. The primary aim of our study was to describe the performance of and establish normative values for the recently developed MAIN score. This score was developed specifically as a discriminative index of morbidity for newborns born beyond 28 weeks’ gestation with no congenital or metabolic abnormality. This population constitutes about 97% of the total newborn population. The MAIN score captures the full range of morbidity, from minimal to severe. Our data show that the score is sensitive enough to capture subtle differences in morbidity according to GA, BW, sex, and maternal influences. We are not aware of any other measures of newborn morbidity that have illustrated such fine discrimination in evaluating severity of morbidity. This study also confirms that the MAIN score agrees well with the current medical awareness about the impact of adverse prenatal exposures on newborn morbidity.

In contrast with the parameters developed for very low BW (<1500 g) babies to predict developmental outcome, mortality, or severe morbidity,7–9 the MAIN score was designed to function as a simple and universal measure of early neonatal morbidity over the entire spectrum of morbidity. Optimal performance of the MAIN score can be attributed to two major factors in the construction of this tool: (1) a comprehensive inventory of standard assessment items that reflect pathophysiology in early newborn period; and (2) the use of IRT to produce robust yet sensitive scaling units for each individual morbidity item.

We had excluded babies born with major structural anomaly or life threatening birth defect diagnosed at birth; however, we recognise that a small proportion of babies whose defect was not detected at birth (for example, metabolic anomaly or heart defect) may have been included in our dataset. Also, the four morbidity subgroups used to describe the severity of morbidity, from very mild to severe, are merely empirical, which should be substantiated in future studies by involving a larger number of perinatal consultants and with the help of large datasets.

We propose several potential uses for the validated MAIN score tool for policy makers and health administrators. Firstly, the score has utility in comparing the health of newborn populations across regions/countries or within regions after a change in the healthcare delivery system. Secondly, the MAIN score could be used as an outcome measure in research projects investigating the effects of novel interventions in the maternal-child healthcare delivery system. Furthermore, the score could be used to support hypotheses regarding the effects of adverse prenatal exposures on neonatal outcome. An example of this could be the data presented in this manuscript regarding the effects of maternal smoking or drug misuse on newborn’s health.

Policy implications

We propose several potential uses of the validated MAIN score tool for policy makers and health administrators:

  • The score has utility in comparing the health of newborn populations across regions/countries or within regions after a change in the healthcare delivery system. (Who could initiate this change? Public health decision makers and advocacy groups)

  • The MAIN score could also be used as an outcome measure in research projects investigating the effects of novel interventions in the maternal-child healthcare delivery system. (Who could initiate this change? Public health, community health, and clinical researchers, and epidemiologists)

  • The score could be used to support hypotheses regarding the effects of adverse prenatal exposures on neonatal outcome. An example of this could be the data presented in this manuscript regarding the effects of maternal smoking or drug misuse on newborn’s health. (Who could initiate this change? Public health, community health, and clinical researchers, and clinical professionals)

The MAIN score fulfills the need for a simple, universal, yet sensitive and robust tool to provide numerical index of the influence of prenatal care and adverse prenatal exposures on early newborn morbidity. In summary, data presented herein show that the MAIN score provides fine discrimination across the spectrum of newborn morbidity in the general newborn population. In the public health domain, this tool could assist in community health programming by early identification of infants who may require increased use of community healthcare resources. Currently analyses of studies are underway to determine predictive value of MAIN score in health services use by one year of infant’s age.

REFERENCES

View Abstract

Footnotes

  • Funding: this research was supported by funds from three sources: The Alberta Heritage Foundation for Medical Research, record number 199800308; (2) University of Alberta Industry Liaison Office; (3) The University of Alberta Perinatal Research Center, Canada.

  • Conflicts of interest: none declared.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles