Reliability and validity of a comprehensive health status measure in a heterogeneous population of children admitted to intensive care

doi:10.1016/0895-4356(95)00528-5

Journal of Clinical Epidemiology

Volume 49, Issue 3, March 1996, Pages 327-333

https://doi.org/10.1016/0895-4356(95)00528-5 Get rights and content

Abstract

This study assesses psychometric and empirical characteristics of the Multi-Attribute Health Status Classification (MAHSC) in a heterogeneous population of children admitted to intensive care. The MAHSC encompasses six health domains (sensation, mobility, emotion, cognition, self-care, and pain), each with four or five hierarchic levels of dysfunction.

The health status of 254 consecutively admitted children was determined independently by 3 different observers (parents, attending clinicians, and investigators). The proportion of children with health impairment varied between 13% (in sensation) and 58.7% (in mobility). Interrater reliability of domain scores, as analyzed by Spearman's rank, Pearson product moment, and intraclass correlations, was high, generally exceeding 0.80 for all pairs of observers. Intraclass correlation appeared to be highest in nonsurgical patients (range, 0.89–0.98) and lowest in cardiovascular surgery patients (range, 0.50–0.84). The validity of the classification was supported by (1) the presence of expected specific health status impairment in patients suffering from diseases with an acknowledged impact on specific domains and (2) the presence of a relation between health impairment and medical consumption. The absence of a relation between the average (chronic) health status preceding admission and the acute risk of mortality precipitating the admission confirmed their independent prominence in outcome assessment. We conclude that the MAHSC is a feasible, reliable, and valid measure for outcome assessment in a heterogeneous population of children within a demanding clinical situation. Excellent interrater reliability allows the use of various raters, adjusted to practical requirements.

References (39)

WA Knaus et al.
The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults
Chest
(1991)
DH Fiser
Assessing the outcome of pediatric intensive care
J Pediatr
(1992)
S Saigal et al.
Comparison of the health-related quality of life of extremely low birth weight children and a reference group of children at age eight years
J Pediatr
(1994)
F Guillemin et al.
Cross-cultural adaptation of health related quality of life measures: Literature review and proposed guidelines
J Clin Epidemiol
(1993)
MAG Sprangers et al.
The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: A review
J Clin Epidemiol
(1992)
RJBJ Gemke et al.
Assessing the outcome of pediatric intensive care
J Pediatr
(1993)
TM Achenbach et al.
Epidemiological comparisons of American and Dutch children. I. Behavioral/emotional problems and competencies reported by parents for ages 4 to 16
J Am Acad Child Adolesc Psychiatry
(1987)
S Lemeshow et al.
Refining intensive care unit outcome prediction by using changing probabilities of mortality
Crit Care Med
(1988)
MM Pollack et al.
Accurate prediction of the outcome of pediatric intensive care
N Engl J Med
(1987)
MM Pollack et al.
Improved outcomes from tertiary center pediatric intensive care: A statewide comparison of tertiary and nontertiary care facilities
Crit Care Med
(1991)

X Castella et al.

Mortality prediction models in intensive care: Acute physiology and chronic health evaluation II and mortality prediction model compared

Crit Care Med

(1991)

Z Yinnon et al.

Quality of life and survival following intensive medical care

Q Med J

(1989)

RG Hulsebos et al.

Measuring quality of life with the sickness impact profile: A pilot study

Intens Care Med

(1991)

GV Mata et al.

Factors related to quality of life 12 months after discharge from the intensive care unit

Crit Care Med

(1992)

NL Glass et al.

Pediatric intensive care: Who, why and how much

Crit Care Med

(1986)

W Butt et al.

Long term outcome of children after intensive care

Crit Care Med

(1990)

I McDowell et al.

C Jenkinson et al.

Short Form 36 (SF 36) health survey questionnaire: Normative data for adults of working age

Br Med J

(1984)

D Feeny et al.

A comprehensive multiattribute system for classifying the health status of survivors of childhood cancer

J Clin Oncol

(1992)

Cited by (72)

Systematic review of quality of life and other patient-centred outcomes after cardiac arrest survival
2011, Resuscitation
Citation Excerpt :
There are several validated tools available that assess generic health related quality of life. The ‘Health Utilities Index 3’ has been shown to be reliable, reproducible and valid in a variety of patient populations.42–44 It is quick to use and can be used to interview family members/caregivers when patients are not able to respond, which prevents those patients with the poorest functional/cognitive outcome being excluded from these studies; others have questioned the validity of using a proxy in this type of assessment.45
In cardiac arrest patients (in hospital and pre hospital) does resuscitation produce a good Quality of Life (QoL) for survivors after discharge from the hospital?
Embase, Medline, The Cochrane Database of Systematic Reviews, Academic Search Premier, the Central Database of Controlled Trials and the American Heart Association (AHA) Resuscitation Endnote Library were searched using the terms (‘Cardiac Arrest’ (Mesh) OR ‘Cardiopulmonary Resuscitation’ (Mesh) OR ‘Heart Arrest’ (Mesh)) AND (‘Outcomes’ OR ‘Quality of Life’ OR ‘Depression’ OR ‘Post-traumatic Stress Disorder’ OR ‘Anxiety OR ‘Cognitive Function’ OR ‘Participation’ OR ‘Social Function’ OR ‘Health Utilities Index’ OR ‘SF-36’ OR ‘EQ-5D’ as text term.
There were 9 inception (prospective) cohort studies (LOE P1), 3 follow up of untreated control groups in randomised control trials (LOE P2), 11 retrospective cohort studies (LOE P3) and 47 case series (LOE P4). 46 of the studies were supportive with respect to the search question, 17 neutral and 7 negative.
The majority of studies concluded that QoL after cardiac arrest is good. This review demonstrated a remarkable heterogeneity of methodology amongst studies assessing QoL in cardiac arrest survivors. There is a requirement for consensus development with regard to quality of life and patient centred outcome assessment in this population.
Comparison of the Cerebral Performance Category Score and the Health Utilities Index for Survivors of Cardiac Arrest
2009, Annals of Emergency Medicine
Citation Excerpt :
This scoring method was derived from measurement of utilities, using the standard gamble in a large sample of the public.21 The Health Utilities Index has been shown to be both reliable and valid in other populations such as patients hospitalized in a pediatric ICU, survivors of hematologic malignancy, and in the general population.25-27 The second test, evaluated by the same study nurse, was the Cerebral Performance Category scale, a simple 5-point measurement of cerebral and functional status that ranges from category 1 to indicate good performance and category 5 to indicate brain death (Figure 1).28
The Cerebral Performance Category score is an easy to use but unvalidated measure of functional outcome after cardiac arrest. We evaluate the comparability of results from the Cerebral Performance Category scale versus those of the validated but more complex Health Utilities Index scale for health-related quality of life.
This prospective substudy of the Ontario Prehospital Advanced Life Support (OPALS) Study included adult out-of-hospital cardiac arrest patients treated in 20 cities. This prospective cohort study included all survivors of out-of-hospital adult cardiac arrest enrolled in phase II (rapid basic life support with defibrillation) and phase III (advanced life support) of the OPALS Study, as well as the intervening run-in phase. Survivors were interviewed at 12 months for Cerebral Performance Category Score and the Health Utilities Index Mark 3 (Health Utilities Index).
Of 8,196 eligible out-of-hospital cardiac arrest patients between 1995 and 2002, 418 (5.1%) survived to discharge, and 305 (3.7%) completed the Health Utilities Index interview and had Cerebral Performance Category scored at 12 months. The 305 patients had the following data: mean age 63.9 years; male 78.0%; paramedic-witnessed arrest 25.6%; bystander cardiopulmonary resuscitation 32.1%; initial rhythm ventricular fibrillation/ventricular tachycardia 86.9%, Cerebral Performance Category 1 267, Cerebral Performance Category 2 26, Cerebral Performance Category 3 12. Overall, the median scores (interquartile range) were Cerebral Performance Category 1 (1 to 1) and Health Utilities Index 0.84 (0.61 to 0.97). The Cerebral Performance Category score ruled out good quality of life (Health Utilities Index >0.80), with a sensitivity of 100% (95% confidence interval [CI] 98% to 100%) and specificity 27.1% (95% CI 20% to 35%); thus, when the Cerebral Performance Category score was 2 or 3, it was unlikely that the Health Utilities Index score would be good. The Cerebral Performance Category score had sensitivity 55.6% (95% CI 42% to 67%) and specificity 96.8% (95% CI 94% to 98%) for predicting poor quality of life (Health Utilities Index >0.40); ie, when Cerebral Performance Category was 1, it was highly unlikely that the Health Utilities Index score would be poor. The weighted κ was 0.39 and the interclass correlation was 0.51.
This represents the largest study yet conducted of the performance of the Cerebral Performance Category score in 1-year survivors of out-of-hospital cardiac arrest. Overall, the Cerebral Performance Category score classified patients well for their quality of life, ruling out a good Health Utilities Index score with high sensitivity and ruling in poor Health Utilities Index score with high specificity. The Cerebral Performance Category is an important tool in that it indicates broad functional outcome categories that are useful for a number of key clinical and research applications but should not be considered a substitute for the Health Utilities Index.
Resuscitation Outcomes Consortium (ROC) PRIMED cardiac arrest trial methods. Part 2: Rationale and methodology for "Analyze Later vs. Analyze Early" protocol
2008, Resuscitation
The primary objective of the trial is to compare survival to hospital discharge with modified Rankin score (MRS) ≤3 between a strategy that prioritizes a specified period of CPR before rhythm analysis (Analyze Later) versus a strategy of minimal CPR followed by early rhythm analysis (Analyze Early) in patients with out-of-hospital cardiac arrest.
Design—Cluster randomized trial with cluster units defined by geographic region, or monitor/defibrillator machine.
Population—Adults treated by emergency medical service (EMS) providers for non-traumatic out-of-hospital cardiac arrest not witnessed by EMS.
Setting—EMS systems participating in the Resuscitation Outcomes Consortium and agreeing to cluster randomization to the Analyze Later versus Analyze Early intervention in a crossover fashion.
Sample size—Based on a two-sided significance level of 0.05, a maximum of 13,239 evaluable patients will allow statistical power of 0.996 to detect a hypothesized improvement in the probability of survival to discharge with MRS ≤3 rate from 5.41% after Analyze Early to 7.45% after Analyze Later (2.04% absolute increase in primary outcome).
If this trial demonstrates a significant improvement in survival with a strategy of Analyze Later, it is estimated that 4000 premature deaths from cardiac arrest would be averted annually in North America alone.
What are the costs of childhood-onset rheumatic disease?
2006, Best Practice and Research: Clinical Rheumatology
Citation Excerpt :
The HUI and EQ-5D are the preference-weighted questionnaires that are used most commonly to measure HRQoL in paediatric populations. The construct validity of both has been demonstrated in paediatric populations, including children with musculoskeletal diseases.63–66 However, the discriminant validity of the EQ-5D was found to be poor, and the construct of HUI was found to be different from that of other preference-based measures (SG or rating scales).
Juvenile rheumatic diseases have important impacts on health, i.e. on patients' body functions and structures, activities, and social participation. The identification and treatment of these disorders is costly. Treatment involves periods of hospitalization, the use of physicians and other professional services, drugs and other treatments. Frequent outpatient visits are needed, as may be surgery. This all imposes a large financial burden on health-care systems and on families of children who suffer from rheumatic illnesses. Costs borne by families are both out-of-pocket as well as related to time spent in providing care, which may involve loss of income. Of course, the burdens are not only monetary, and they sometimes continue for life. The measurement of the different types of cost is essential to get a full picture of the burden of childhood-onset rheumatic illnesses. This chapter presents data on the costs of these illnesses, and introduces methodologies and their limitations for cost evaluation within paediatric rheumatology.
Systematic Review of the Psychometric Performance of Generic Childhood Multi-attribute Utility Instruments
2023, Applied Health Economics and Health Policy
Health-related quality of life outcome measures for children surviving critical care: a scoping review
2021, Quality of Life Research

View all citing articles on Scopus

^☆: This study was supported by Grant 92-56A from the Dutch Ministry of Welfare, Health Care and Culture.

View full text

Original articleReliability and validity of a comprehensive health status measure in a heterogeneous population of children admitted to intensive care☆

Abstract

Chest

J Pediatr

J Pediatr

J Clin Epidemiol

J Clin Epidemiol

J Pediatr

J Am Acad Child Adolesc Psychiatry

Refining intensive care unit outcome prediction by using changing probabilities of mortality

Crit Care Med

Accurate prediction of the outcome of pediatric intensive care

N Engl J Med

Improved outcomes from tertiary center pediatric intensive care: A statewide comparison of tertiary and nontertiary care facilities

Crit Care Med

Mortality prediction models in intensive care: Acute physiology and chronic health evaluation II and mortality prediction model compared

Crit Care Med

Quality of life and survival following intensive medical care

Q Med J

Measuring quality of life with the sickness impact profile: A pilot study

Intens Care Med

Factors related to quality of life 12 months after discharge from the intensive care unit

Crit Care Med

Pediatric intensive care: Who, why and how much

Crit Care Med

Long term outcome of children after intensive care

Crit Care Med

Short Form 36 (SF 36) health survey questionnaire: Normative data for adults of working age

Br Med J

A comprehensive multiattribute system for classifying the health status of survivors of childhood cancer

J Clin Oncol

Original article
Reliability and validity of a comprehensive health status measure in a heterogeneous population of children admitted to intensive care☆