Article Text
PDF
Abstract
Background Most of the standard verbal autopsy tools are long and are used in a research setting. This study aims to compare a short verbal autopsy (VA) tool developed at Ballabgarh, India to be used by health workers for routine mortality surveillance with a standard tool.
Methods A short VA tool was developed which was used by health workers during their routine house visits while a standard International Network of Field Sites with continuous Demographic Evaluation (INDEPTH) VA tool was filled by trained research workers for all adult deaths that occurred in 2008. The cause-specific mortality fraction using two tools, validity of the Comprehensive Rural Health Services Project (CRHSP) VA tool with INDEPTH VA tool as reference and agreement between the two tools, was compared.
Results The cause-specific mortality fraction was 11.6% and 12% for ischaemic heart disease (IHD), 10.6% and 11.8% for chronic pulmonary obstructive disease (COPD), and 9.4% and 7.3% for tuberculosis, using the INDEPTH and CRHSP VA tool, respectively. 16% and 21% of the deaths could not be classified using the INDEPTH and CRHSP VA tool respectively. The sensitivity of the CRHSP VA tool was 78.5% for IHD, 80% for COPD, 58.3% for tuberculosis, 92.8% for malignant neoplasm and 97.2% for intentional self harm. The kappa between two tools for IHD, COPD, tuberculosis, malignant neoplasm and intentional self harm was 0.754, 0.711, 0.628, 0.876 and 0.892 respectively.
Conclusion The short VA tool had a good sensitivity and fair to excellent agreement with the standard tool in different age groups across the major causes of death. It can be used within the routine healthcare delivery framework and can fill the gap in mortality surveillance.
- Verbal autopsy
- health workers
- agreement
- cause-specific mortality fraction
- health services research
- mortality SI
- vital statistics
Statistics from Altmetric.com
- Verbal autopsy
- health workers
- agreement
- cause-specific mortality fraction
- health services research
- mortality SI
- vital statistics
Introduction
Information on the distribution and trends of the causes of mortality is key to the development of effective health policies, evaluating existing programmes and policies as well as in improving accountability for expenditures on disease control. However, this information is lacking in India and other developing countries. There is a considerable lacuna in vital registration as well as cause-of-death (COD) reporting.1 The completeness of the vital registration system for the country has been estimated to be 50% of all deaths.2 More reliable medically certified COD data are available only in 14.5% of all registered deaths.2 3 This is because a large proportion of deaths are not attended for various reasons.
Verbal autopsy (VA) is an alternate method to obtain COD data and has been used in various settings including Sample Registration System in India.4 This method of ascertainment of the cause of mortality assumes that most causes of death have distinct symptom complexes that can be recognised, remembered and reported by lay respondents. Factors that may affect the validity of a VA include questionnaire design, type of interviewers and respondents, recall period, COD ascertainment mechanisms and COD classification.5
Despite increasing and widespread use of VA in field-data collection, utilisation of VA data for national epidemiological monitoring and global/regional burden of disease estimation has so far been limited. Verbal autopsies have been used to ascertain the underlying COD in neonates and children under 5 years of age,5–11 as well as for investigating CODs in case of maternal mortalities.12–15 VA has also been used in investigations of infectious disease outbreaks and risk factors for certain diseases, and in measuring the effect of public health interventions.16 17 Use of the VA in the Million Deaths study, which analysed data from the Sample Registration System in India, suggested that VA can ascertain the leading COD and reduce the misclassification of causes, and yields broad classification of the underlying causes in about 90% of deaths before the age of 70.18 Various VA tools have been used in different settings to obtain COD data. The International Network of Field Sites with continuous Demographic Evaluation (INDEPTH) VA tool is a standard tool and is used by many demographic surveillance sites.
The ideal mortality measurement system has several characteristics. It should be routine, reproducible, long-term, low-cost and sustainable. Most of the available VA tools require specialised training and take a longer time to complete, precluding their use in routine mortality surveillance, and thus can be used only in research settings or on a sample of the population. A short tool for routine use needs to be developed. Keeping in mind the need for information on causes of death, a VA tool for adult deaths was developed at the Comprehensive Rural Health Services Project (CRHSP), Ballabgarh in 2002 and revised in 2007. This replaced the lay-worker reporting system in existence previously which had only the narrative section. While experience with its use has been encouraging, the tool needs to be validated. With this background, this study was carried out with the objective of comparing the cause-specific mortality fraction (CSMF) and agreement between the CRHSP VA tool and the INDEPTH VA tool and determining the validity of the CRHSP VA tool for major causes of death.
Methods
This was a cross-sectional study carried out from January to December 2008 in the Intensive Field Practice Area (IFPA) of CRHSP, Ballabgarh. CRHSP, Ballabgarh is a field-practice area of All India Institute of Medical Sciences, New Delhi as well as an INDEPTH Health and Demographic Surveillance Site. It covers a population of about 86 000 in 28 villages through its two primary health centres, and provides all medical and health-related services. Health workers make house-to-house visits to provide basic health services. All the deaths are routinely reported by the health workers, and verbal autopsies are carried out. Assignment of COD is done by the Medical Officer of the Primary Health Centre.
Description of VA tools
CRHSP VA tool requires minimal training and takes less time to complete. On average, the time taken by a trained health worker to complete the INDEPTH VA tool process was approximately 90 min in comparison with 30 min for the CRHSP VA tool. Similarly, more time is taken by the physician to assign a COD after using the INDEPTH VA tool than with the CRHSP VA tool. The CRHSP VA tool has five portions and includes an open narrative portion and a 42-symptom checklist. The INDEPTH VA tool has 10 modules, comprising an open-ended narrative portion as well as a structured questionnaire with a set of 20 leading questions to elicit signs and symptoms of the final illness leading to death.
Tool development
The INDEPTH VA tool was translated into local dialect and pretested at Ballabgarh in a non-project village. Subsequently, independent back-translation of the tool was done. The CRHSP VA tool was developed by modification of an earlier VA tool by two investigators (SKK and AK). The earlier VA tool did not include the narrative portion which was included later. The CRHSP VA tool is already being used for routine mortality surveillance in the area.
Training
Field-research assistants were recruited and trained for 3 days to administer the INDEPTH VA tool. They were female lay-workers with previous experience in data collection in health-related projects. They were trained in interview techniques as well as causes of death. Health workers and health supervisors were trained for 1 day in administering the standardised CRHSP VA tool. Their training was shorter, as they were already using the tool.
Data collection
All the adult deaths that took place in the IFPA of CRHSP, Ballabgarh during January to December 2008 were included in the study. None of the deaths were excluded from the study. All deaths over 12 years of age were considered as adult deaths.
For all the adult deaths that occurred in IFPA, first the CRHSP VA tool was used by health workers. Since all the households are visited twice in a month by health workers in a month, none of the deaths occurring in the area are missed. Keeping in mind the sensitive nature of the information to be collected and the emotional trauma that the recall of the events preceding the death of a beloved is likely to cause to a relative, a reasonable time gap was kept between the form completion and the date of death (over 2 weeks but not exceeding 6 weeks for the CRHSP VA tool). After a gap of 2 weeks and within 3 months of the death, the same families were visited by the field-research assistants who completed the translated INDEPTH VA tool. These research assistants were also asked to look for additional deaths in the villages, and none were identified by them.
Quality control
Field supervisors verified the details in the INDEPTH VA tool. Details in the case of the CRHSP VA tool were verified by health supervisors after visiting the household. They also checked the forms for completeness.
COD assignment
The COD assignment for the INDEPTH VA tool was carried out independently by two investigators. For the CRHSP VA tool, the COD assignment was carried out independently by the Medical Officer of the Primary Health Centre and one of the investigators. In case of any discrepancy between the two physicians, the form was referred to an external expert in the field of Public Health for COD assignment (SKK), and his decision was adjudged to be final. Subsequently, all the COD was coded as per International Statistical Classification of Diseases 10th Revision (ICD-10) criteria. The agreement between two coders for both tools was calculated.
Statistical analysis
Data were entered into Microsoft Excel 2007. Data were analysed using SPSS version 17.0. The cause-specific mortality fractions were calculated as per the various ICD-10 codes and were compared between the CRHSP VA and INDEPTH VA tools. An a priori CSMF estimate using the CRHSP VA tool within 20% of the INDEPTH VA tool was deemed acceptable.19 20 The sensitivity, specificity and positive predictive value of CRHSP VA tool were calculated with COD from the INDEPTH VA tool as a reference. Agreement was also estimated for various ICD-10 codes using Cohen's kappa. A kappa value of more than 0.75 was considered excellent, 0.40 to 0.75 fair to good and below 0.40 poor agreement. A subgroup analysis was also carried out on the basis of sex and age.
Ethical issues
Approval from the Institute Ethics committee at AIIMS was obtained before the start of the study. Field workers and assistants were trained in interview techniques, so as to minimise any trauma to the respondent when discussing the death of a loved one. Written informed consent was taken from all interviewees before the administration of the INDEPTH VA tool, which is an added activity for this study. The routine VA tool was already being used as a routine surveillance tool, and no consent was taken for it. All the information gathered from the interviewees was kept confidential.
Results
Five hundred and nine persons aged above 12 years died during the year 2008. The mean age was 59.9 (SD±23.08) (table 1). There was perfect agreement between two coders for the CRHSP VA tool for ischaemic heart disease (IHD), tuberculosis, malignant neoplasm and intentional self harm, and κ was 0.834 for chronic obstructive pulmonary disease (COPD). The agreement between two coders of INDEPTH VA tool was 0.768, 0.834, 0.865, 1 and 1 for IHD, COPD, tuberculosis, malignant neoplasm and intentional self harm respectively.
Characteristics of persons >12 years who died between January and December 2008
The most common COD was IHD using both the INDEPTH VA tool and the CRHSP VA tool. The second major COD was COPD. Tuberculosis, malignant neoplasm and intentional self harm were other common causes of death (table 2). Among the unclassified category in the INDEPTH VA tool, 40% were due to senility, 30% due to unspecified fever and 15% due to sudden death. In the CRHSP VA tool, 75% of the deaths in unclassified category were due to senility, and 9% were due to unspecified fever. Significantly more women were coded in the unclassified category (p=0.001) by both VA tools. Among the deceased who had not been attended by a qualified physician prior to death, IHD was the most common COD by both the INDEPTH VA tool and the CRHSP VA tool (12.6% and 15.4%). Other common causes of death were COPD (10.9% vs 11.6%), intentional self harm (9.7% vs 9.2%), malignant neoplasm (7.4% vs 9.9%) and tuberculosis (5.9% vs 5.7%), respectively; 24.5% and 17.3% of the deaths could not be classified by the INDEPTH VA tool and CRHSP VA tool respectively. Among the deceased who had been attended by a qualified physician prior to death, only 3.1% of the deaths could not be classified by either of the tools.
Cause-specific mortality fraction determined by the INDEPTH verbal autopsy (VA) tool and Comprehensive Rural Health Services Project (CRHSP) VA tool
In persons younger than 25 years, most of the deaths were due to injuries, either self-inflicted or in road-traffic injuries. Intentional self harm was the COD in half the women; 46% by the INDEPTH VA tool and 50% by the CRHSP VA tool. It was also the COD in more than one-quarter of men in this age group by both tools. There were no unclassified deaths among men, whereas in women deaths in the unclassified category were 6.7% and 13.3%, respectively, by the INDEPTH and CRHSP VA tools.
In persons in the age group 25–49 years, road-traffic injuries were most common among men, determined by both INDEPTH and CRHSP VA tools (16.7% and 15.4% respectively). Other major causes of death among men in this age group were IHD (14.1% and 12.8% respectively), intentional self harm (12.8% and 14.1% respectively), tuberculosis (12.8% by both tools), and other accidents and injuries (11.5% and 10.3% respectively). Among women in this age group, the most common COD was malignant neoplasm (26.9% by both tools) and intentional self harm (19.2% and 23.1% respectively). There were very few deaths in the unclassified category among both men and women.
In persons older than 50 years, COPD was the most common COD, among both men and women, determined by both the INDEPTH VA tool and the CRHSP VA tool (17.3% and 19.3% respectively in men; 9.9% and 11.3% respectively in women). Other major causes of death determined by the INDEPTH and CRHSP VA tool were IHD (13.9% and 14.9% respectively in men; 9.2% by both tools in women), tuberculosis (11.4% and 8.4% respectively in men; 7.0% and 4.2% respectively in women), malignant neoplasm (8.4% and 9.4% respectively in men; 7.7% by both tools in women), and cerebro-vascular diseases (5.0% and 5.4% respectively in men; 9.9% by both tools in women). A large number of deaths could not be classified by both tools (17.3% and 23.8% respectively in men; 26.1% and 34.5% respectively in women).
The sensitivity of the CRHSP VA tool was 77.9% for IHD, 80% for COPD, 58.3% for tuberculosis, 92.8% for malignant neoplasm, 97.2% for intentional self harm, 73.3% for cerebro-vascular diseases, 45.1% for other infectious diseases and 79.6% for other injuries. The specificity and positive predictive value of the CRHSP VA tool were 96.3% and 76.7% for IHD, 96.2% and 70.1% for COPD, 98.1% and 75.6% for tuberculosis, 98.5% and 85% for malignant neoplasm, 98.3% and 81.3% for intentional self harm, 97.9% and 68.8% for cerebro-vascular diseases, 98.7% and 70% for other infectious diseases, and 98.6% and 86.7% for other injuries respectively. For the deceased who had not been attended by a qualified physician, the sensitivity was only marginally poor: 72.3% for IHD, 78.7% for COPD, 55% for tuberculosis, 94.5% for malignant neoplasm and 95.2% for intentional self harm.
A misclassification matrix between two tools is presented in table 3. Of 59 deaths classified as IHD by the INDEPTH VA tool, only 46 could be identified by the CRHSP VA tool. Out of 48 deaths by tuberculosis determined by the INDEPTH VA tool, only 28 could be identified by the CRHSP VA tool, whereas of 31 deaths by other infectious diseases, only 14 could be identified by the CRHSP VA tool. Misclassification was also high for COPD. At the population level, though, misclassification is largely compensatory. Agreement between the INDEPTH VA tool and CRHSP VA tool was fair to good in COPD and tuberculosis (κ=0.711 and 0.628 respectively) and excellent in IHD, malignant neoplasm and intentional self harm (κ=0.754, 0.876 and 0.892 respectively) (table 4). For all five major causes of death, agreement between the INDEPTH VA tool and CRHSP VA tool was better among the deceased who had been attended by a qualified physician prior to death as compared with the deceased who had not been attended by a qualified physician prior to death. For IHD, COPD, tuberculosis, malignant neoplasm and intentional self harm, κ was 0.861 and 0.715, 0.789 and 0.721, 0.782 and 0.594, 0.913 and 0.824, 0.911 and 0.862 respectively. In tuberculosis and malignant neoplasm, the agreement was better among men, whereas in IHD, COPD and intentional self harm, the agreement was better among women. In IHD, COPD and intentional self harm, the agreement was better in the age group 25–49 years, while in tuberculosis, the agreement was better in the age group younger than 25 years. Agreement between the INDEPTH VA tool and CRHSP VA tool was better in non-communicable diseases than in communicable diseases (κ=0.689 and 0.601 respectively). In non-communicable diseases, the agreement was better among women, while it was better among men in communicable diseases. For both communicable and non-communicable diseases, the agreement was better among 25–49-year-old deceased compared with the deceased in other age groups.
Misclassification matrix for major causes of death
Agreement between International Network of Field Sites with continuous Demographic Evaluation verbal autopsy tool and Comprehensive Rural Health Services Project verbal autopsy tool in five major causes of death
Discussion
The study reports on the use of a short VA tool for routine mortality surveillance and compares its results with a longer tool primarily used for research purposes. While it would have been better to compare the performance against medical certification, this was not possible, as most of the deaths in the area occur at home with the deceased not having received any form of medical attention in the hours prior to the death. Moreover, the highly selective nature of hospital admission in rural area may limit the use of hospital-based data to validate a tool which is to be used in the field. CSMF obtained from hospital deaths may also be distorted owing to selection bias. CSMF is an acceptable way to assess the performance of VA methods with population-level estimates within 20 to 25% of the true value considered adequate.19 20
Tools with both a narrative section and closed ended questions increase the proportion of classifiable causes of death and are recommended by WHO.11 21 22 The CRHSP VA tool contains both a narrative section and closed ended questions with filters. Although the proportion of unclassified deaths was greater in the CRHSP VA tool than in the INDEPTH VA tool, a study conducted in rural Tamil Nadu in India reported more unclassified deaths than reported with the CRHSP VA tool.23 A more extensive checklist of symptoms could be the reason for fewer unclassified deaths determined by the INDEPTH VA tool.
Various studies have suggested the optimal time to conduct VA to be within 1 month to 2 years of death.24 25 Since the entire VA was conducted within 3 months of death, any problems with recall are likely to be minimal. Health workers were used to conduct the VA using the CRHSP VA tool. While some studies have reported that VA using well-trained lay-persons can yield accurate information,24 others have reported that the use of medically trained health professionals may be more appropriate.26 27 Health workers were used in this study, as their role in conducting VA has already been established in this area in a previous study, and also their use may help in routine mortality surveillance.6 This study also reinforces their role in VA.
There are several approaches to derive COD from VA; physician review, predefined expert algorithms and data-driven algorithms. The most widely used approach to derive causes of death from VA is physician review, in which a panel of physicians assigns COD based on clinical judgement of the information from the VA questionnaire. The validity of physician review of VA has been tested in children and adults, and shown to have reasonable sensitivity and specificity for selected causes of death.28–31 However, physician review is a costly method of ascertainment of COD and may not be feasible, given the unavailability of physicians in remote areas. Also, the repeatability of causes of death derived by physician review is low.32 In such a scenario, the use of algorithm-based protocols may be considered. Since the information is collected by health workers and the COD assigned by a medical officer from the primary health centre, the data are generated and used for planning and evaluating the healthcare at the local level.
CSMF determined by the CRHSP VA tool was within 20% of the CSMF determined by the INDEPTH VA tool for most of the major causes of death across various age groups among both sexes, indicating the adequacy of the CRHSP VA tool. The CSMF determined by both tools was similar and comparable with another study in rural India, though the latter had a lower proportion of COPD.33 Unclassified deaths were significantly more common in women. CSMF was also similar in all age groups, in both men and women, though the agreement was stronger in the younger age groups. This may be explained by the fact that for most of the causes of death in the younger age group, the two tools had a stronger agreement. The CSMF determined by the CRHSP VA tool was within 20% of the CSMF determined by the INDEPTH VA tool, also among the deceased who had not been attended by a qualified physician except for IHD. Most of these were sudden deaths and therefore could not be classified as IHD. Most of the deaths among hospitalized patients could be assigned specific cause of death, as the relatives of the deceased were well informed regarding the chain of events leading to death and were also informed of the diagnosis by treating physicians.
Misclassification depends upon the coexistence of diseases with similar features in a geographical region.23 The high prevalence of both tuberculosis and COPD with similar features might be the reason for the higher level of misclassification. In case of infectious diseases, the presence of non-specific symptoms may have caused the misclassification. In the case of IHD, most of the misclassification concerns unclassified deaths. This probably suggests that the VA tool should include more specific questions to avoid misclassification. Since the misclassification is largely compensatory at the population level, the CRHSP VA tool gives a less distorted picture of CSMF, though the agreement was only modest in two major causes of death. With the CRHSP VA tool, it was possible to classify more cases of injuries as intentional self harm than with the INDEPTH VA tool. This may be explained by the better rapport between informant and health worker, as health workers work in the community and are able to extract this sensitive information. This also underlines the importance of rapport between interviewer and informant.
The sensitivity of major causes of death such as IHD, cerebro-vascular diseases and tuberculosis was comparable with that obtained in a study in north India which used the physician's diagnosis as a reference, though the CRHSP VA tool has a much higher sensitivity for COPD.21 The sensitivity for malignant neoplasm was quite high and was comparable with a study in south India.23 The sensitivity of major causes of death was also similar to that obtained in a study among an urban population in China, which reported a sensitivity of 64% for IHD, 81% for stroke, 62% for tuberculosis and 97% for accidents and injuries.25 In contrast to the current study, a study in rural South Africa found the sensitivity of the VA tool to be much higher for tuberculosis and other infectious disease, and much lower for cardiac disorders, malignant neoplasm and injuries, though the report from other studies in Africa presents a conflicting picture.29 34
Among the five major causes of death, agreement was excellent in intentional self harm, malignant neoplasm and IHD, and fair to good in COPD and respiratory tuberculosis. The CRHSP VA tool had a better agreement with the INDEPTH VA tool in non-communicable diseases than with communicable diseases. This is in contrast to a multicentre study in Africa, where VA tools were found to have a higher sensitivity for detecting communicable diseases than for non-communicable diseases.34 Agreement was better among deceased who had been attended by a qualified physician prior to death than among those who did not receive medical attention. This is because the relatives were aware of the diagnosis in the case of those who died in hospital.
In conclusion, the shorter CRHSP VA tool has several advantages which make it useful for routine mortality surveillance. It takes less time, routine health workers can use it after appropriate training, and it provides data that are useful for priority setting. Thus, it can be used within a routine healthcare-delivery framework and can fill the gap in mortality surveillance in resource-poor settings.
What is known on this subject
Verbal autopsy is used to obtain cause of death data where system of medical certification of death is weak.
Most of the standard tools are long; hence preclude their use in routine healthcare delivery framework.
What this study adds
A shorter tool used in this study has good sensitivity and fair to excellent agreement with a standard tool.
Shorter tool can be used by health workers during their routine visits and can fill the gap in mortality surveillance.
Acknowledgments
CRHSP Ballabgarh is a member of the INDEPTH Network and acknowledges its contribution in strengthening its demographic surveillance activities.
References
Footnotes
Funding This study was funded by the intramural research grant of the All India Institute of Medical Sciences, New Delhi.
Competing interests None.
Ethics approval Ethics approval was provided by the Institutional Ethics Committee, All India Institute of Medical Sciences.
Provenance and peer review Not commissioned; externally peer reviewed.