Identifying problems with data collection at a local level: survey of NHS maternity units in England
BMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7210.619 (Published 04 September 1999) Cite this as: BMJ 1999;319:619
- Natalie Kenney, survey researcher,
- Alison Macfarlane, reader in perinatal and public health statistics (alison.macfarlane{at}perinat.ox.ac.uk)
- Correspondence to: A Macfarlane
- Accepted 29 July 1999
Abstract
Objectives: To document the extent to which maternity data are collected and how they are recorded, and to identify problems that may affect their availability throughout the NHS.
Methods: Postal survey in September 1997 with structured questionnaires.
Setting: 207 NHS trusts with maternity units in England.
Participants: Heads of midwifery in maternity units.
Main outcome measures: Extent to which maternity data were routinely recorded, how they were recorded and evaluated, and to whom they were made available.
Results: 167 (81%) of questionnaires were returned, representing 166 trusts. Of these trusts, 165 collected ≥17 of the 19 data items in HES maternity tail, and 158 collected ≥40 of the 45 items selected from Körner dataset. Only 18 collected all five items selected from the “indicators of success,” and 17 did not collect any. In 58 of trusts data were primarily recorded on paper. A computerised maternity information system was used by 106 (63%) of trusts, but many recorded data on paper first. Thirty four did not audit data for accuracy. Most trusts analysed data not routinely collected at national level, but 18 did not analyse HES maternity tail and 17 did not analyse Körner data.
Conclusions: Improvement is needed in quality, completeness, and availability of maternity data at a national level, particularly if the NHS information strategy is to be successfully implemented. Although most of the data items in national datasets are recorded locally, variations in the way data are defined, recorded, and analysed and lack of linkage between computer systems restrict their access, availability, and use at local, district, and national levels.
Key messages
Most NHS maternity data missing at a national level are collected at a local level
Variations in the way in which maternity data are defined, recorded, audited, and analysed can affect the quality, completeness, and availability of this information throughout the NHS
Considerable work and investment will be required to implement the plans set out in the NHS information strategy Information for Health
Introduction
The NHS information strategy for England, described in Information for Health,1 reiterated the need to improve the accuracy, completeness, and availability of NHS data.2 Data about maternity care are needed at all levels of the NHS to monitor the health of women and their babies and the services provided.3 4 These data are also required to audit, monitor, and evaluate changes in the provision of care.5 Women need this information to inform decisions about their care.1 6 It is widely acknowledged that national maternity data for England are incomplete and that some are inaccurate or unavailable7–9 and that they need to be improved.7 9 10
At a national level, the Department of Health's hospital episode statistics (HES) system collects a core set of data about each episode of “admitted patient care.” For episodes in which a baby is born, a supplementary set of data should be appended in the HES “maternity tail.”11 These data are lacking from about a third of delivery records.11 The core HES record, the maternity tail, and additional maternity data are commonly referred to jointly as “Körner” datasets.12
To investigate the reasons for deficiencies in national maternity data, a project was commissioned to document the extent to which maternity data are recorded in NHS trusts and to identify problems that may affect the quality or availability of this information for use elsewhere in the NHS and to women using the maternity services. This paper reports, in brief, on the survey of NHS trusts with maternity units in England.
Methods
We designed a structured questionnaire that asked NHS trusts which maternity data items were routinely collected, how the data were recorded, whether they were audited for accuracy, the extent to which they were analysed, how accessible they were, how they were used, and to whom they were made available. The questionnaire was piloted in Northern Ireland.
In September 1997 we sent questionnaires to heads of midwifery in all 207 NHS trusts with maternity units in England. Reminder letters were sent to non-respondents three and six months later.
The replies were checked for anomalies and analysed with SPSS 7.5 for Windows.
Results
We received replies from 167 (81%) of maternity units, representing 166 NHS trusts. Some of the respondents did not answer all of the questions. Midwifery managers and midwives usually completed the questionnaires.
Extent of data collection
Of the 165 NHS trusts that replied to the question about data collection, all routinely collected ≥17 of the 19 data items contained in the HES maternity tail11 and 158 (96%) routinely collected ≥40 of the 45 data items selected from the Körner maternity dataset.12 Most trusts also recorded a substantial number of other maternity data items. Of the 10 “indicators of success” listed in Changing Childbirth,6 five are easily quantified, but only 18 trusts routinely collected all five items, while 17 collected none (table 1).
Extent to which maternity units of 166 NHS trusts in England collected five “indicators of success” listed in Changing Childbirth6
How data were recorded
In 58 of the 166 trusts, data were primarily recorded on paper. Some data may have been recorded on the patient administration system, but we did not investigate the extent to which this was done. Of these trusts, 48 stated that they experienced problems recording data.
In 106 (63%) of the trusts data were recorded on a computerised maternity information system. In only 35 of all trusts did midwives enter data directly on computer systems. In 50 trusts data were entered separately on both the maternity information system and the patient administration system, and only 17 trusts were able to transfer data from the maternity system to the patient administration system automatically.
Overall, 113 (68%) of the trusts used both paper and computerised recording methods, with many recording some data on paper and other data on computers. Two trusts did not indicate how data were recorded.
Audit of data
One hundred and twenty six (76%) of the trusts stated that data were routinely audited for accuracy, with 63 doing so monthly or more frequently. Resources did not allow us to check the extent or quality of these audits. Data were more likely to be audited if midwives collected data on paper and subsequently entered them onto computers.
Analysis of data
Data from the HES maternity tail and Körner dataset were analysed by 150 (90%) of the trusts, and most trusts produced at least some statistical analyses for the additional data that they collected. We did not ascertain the extent to which data were analysed. Most trusts analysed data with computer software, but 28% did so manually.
Access to data
We asked the trusts to provide aggregated figures for six data categories for the year 1996, including items in the HES maternity tail and Körner dataset (table 2). Only 15 trusts could provide figures for all the data items. Although a further 151 trusts were able to provide some of the data, many could not provide them in the form requested, either because aggregated statistics were not routinely produced for those items or there were difficulties accessing paper records. Some trusts provided aggregated statistics for the whole data category. Many NHS trusts could not provide the required figures because they used different definitions or groupings of data. Other researchers have encountered the same problem.13
Maternity units' access to maternity data in 166 NHS trusts in England
Ninety six of the 151 trusts that could not provide some data claimed that the data could be accessed if required urgently. Of these trusts, 42 could access data in less than a day, but 11 stated that it would take two weeks or more to obtain the information.
Availability and use of data
Of those trusts that answered the question, 93% (141/152) presented maternity data at local clinical meetings. Only 66% (97/146) routinely provided data to all clinicians, 53% (57/108) provided data to general practitioners, and 27% (32/118) included data in information leaflets for women. The type of data made routinely available was not specified. Although 91% (128/140) of trusts provided data to their local health authority, our parallel survey of health authorities showed that these were more likely to relate to contracting than to clinical practice.14
Discussion
Problems in the collection of maternity data in English NHS trusts lead to inadequacies in the quality, completeness, and availability of the information. These problems restrict the use of the data by the NHS and may impede the implementation of the NHS information strategy.1
We found that the NHS maternity units collected a substantial amount of data, but many trusts used different definitions for various data items This limits their use throughout the NHS because of the difficulty in aggregating and comparing data that have been collected using different definitions.15
There may be justifiable reasons for not collecting data for the “indicators of success” listed in Changing Childbirth,16 but the lack of these data makes it difficult to assess the policy's overall impact. This information may be collected on a sample or ad hoc basis for health authorities, but our survey of health authorities found that only 58% monitored the implementation of the policy.14
The combination of computerised and paper methods commonly used by NHS trusts to record data can make linkage and management of data difficult, restrict their use, and affect the flow of data through the NHS and to the Department of Health. The duplication of data recording is wasteful of time and NHS resources.15 Improved linkage is needed between computerised maternity information systems and patient administration systems to ensure that data are recorded only once.
Although computerised maternity information systems are supposed to make it easier to record and use data,17 many problems persist.18 19 Eleven of the trusts in our survey reported that it would take two or more weeks to obtain data that were not immediately accessible Unless information is readily available it cannot be used effectively to inform NHS decision making.1 6 9
It is of concern that 28% of the trusts analysed their data manually. This is laborious, time consuming, and limits the extent of analyses. This can impede the use of data locally and the forwarding of information to NHS systems. If the NHS information strategy is to be successfully implemented, these issues must be addressed.
Evaluating the quality of data will soon become mandatory in the NHS,1 but 24% of the trusts did not routinely audit their maternity data for accuracy. Conducting regular audit of data and educating clinicians about how to audit their maternity records can improve data quality 10 20 Feeding back data to those who collect them is also associated with improving data quality and clinical practice,20 21 but 33% of the trusts did not routinely provide these data to clinicians.
Most trusts produced statistical analyses of data not routinely collected by the Department of Health. This suggests that there is a local need for these analyses, but their use is limited by a lack of comparable national data. Although maternity data were often made available within NHS trusts, the availability of data to others was limited With the emphasis on sharing information in the NHS and the provision of information to users of the health services,1 2 availability must be improved.
Conclusions
Most of the maternity data missing at a national level are widely collected at a local level. Variations in the ways data are recorded, and many other problems, contribute to inadequacies in the quality, completeness, and availability of this information. Lack of computer systems for recording data in some trusts and lack of linkage between “stand alone” computer systems and hospitals' patient administration systems in others are major problems. These problems restrict data use at all levels of the NHS15 and may impede the implementation of the NHS information strategy.1
NHS trusts can do much to improve the quality of their maternity data by conducting regular audits of data, informing clinicians of the need for accurate and complete data, and feeding data back to those collecting them.10 20 21 Now that the Department of Health has started to publish comparative HES maternity data,11 trusts will need to ensure that their data are accurate, complete, and up to date so that their services are accurately represented in comparisons with other trusts.
At a national level, the Department of Health has committed itself to a revised maternity dataset.1 22 Once implemented, this should ensure that data are collected according to agreed definitions, making it easier to aggregate and use this information throughout the NHS. When the maternity dataset is agreed, the Department of Health should allocate resources to maternity units for computer systems that will enable them to collect data in a consistent way and which can be linked to other NHS computer systems.
It has been suggested that contributing maternity data to national systems has a low priority at a local level.8 Now that it has begun to publish maternity data,11 the Department of Health should consider making it mandatory for maternity units to contribute data to national systems. When new policies are implemented it is essential that the relevant data are collected locally and aggregated nationally so that the impact of policies on the maternity services and their benefit to women and their babies can be assessed.
Acknowledgments
We thank all those who completed questionnaires for this project.
Contributors: AM initiated the research at the suggestion of Kate Jackson, director of the Changing Childbirth Implementation Team; cowrote the protocol; assisted with questionnaire design and data analysis; and participated in writing and editing the paper. NK cowrote the protocol, assisted with questionnaire design, and participated in data collection and analysis and writing and editing the paper. Pam Dobson (Kings College Healthcare NHS Trust), Cathy Winter (Southmead NHS Trust), Maggie Jeffries (Oxford Radcliffe NHS Trust), Fiona Alderdice (Queens University Belfast), Lesz Lancucki (Department of Health), Jane Cowl (Changing Childbirth Implementation Team), and Meg Goodman (Maternity Alliance) commented on and discussed core ideas about the content and design of the questionnaire. Sarah Ayers (National Perinatal Epidemiology Unit) designed the computer databases and dealt with all computing issues. Jo Garcia, Rona McCandlish, Sally Marchant, Valerie King, and Kirstie McKenzie-McHarg commented on the questionnaire design and drafts of the paper. Suzanne Williams, Sarah Bowler, and Hedwig Van Asten entered the data. AM and NK are guarantors for the paper.
Footnotes
-
Funding AM was funded through the Department of Health's policy research programme. NK was funded by the Changing Childbirth Implementation Team, Department of Health.
-
Competing interests None declared.