Inter-rater agreement in defining chemical incidents at the National Poisons Information Service, London

Abstract

Background: National surveillance for chemical incidents is being developed in the UK. It is important to improve the quality of information collected, standardise techniques, and train personnel.

Objective: To define the extent to which eight National Poison Information Service specialists in poison information agree on the classification of calls received as “chemical incidents” based on the national definition.

Design: Blinded, inter-rater reliability measured using the κ statistic for multiple raters.

Setting: National Poison Information Service and Chemical Incident Response Service, Guy’s and St Thomas’s NHS Trust, London.

Participants: Eight specialists in poison information who are trained and experienced in handling poisons information calls and have been involved in extracting information for surveillance.

Results: The overall level of agreement observed was at least 69% greater than expected by chance (κ statistic). Fire and incidents where chemicals were released within a property had a very good level of agreement with κ statistic of 83% and 80% respectively. The lowest level of agreement was observed when no one or only one person was exposed to a chemical (33%) and when the chemical was released into the air (48%).

Conclusion: High levels of agreement were observed. There is a need for more training and improvement in consistency of the data collected by all organisations.