International Journal for Quality in Health Care Advance Access originally published online on June 19, 2009
International Journal for Quality in Health Care 2009 21(4):292-300; doi:10.1093/intqhc/mzp023
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Analysis of unintended events in hospitals: inter-rater reliability of constructing causal trees and classifying root causes
1 NIVEL, Netherlands Institute for Health Services Research, Utrecht, The Netherlands
2 Department of Public and Occupational Health, EMGO Institute, VU University Medical Center, Amsterdam, The Netherlands
3 Department of Sociology, Utrecht University, Utrecht, The Netherlands
4 Department of Human Geography, Utrecht University, Utrecht, The Netherlands
Background. Root cause analysis is a method to examine causes of unintended events. PRISMA (Prevention and Recovery Information System for Monitoring and Analysis) is a root cause analysis tool. With PRISMA, events are described in causal trees and root causes are subsequently classified with the Eindhoven Classification Model (ECM). It is important that root cause analysis tools are reliable, because they form the basis for patient safety interventions.
Objectives. Determining the inter-rater reliability of descriptions, number and classifications of root causes.
Design. Totally, 300 unintended event reports were sampled from a database of 2028 events in 30 hospital units. The reports were previously analysed using PRISMA by experienced analysts and were re-analysed to compare descriptions and number of root causes (n = 150) and to determine the inter-rater reliability of classifications (n = 150).
Main outcome measures. Percentage agreement and Cohen's kappa (
).
Results. Agreement between descriptions of root causes was satisfactory: 54% agreement, 17% partial agreement and 29% no agreement. Inter-rater reliability of number of root causes was moderate ( = 0.46). Inter-rater reliability of classifying root causes with the ECM was substantial from highest category level ( = 0.71) to lowest subcategory level ( = 0.63). Most discrepancies occurred in classifying external causes.
Conclusions. Results indicate that causal tree analysis with PRISMA is reliable. Analysts formulated similar root causes and agreed considerably on classifications, but showed variation in number of root causes. More training on disclosure of all relevant root causes is recommended as well as adjustment of the model by combining all external causes into one category.
Keywords: adverse events, patient safety, incident reporting and analysis, medical errors, human factors
Address reprint requests to: Marleen Smits, Netherlands Institute for Health Services Research, PO Box 1568, NIVEL, 3500 BN Utrecht, The Netherlands. Tel: +31 30 2729663; Fax: 31 30 2729729; E-mail: m.smits{at}nivel.nl
Accepted for publication May 28, 2009.