Skip Navigation


International Journal for Quality in Health Care Advance Access originally published online on January 23, 2006
International Journal for Quality in Health Care 2006 18(3):246-255; doi:10.1093/intqhc/mzi098
This Article
Right arrow Full Text Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
18/3/246    most recent
mzi098v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (4)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Williams, S. C.
Right arrow Articles by Loeb, J. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Williams, S. C.
Right arrow Articles by Loeb, J. M.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

International Journal for Quality in Health Care vol. 18 no. 3 © The Author 2006. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved

Assessing the reliability of standardized performance indicators

Scott C. Williams, Ann Watt, Stephen P. Schmaltz, Richard G. Koss and Jerod M. Loeb

Division of Research, Joint Commission on Accreditation of Healthcare Organizations, Oakbrook Terrace, IL, USA

Objectives. To investigate the reliability of self-reported standardized performance indicators introduced by the Joint Commission on Accreditation of Healthcare Organizations in July 2002 and implemented in approximately 3400 accredited US hospitals. The study sought to identify the most common data quality problems and determine causes and possible strategies for resolution.

Design. Data were independently reabstracted from a random sample of 30 hospitals. Reabstracted data were compared with data originally abstracted, and discrepancies were adjudicated with hospital staff. Structured interviews were used to probe possible reasons for abstraction discrepancies.

Results. The mean data element agreement rate for the 61 data elements evaluated was 91.9%, and the mean kappa statistic for binary data elements was 0.68. The rate of agreement for individual data elements ranged from 100 to 62.4%. The mean difference between calculated indicator rates was 4.88% (absolute value) and the range of differences was 0.0–13.3%. Symmetry of disagreement among original abstractors and reabstractors identified eight indicators whose differences in calculated rates were statistically unlikely to have occurred through random chance (P < 0.05).

Conclusion. Although improvement in the accuracy and completeness of the self-reported data is possible and desirable, the baseline level of data reliability appears to be acceptable for indicators used to assess and improve hospital performance on selected clinical topics.

Keywords: performance indicators, performance measures, reliability, hospital data quality

Address reprint requests to Scott C. Williams, Division of Research Joint Commission, One Renaissance Blvd, Oakbrook Terrace, IL 60181, USA. E-mail: swilliams{at}jcaho.org

Accepted for publication December 2, 2005.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Circ Cardiovasc Qual OutcomesHome page
A. R. Campbell, D. Satran, D. M. Larson, I. J. Chavez, B. T. Unger, B. P. Chacko, C. Kapsner, and T. D. Henry
ST-Elevation Myocardial Infarction: Which Patients Do Quality Assurance Programs Include?
Circ Cardiovasc Qual Outcomes, November 1, 2009; 2(6): 648 - 655.
[Abstract] [Full Text] [PDF]

Home page
 J Antimicrob ChemotherHome page
C. Pulcini, S. Defres, I. Aggarwal, D. Nathwani, and P. Davey
Design of a 'day 3 bundle' to improve the reassessment of inpatient empirical antibiotic prescriptions
J. Antimicrob. Chemother., June 1, 2008; 61(6): 1384 - 1388.
[Abstract] [Full Text] [PDF]

Home page
 Int J Qual Health CareHome page
D. Greenfield and J. Braithwaite
Health sector accreditation research: a systematic review
Int. J. Qual. Health Care, June 1, 2008; 20(3): 172 - 183.
[Abstract] [Full Text] [PDF]

Home page
 Int J Qual Health CareHome page
S. C. Williams, R. G. Koss, D. J. Morton, S. P. Schmaltz, and J. M. Loeb
Case volume and hospital compliance with evidence-based processes of care
Int. J. Qual. Health Care, April 1, 2008; 20(2): 79 - 87.
[Abstract] [Full Text] [PDF]

Home page
 J Antimicrob ChemotherHome page
C. Marwick, E. Watts, J. Evans, and P. Davey
Quality of care in sepsis management: development and testing of measures for improvement
J. Antimicrob. Chemother., September 1, 2007; 60(3): 694 - 697.
[Abstract] [Full Text] [PDF]

Home page
 Arch Intern MedHome page
B. E. Landon, S.-L. T. Normand, A. Lessler, A. J. O'Malley, S. Schmaltz, J. M. Loeb, and B. J. McNeil
Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals
Arch Intern Med, December 11, 2006; 166(22): 2511 - 2517.
[Abstract] [Full Text] [PDF]

Home page
 CirculationHome page
S. C. Williams, R. G. Koss, D. J. Morton, and J. M. Loeb
Performance of Top-Ranked Heart Care Hospitals on Evidence-Based Process Measures
Circulation, August 8, 2006; 114(6): 558 - 564.
[Abstract] [Full Text] [PDF]


Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.