International Journal for Quality in Health Care Advance Access published online on January 23, 2006
International Journal for Quality in Health Care, doi:10.1093/intqhc/mzi098
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1 Division of Research, Joint Commission on Accreditation of Healthcare Organizations, Oakbrook Terrace, IL, USA
* To whom correspondence should be addressed. Objectives. To investigate the reliability of self-reported standardized performance indicators introduced by the Joint Commission on Accreditation of Healthcare Organizations in July 2002 and implemented in approximately 3400 accredited US hospitals. The study sought to identify the most common data quality problems and determine causes and possible strategies for resolution. Design. Data were independently reabstracted from a random sample of 30 hospitals. Reabstracted data were compared with data originally abstracted, and discrepancies were adjudicated with hospital staff. Structured interviews were used to probe possible reasons for abstraction discrepancies. Results. The mean data element agreement rate for the 61 data elements evaluated was 91.9%, and the mean kappa statistic for binary data elements was 0.68. The rate of agreement for individual data elements ranged from 100 to 62.4%. The mean difference between calculated indicator rates was 4.88% (absolute value) and the range of differences was 0.0-13.3%. Symmetry of disagreement among original abstractors and reabstractors identified eight indicators whose differences in calculated rates were statistically unlikely to have occurred through random chance (P < 0.05). Conclusion. Although improvement in the accuracy and completeness of the self-reported data is possible and desirable, the baseline level of data reliability appears to be acceptable for indicators used to assess and improve hospital performance on selected clinical topics.
Accepted December 2, 2005
Article
Assessing the reliability of standardized performance indicators
Scott C. Williams 1 *,
Ann Watt 1,
Stephen P. Schmaltz 1,
Richard G. Koss 1,
and
Jerod M. Loeb 1
Scott C. Williams, E-mail: swilliams{at}jcaho.org
Abstract 
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. R. Campbell, D. Satran, D. M. Larson, I. J. Chavez, B. T. Unger, B. P. Chacko, C. Kapsner, and T. D. Henry ST-Elevation Myocardial Infarction: Which Patients Do Quality Assurance Programs Include? Circ Cardiovasc Qual Outcomes, November 1, 2009; 2(6): 648 - 655. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Pulcini, S. Defres, I. Aggarwal, D. Nathwani, and P. Davey Design of a 'day 3 bundle' to improve the reassessment of inpatient empirical antibiotic prescriptions J. Antimicrob. Chemother., June 1, 2008; 61(6): 1384 - 1388. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Greenfield and J. Braithwaite Health sector accreditation research: a systematic review Int. J. Qual. Health Care, June 1, 2008; 20(3): 172 - 183. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. C. Williams, R. G. Koss, D. J. Morton, S. P. Schmaltz, and J. M. Loeb Case volume and hospital compliance with evidence-based processes of care Int. J. Qual. Health Care, April 1, 2008; 20(2): 79 - 87. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Marwick, E. Watts, J. Evans, and P. Davey Quality of care in sepsis management: development and testing of measures for improvement J. Antimicrob. Chemother., September 1, 2007; 60(3): 694 - 697. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. E. Landon, S.-L. T. Normand, A. Lessler, A. J. O'Malley, S. Schmaltz, J. M. Loeb, and B. J. McNeil Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals Arch Intern Med, December 11, 2006; 166(22): 2511 - 2517. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. C. Williams, R. G. Koss, D. J. Morton, and J. M. Loeb Performance of Top-Ranked Heart Care Hospitals on Evidence-Based Process Measures Circulation, August 8, 2006; 114(6): 558 - 564. [Abstract] [Full Text] [PDF]
|
|