International Journal for Quality in Health Care 16:181-182 (2004)
International Journal for Quality in Health Care vol. 16 no. 2 © International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved
Book review |
Risk Adjustment for Measuring Health Care Outcomes, 3rd edition
Lisa I. Iezzoni (editor)Hardback, 508 pp, August 2003, ISBN 1-56793-207-X, $76.50, AcademyHealth/HAP (http://www.ache.org/hap.cfm)
Lisa Iezzoni and colleagues produced the first edition of Risk Adjustment for Measuring Health Care Outcomes nearly 10 years ago. At that time, payers and researchers were concerned with risk adjustment primarily as a tool for hospital payment and for assessing hospital performance. Although the Health Care Financing Administration had ceased issuing Medicare risk-adjusted mortality rate reports, several other hospital performance reporting projects were then underway, such as New York State’s coronary artery bypass grafting (CABG) surgery mortality project, Pennsylvania’s hospital outcomes reporting project, and the Cleveland Health Quality Choice program. In the years since the publication of the first edition, interest in risk adjustment in health care has expanded greatly. Risk adjustment continues to be used in hospital quality and efficiency reports, but now it is also a frequent input to health plan capitation calculations, and it is used with increasing frequency for measuring and reporting the performance of physicians, networks, groups, and even individuals. In addition to state agencies, researchers, and health plans, now purchasers—primarily large employers and business coalitions—have entered the field as major consumers of health provider performance information. Additional types of outcomes and new data sources and statistical methodologies have emerged, and the literature on risk adjustment issues has expanded substantially.
One of the most important lessons learned during the 1990s is that appropriate risk adjustment must be context specific. Dr Iezzoni indicates that ‘devising appropriate risk-adjustment strategies requires answers to four major questions:
Risk of what outcome?
Over what time frame?
For what population?
For what purpose?’
In their discussions of conceptual, methodological, and applications issues, the authors frequently return to these questions. Among the topics addressed in the book are: specific patient-level characteristics used in risk-adjustment models (e.g. demographic, clinical, functional); windows of observation for risk adjustment (e.g. hospitalization, episode, fixed time period); data sources for developing, validating, and applying risk-adjustment models (e.g. administrative data, medical records, surveys); statistical and conceptual issues related to model development and validation; methodological issues related to application of risk models, including comparison of outcomes across providers; and risk-adjustment issues specific to four populations: pediatric patients, mental health patients, long-term care patients, and people with disabilities.
Although the topics addressed can be quite technical, the authors eschew the use of clinical and statistical jargon, and they attempt to make the text comfortably readable by people from a variety of backgrounds. Because it represents a comprehensive treatment of the subject or health care risk adjustment, the book may prove useful to a number of groups. It can help consumers of risk-adjusted outcome information evaluate whether or not data on provider performance is credible or methodologically flawed. It can aid providers who are the subjects of risk-adjusted outcome reports to determine whether or not their performance has been fairly portrayed. Further, it can be useful to researchers and analysts wishing to avoid methodological pitfalls as they work to develop new performance monitoring and reporting programs.
My own research background has focused primarily on issues of validity in risk-adjusted outcomes studies. Thus, in reviewing Risk Adjustment for Measuring Health Care Outcomes, I found myself being particularly sensitive to questions that deal with the limitations and appropriate uses of risk-adjusted measures. I was very pleased to see Dr Iezzoni emphasize that the purpose for which a measure is used dictates how well it must perform, and that the measure must be sufficiently valid for the explicit purpose for which it is used. A mortality rate analysis using administrative data and based on a generic risk model might be satisfactory for longitudinal monitoring of trends within a health plan, but it would be highly inappropriate for public reporting of provider performance. For the same reason, I was also pleased to see the discussion by Ash, Swartz, and Pekoz on issues associated with comparison of observed to expected outcomes across providers. These authors note that, using the same data, ratio-based measures can lead to conclusions other than difference-based measures. They also note that, when comparing providers, standard errors of observed outcomes can be problematic: ‘. . . if a provider’s expected death rate is 10%, an observed rate of 15% based on 400 cases is more worrisome that either an observed rate of 15% based on 100 cases or an observed rate of 20% based on 10 cases’. And they point out that even when appropriate methodological cautions are observed, risk-adjusted measures must be interpreted very carefully. Summarizing the principal conclusion from their comparative study of hospital severity measures, Ash et al. state that ‘. . . whether hospital B was identified as a particularly high quality hospital . . . depended on which severity measure was used for risk adjustment’. Finally, in the chapter on validity and reliability, Daley, Ash, and Iezzoni include ‘attributional’ validity as one of eight dimensions of validity that should be considered when evaluating risk-adjusted measures. Attributional validity refers to the degree to which changes in outcomes can be attributed to the care being evaluated, and the authors note that in the context of using risk-adjusted measures to motivate practice changes or to monitor provider performance, this is the key dimension of validity. So when promotional materials of commercial vendors describe risk-adjustment models as ‘validated’, they invariably refer to the type of statistical validation discussed in Chapter 10 by Swartz and Ash, not to attributional validity, which would allow consumers of risk-adjusted outcome information to have confidence that differences in observed outcomes truly reflect differences in the care provided.
In her introductory chapter, Dr Iezzoni notes that public and private initiatives involving examination of risk-adjusted outcomes are now too numerous to list. And in her final observations, she comments ‘With relatively few exceptions, little objective independent information is available about what most risk-adjusted outcome results actually mean . . . . The need of purchasers to act aggressively to control costs must be balanced against legitimate questions about the validity of inferences about comparative provider quality from risk-adjusted data’. The third edition of Risk Adjustment for Measuring Health Care Outcome will be a valuable reference both for developers and consumers of risk-adjusted outcome information, and it may help to tilt the balance towards more valid and ultimately more useful information on health care quality and efficiency.
Muskie School of Public Service, University of Southern Maine, Portland, ME, USA, E-mail: jwthomas{at}usm.maine.edu
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||