International Journal for Quality in Health Care Advance Access originally published online on March 23, 2005
International Journal for Quality in Health Care 2005 17(4):347-355; doi:10.1093/intqhc/mzi034
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The insufficiency of evidence to establish the business case for quality
1 University of North Carolina at Chapel Hill, Department of Health Policy and Administration, Chapel Hill, NC, USA and 2 University of North Carolina at Chapel Hill, Cecil G. Sheps Center for Health Services Research, Chapel Hill, NC, USA
Purpose. To determine whether a positive financial return on investment for quality-enhancing interventions is more likely for particular health conditions, in specific organizational settings, or with the use of particular interventions.
Data sources. Electronic search of MEDLINE®.
Data extraction. Search keywords included: business case, cost-effectiveness, cost-benefit, return on investment, costs, cost savings, quality, quality improvement, and program evaluation.
Results. Only 15 of 1968 articles identified contained sufficient information on both the costs of implementing quality-enhancing interventions and the resultant changes in costs of care or revenues to permit the calculation of a return on investment.
Conclusions. Scant attention is currently paid in the quality-of-care literature to the cost of implementing quality-enhancing interventions. To understand which quality-enhancing interventions are likely to produce positive returns on investments, data collection and analysis must include tracking the investment and operating costs of implementing the intervention as well as the changes in revenues and costs that result from the intervention.
Keywords: cost of care, health services utilization, quality-enhancing intervention, quality of care, return on investment
Address reprint requests to Kerry Kilpatrick, University of North Carolina at Chapel Hill, Department of Health Policy and Administration, Chapel Hill, NC 27599-7411, USA. E-mail: kerry_kilpatrick{at}unc.edu
Accepted for publication February 4, 2005.
Deficiencies in the quality of health care remain prevalent despite an increasing body of evidence to guide the implementation of proven quality interventions (QIs) [1–3]. Policy-makers, payers, and employers continue to express their frustration that quality-enhancing interventions of demonstrated effectiveness are not being implemented on a broad basis. Even after decades of careful evidence-based practice research, one of the principal reasons that hospitals, health care delivery systems, and individual providers in the United States give for not implementing promising health care quality-enhancing interventions is that no ‘business case’ for quality can be made (see Box 1 for an operational definition of the term business case [4]). Recent case studies confirmed that, in the absence of a convincing business case, QIs have a low probability of widespread adoption and a lower probability of being sustained over time [4].
| Box 1 A business case for a health care improvement intervention exists if the entity that invests in the intervention realizes a financial return on its investment in a reasonable time frame, using a reasonable rate of discounting. This may be realized in ‘bankable dollars’ (profit), a reduction in losses for a given program or population, or avoided costs. In addition, a business case may exist if the investing entity believes that a positive indirect effect on organizational function and sustainability will accrue within a reasonable time frame.
|
Prompted by these findings, we attempted to determine from the published literature the characteristics of quality-enhancing interventions that might be associated with positive returns on investment for the business entity making the investment. Dimensions of interest are the clinical or other setting for the quality-enhancing intervention, the health condition targeted, and the type of intervention (ranging from innovative use of personnel to patient education and counseling to guideline implementation). Our efforts were mostly unsuccessful: few published articles on quality-enhancing interventions give adequate information on the cost of the intervention or the monetary gains from the intervention.
Rather than also search the ‘gray’ literature, we restricted our search to the peer-reviewed literature for a number of reasons. The peer-reviewed literature is more likely to contain studies that would typically have stronger methods, be more generalizable, and be less prone to bias than studies specifically developed by corporations or even by governments to demonstrate the value of quality-enhancing interventions. Another very practical reason for restricting our search to the peer-reviewed literature is that a search of the gray literature would have been inordinately time consuming and expensive. Further, no universally agreed methods exist for searching the gray literature.
We document here our findings, speculate on possible explanations, and call for more attention to the need to gather data on both the intervention costs and resultant changes in revenues and costs of care to move the quality-of-care agenda forward.
 |
Methods |
|---|
 Top Methods ResultsDiscussionAcknowledgementsReferences |
|---|
Search process
We conducted searches of the National Library of Medicine (NLM) database MEDLINE® in two rounds. In the first round, covering 1966 to 1 August 2003, we sought articles that (i) identified a specific QI and (ii) reported cost savings or return on investment (ROI) results. We used medical subject headings (MeSH) terms or keywords, including ‘business case’, ‘quality’, ‘quality indicators’, ‘quality assurance (health care)’, ‘costs and cost analysis’, ‘return on investment’, ‘medical errors’, ‘safety management’, and ‘diagnostic errors’. This initial search strategy produced a low, and often irrelevant, yield of articles, so we expanded it and reran it to cover the period from 1 January 1980 to 31 August 2004.
In the second round, we expanded the search terms to include a variety of keywords, MeSH headings, or combinations that discussed the impact of QIs (Table 1). This strategy considerably broadened the pool of potential articles, although we also found many papers that made no attempt to quantify the revenue or cost impact of QIs. During these searches, we discovered that the concept of ‘business case’ is not yet sufficiently widespread in the health care literature (or in MeSH trees and keywords) to yield enough studies of interest.
|
Because our focus was principally on QIs within the United States, we did not formally search other databases such as the electronic Cochrane Library and EMBASE databases. However, we did informally review both these sources to ensure that we did not miss a large number of relevant articles. We used a search strategy similar to the one we used with PubMed. We found that most articles retrieved from the Cochrane Library would not meet our inclusion criteria because they were classic cost-effectiveness analyses, lacked cost data, or lacked a clear description of the QI. Using the keyword ‘return on investment’, we found only six articles with the potential to be included. Moreover, we were able to retrieve these with PubMed.
The two search strategies resulted in a total of 1968 articles to review. From the 1968 articles, we were able to select 234 articles by eliminating those articles that presented only a description of the QI with no associated cost and patient quality data.
Data extraction
To select articles that most closely addressed the business case for quality, we established a priori inclusion and exclusion criteria (Table 2). We applied these criteria to the 234 articles that appeared most promising. For completeness and background, we opted to include in this set some review articles that addressed the general issue of the business case although they might not have cited specific findings. One of two senior investigators and one of two research assistants independently applied the inclusion and exclusion criteria to the 234 articles.
|
Although ultimately we were interested in interventions that improved the outcomes of care, we did include papers that demonstrated improved processes of care for which there was sufficient evidence associating those processes with better outcomes over the long term.
As implied by the inclusion and exclusion criteria, we did not include articles that reported cost-effectiveness analyses or cost-benefit studies of clinical trials of drugs, devices, or specific diagnostic or therapeutic procedures. Although using advanced drugs or procedures generally enhances quality of care, we sought evidence of pay-offs from interventions that aimed to enhance quality of care directly and that health care delivery organizations actually implemented. For this reason, we did not review cost-effectiveness analyses that compared competing treatment alternatives under idealized conditions using metrics such as quality-adjusted life years that would not resonate with financial officers or managers entrusted with making actual investment decisions.
Also, for several reasons we excluded studies that reported on the results of generalized quality improvement programs. Firstly, a QI program may involve myriad traditional quality assurance initiatives and quality improvement interventions encompassing several clinical programs and sites of care. Secondly, although we recognize that some QI programs have demonstrated returns on investment, we do not know how to separate the success of the intervention per se from leadership characteristics, the quality culture of the organization, and the financial imperatives facing the quality team—all factors making generalizability exceedingly difficult.
|
Results |
|---|
|
TopMethodsResultsDiscussionAcknowledgementsReferences |
|---|
After reviewing all 234 articles, we found only 15 that passed the inclusion and exclusion screen. These provided sufficient information on the consequences of the intervention and on the cost of implementing the intervention to determine a return on investment for the intervention [5–19]. Not all 15 articles computed a return on investment, but we included any article that gave sufficient information on the cost of the intervention and the financial outcomes to enable other investigators to compute a return on investment. Table 3 summarizes the study designs and their findings.
|
Clearly, from a set of 15 studies derived from a very large body of literature devoted to quality of care, we can draw no firm conclusions regarding the types of interventions, the patient conditions, or treatment settings that are likely to generate positive business cases for quality-enhancing interventions. No patterns emerged for setting or geographical area in which the quality-enhancing intervention was studied. Four papers dealt with asthma, but this likely reflects the significance of this disease (prevalence, morbidity, mortality, and the cost of care) as well as a growing consensus regarding appropriate processes of clinical treatment and associated patient outcomes.
More relevant to our ‘business case’ focus is that nine of the 15 included articles dealt with some aspect of patient self-management of chronic conditions. This finding does not mean that self-management interventions for chronic conditions necessarily yielded better business cases than other quality-enhancing strategies. We suspect, rather, that researchers interested in the efficacy of self-management strategies may be more attuned to the need to establish an economic or business case for them, for instance to encourage health system managers or policy-makers to invest scarce resources in patient self-management initiatives. In addition, self-management interventions are increasingly well understood so they may seem more traditional than many contemporary quality-enhancing intervention ideas and thus lend themselves more readily to return on investment thinking and analysis. By contrast, the organizational, financial, and other considerations inherent in implementing innovative quality-enhancing interventions may be more daunting than for more traditional quality-of-care steps, such as upgrading diagnostic test equipment or adding medications to a formulary.
|
Discussion |
|---|
|
TopMethodsResultsDiscussionAcknowledgementsReferences |
|---|
From our initial combined yield of 1968 articles, we eventually retained only 15 studies: less than 1% of the articles that met our final inclusion criteria. Evidently, authors publishing findings about the impact of QIs either do not collect data to establish a business case for the intervention or consider that information of insufficient interest to report. We postulate that several factors may militate against reporting such information.
Firstly, investigators who conduct and publish studies of interventions meant to enhance health services quality are typically not trained to be concerned with the business case for the interventions. Historically, contributors to quality-of-care literature are clinicians, health services researchers, or quality-of-care professionals whose motivation to date has been driven largely by what they considered to be a compelling imperative to improve the process of care delivery and patient outcomes. This has been seen, explicitly or implicitly, to be a sufficient imperative, and often a focus on costs is taken by authors (or editors) to mean that quality of care was of secondary importance. Secondly, internal sponsors of quality-enhancing interventions (e.g. those within a hospital, health plan, or delivery system) may not be interested in implementation costs or may not require such information; the additional effort required to track such costs would thus be neither budgeted nor expended. Thirdly, an external agency or foundation may have funded the quality-enhancing intervention; in this situation, costs to the organization implementing the quality-enhancing intervention may have been minimal, and thus investigators would likely not track or report them. Fourthly, a series of technical or methods challenges may pose obstacles for this type of work; for example, better research methods to evaluate the financial impact of quality-enhancing interventions are needed, and current cost-accounting systems of hospitals (let alone of private physician practices or community-based entities) may not be adequate to capture the required micro-cost and revenue data. Finally, academic journals and their readership have not required cost information as a requisite for publishing the article.
Whatever the reasons for the lack of evidence, we underscore the paucity of evidence to support a business case for quality in the peer-reviewed literature. Although payers, purchasers, and policy-makers increasingly accept that quality of care is now a necessary rather than a discretionary attribute of health care delivery, the lack of data to illuminate the cost and revenue impacts of investment in various QIs inhibits the speed and predictability of their adoption and diffusion [4]. With the increasing interest in ‘pay-for-performance’ incentives, providers will look to the open literature for guidance on which interventions are more likely to result in positive returns on investment. Furthermore, for an organization to invest its limited capital resources optimally, it should be armed with defensible estimates of the expected returns on investment for competing quality-enhancing interventions.
To address some of these issues, we offer two ideas that might increase the number of articles linking quality-enhancing interventions with the costs of implementation. One is for foundations and government agencies that fund studies of quality-enhancing interventions to mandate that investigators routinely gather and report financial information that could be used by interested parties to analyze the business case for the interventions themselves. A second is for institutions that support their own work on quality-enhancing interventions to underwrite special studies aimed specifically at investigating the business case for quality. Examples of these steps can be found in the work undertaken or financed by the Commonwealth Fund, the Center for Health Management Research, and the Center for Health Care Strategies.
All heath care delivery organizations—whether private sector or governmental—have an interest in knowing the pay-offs from QIs. Our hope in publishing this article is that it will encourage others to include specifically the cost of the quality-enhancing intervention, as well as its financial consequences, in their published work. Only if the published literature expands will we be able to achieve our original objective of determining where positive returns on investment for quality-enhancing interventions are most likely to occur.
|
Acknowledgements |
|---|
|
TopMethodsResultsDiscussionAcknowledgementsReferences |
|---|
We particularly wish to acknowledge the contributions of Anne-Marie Audet, MD, MSc, both for her help with the original conceptualization of the research and for her suggestions on this manuscript. We would also like to express our appreciation for the comments and suggestions of two anonymous reviewers. Their insights have helped us clarify some key points in the paper. This research was supported by the Commonwealth Fund, New York, New York, USA.
|
References |
|---|
|
TopMethodsResultsDiscussionAcknowledgementsReferences |
|---|
- McGlynn EA, Asch SM, Adams J et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348: 2635–2645.
[Abstract/Free Full Text] - Leatherman C, McCarthy D. Quality of Health Care in the United States: A Chartbook. New York: Commonwealth Fund, 2002.
- Leatherman C, McCarthy D. Quality of Health Care for Children and Adolescents: A Chartbook. New York: Commonwealth Fund, 2004.
- Leatherman S, Berwick D, Iles D et al. The business case for quality: case studies and an analysis. Health Aff (Millwood) 2003; 22: 17–30.
[Abstract/Free Full Text] - Bogucki B, Jacobs BR, Hingle J. Computerized reminders reduce the use of medications during shortages. J Am Med Inform Assoc 2004; 11: 278–280.[CrossRef][Web of Science][Medline]
- Cimino MA, Rotstein RM, Moser JE. Assessment of cost-effective antibiotic therapy in the management of infections in cancer patients. Ann Pharmacother 1994; 28: 105–111.[Abstract]
- Gautney LJ, Stanton MP, Crowe C et al. The emergency department case manager: effect on selected outcomes. Lippincotts Case Manag 2004; 9: 121–129.[CrossRef][Medline]
- Greineder DK, Loane KC, Parks P. Reduction in resource utilization by an asthma outreach program. Arch Pediatr Adolesc Med 1995; 149: 415–20.
[Abstract/Free Full Text] - Hatoum HT, Witte KW, Hutchinson RA. Patient care contributions of clinical pharmacists in four ambulatory care clinics. Hosp Pharm 1992; 27: 203–6, 208–209.[Medline]
- Kelly CS, Morrow AL, Shults J et al. Outcomes evaluation of a comprehensive intervention program for asthmatic children enrolled in Medicaid. Pediatrics 2000; 105: 1029–1035.
[Abstract/Free Full Text] - Lorig KR, Sobel DS, Stewart AL et al. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: a randomized trial. Med Care 1999; 37: 5–14.[CrossRef][Web of Science][Medline]
- Lucas DO, Zimmer LO, Paul JE et al. Two-year results from the asthma self-management program: long-term impact on health care services, costs, functional status, and productivity. J Asthma 2001; 38: 321–330.[CrossRef][Web of Science][Medline]
- O’Connell JM, Johnson DA, Stallmeyer J et al. A satisfaction and return-on-investment study of a nurse triage service. Am J Manag Care 2001; 7: 159–169.[Web of Science][Medline]
- Peters L, Simon EP, Folen RA et al. The COPE program: treatment efficacy and medical utilization outcome of a chronic pain management program at a major military hospital. Mil Med 2000; 165: 954–960.[Web of Science][Medline]
- Ringen K, Anderson N, McAfee T et al. Smoking cessation in a blue-collar population: results from an evidence-based pilot program. Am J Ind Med 2002; 42: 367–377.[CrossRef][Web of Science][Medline]
- Rogowski JA, Horbar JD, Plsek PE et al. Economic implications of neonatal intensive care unit collaborative quality improvement. Pediatrics 2001; 107: 23–29.
[Abstract/Free Full Text] - Sackett K, Pope RK, Erdley WS. Demonstrating a positive return on investment for a prenatal program at a managed care organization. An economic analysis. J Perinat Neonatal Nurs 2004; 18: 117–127.[Web of Science][Medline]
- Simon GE, Katon WJ, VonKorff M et al. Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psychiatry 2001; 158: 1638–1644.
[Abstract/Free Full Text] - Wheeler JR, Janz NK, Dodge JA. Can a disease self-management program reduce health care costs? The case of older women with heart disease. Med Care 2003; 41: 706–715.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  V. Cooke, G. Arling, T. Lewis, K. A. Abrahamson, C. Mueller, and L. Edstrom Minnesota's Nursing Facility Performance-Based Incentive Payment Program: An Innovative Model for Promoting Care Quality Gerontologist, October 21, 2009; (2009) gnp140v1. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Arling, C. Job, and V. Cooke Medicaid Nursing Home Pay for Performance: Where Do We Stand? Gerontologist, October 1, 2009; 49(5): 587 - 595. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Leatherman and K. Sutherland Designing national quality reforms: a framework for action Int. J. Qual. Health Care, December 1, 2007; 19(6): 334 - 340. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. E S Brown, M. H Chin, and E. S Huang Estimating costs of quality improvement for outpatient healthcare organisations: a practical methodology Qual. Saf. Health Care, August 1, 2007; 16(4): 248 - 251. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. L. Reiter, K. E. Kilpatrick, S. B. Greene, K. N. Lohr, and S. Leatherman How to develop a business case for quality Int. J. Qual. Health Care, February 1, 2007; 19(1): 50 - 55. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T Smith From rhetoric to reality: the need for external quality initiatives to understand and better relate to organisational inner worlds Qual. Saf. Health Care, December 1, 2005; 14(6): 459 - 461. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||