OUP user menu

Impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care

David J. Ballard, Gerald Ogola, Neil S. Fleming, Brett D. Stauffer, Bradley M. Leonard, Rainer Khetan, Clyde W. Yancy
DOI: http://dx.doi.org/10.1093/intqhc/mzq051 437-444 First published online: 8 October 2010


Objective To determine the impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care.

Design Observational study.

Setting Eight acute care hospitals and two specialty heart hospitals.

Participants All adults (>18 years) discharged from one of the included hospitals between December 2007 and March 2009 with a diagnosis of heart failure, who had not undergone heart transplant, did not have a left ventricular assistive device, and with a length of stay of 120 or less days.

Interventions A standardized heart failure order set was developed internally, with content driven by the prevailing American College of Cardiology/American Heart Association clinical practice guidelines, and deployed systemwide via an intranet physician portal.

Main Outcome Measures Publicly reported process of care measures, in-patient mortality, 30-day mortality, 30-day readmission, length of stay, and direct cost of care were compared for heart failure patients treated with and without the order set.

Results Order set used reached 73.1% in March 2009. After propensity score adjustment, order set use was associated with significantly increased core measures compliance [odds ratio (95% confidence interval) = 1.51(1.08; 2.12)] and reduced in-patient mortality [odds ratio (95% confidence interval) = 0.49(0.28; 0.88)]. Reductions in 30-day mortality and readmission approached significance. Direct cost for initial admissions alone and in combination with readmissions were significantly lower with order set use.

Conclusions Implementing an evidence-based standardized order set may help improve outcomes, reduce costs of care and increase adherence to evidence-based processes of care.

  • quality improvement
  • quality indicators
  • mortality
  • readmissions
  • cardiovascular diseases
  • hospital care


The estimated prevalence of heart failure in adults age 20 and older in the USA was 5.7 million in 2006, with an annual incidence of 10 per 1000 persons over age 65 [1]. Hospital discharges for heart failure rose from 877 000 in 1996 to 1.1 million in 2006—an increase of 26%—and the estimated cost of heart failure in 2009 is $37.2 billion [1]. Substantial evidence has demonstrated reduced morbidity and mortality with angiotensin-converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARBs), β-blocker therapy and other evidence-based treatment modalities in patients with chronic heart failure caused by left ventricular systolic dysfunction, with clinical trials showing improved survival, reduced risk of hospitalization or death and improved New York Heart Association functional class [26]. Despite this evidence and clinical practice guidelines that strongly recommend evidence-based therapies as part of standard care for heart failure, both ACEIs/ARBs and β-blockers (as well as other evidence-based therapies) are underutilized in heart failure patients without contraindications or documented intolerance [7]. Studies of hospitalized heart failure patients show that only ∼60–80% of eligible patients are prescribed ACEIs and β-blockers at discharge [6, 812], 24–55% receive discharge instructions [11, 12], 86–88% receive left ventricular function assessment [11, 12] and 43–72% receive smoking cessation counseling [11, 12]. Hospital Quality Alliance data for patients receiving care in non-federal US hospitals in 2005 showed only 57.3% of heart failure patients received all process of care measures for which they were eligible [13]. Given the high mortality and morbidity rates for heart failure, as well as the fact that it is the primary or secondary cause of approximately 3.6 million hospitalizations annually in the USA [1, 14], inpatient heart failure care is an important target for quality improvement.

Baylor Health Care System (BHCS), an integrated health-care delivery system in North Texas, developed and implemented standardized order sets for heart failure care across 10 hospitals, 8 acute care and 2 specialty hospitals, as part of a system-wide approach focused on heart failure [15]. Previously, we have shown successful improvement in compliance with treatment recommendations and in mortality and costs of care for adult pneumonia patients following widespread adoption of a standardized pneumonia order set [15, 16]. We sought to determine whether a similar effect was achieved following the implementation of a standardized order set for heart failure.


Study setting

BHCS is a not-for-profit, multi-hospital system in Dallas-Fort Worth, Texas incorporating 20 owned, leased, affiliated and short-stay hospitals with more than 103 000 admissions per year. This study included patients treated in the eight BHCS acute care hospitals, as well as the two specialty heart hospitals.


The standardized heart failure order set was developed internally [15], with content driven by the prevailing American College of Cardiology/American Heart Association clinical practice guidelines, and deployed systemwide via the intranet physician portal in December 2007. For a copy of the standardized order set, see Appendix. ‘Order set used’ was made a required field in MIDAS+™—the integrated outcomes, resource and case management system used at all BHCS hospitals—for heart failure patients to facilitate tracking of order set usage. Increasing use of the order set is one of four action items in the BHCS Heart Failure Initiative—along with medication reconciliation focused on heart failure; establishing a continuum of care for inpatient heart failure patients; and standardizing end-of-life care, palliative care and advance directives practices for heart failure. The initial goal was for 80% of health failure admissions systemwide to use the order set by March 2009. For hospitalists—who treat more heart failure patients than any other single specialty—this goal was set to the higher standard of 95%. To spearhead implementation of the Heart Failure Initiative, BHCS established a Heart Failure Task Force and designated a heart failure physician champion for each hospital. Monthly reports, containing order set usage and missed opportunities for use, were provided to physician champions and health-care improvement directors at each facility. The reports included physician-specific use to aid in academic detailing and allow physicians to track individual performance. With increased acceptance of the order set, the health system promoted further adoption by mandating use of the order set, with consequences for omission established my each hospital's Medical Executive Committee.


All adults (>18 years) discharged from one of the included hospitals between December 2007 and March 2009 with a diagnosis of heart failure, who had not undergone heart transplant, did not have a left ventricular assistive device, and with a length of stay 120 or less days (inclusion criteria defined in Appendix A) were eligible for this study. Patients with physician orders for ‘comfort measures only’ recorded in the admitting physician orders or note, consultation notes, emergency department record, history and physical, physician orders or progress notes were excluded from the study, as were patients for whom the heart failure order set use data were missing or recorded as ‘not applicable’ or for whom a heart failure order set other than the BHCS standardized order set was used.

Outcome measures

Primary outcome measures investigated were differences in in-hospital mortality, 30-day mortality, 30-day readmission and compliance with an all-or-none bundle of process of care measures. Compliance of the all-or-none bundle is calculated as the proportion of heart failure patients eligible for the four heart failure measures—discharge instructions, evaluation of left ventricular function, ACEI or ARB for left ventricular systolic dysfunction and smoking cessation counseling—who receive all the measures for which they are eligible [17]. In addition, we examined length of stay and direct cost of treatment.

Data collection

Age, gender, race/ethnicity, admitting BHCS hospital, delivery of core measures for heart failure and information on order set use was collected from MIDAS for each patient. In-patient mortality, readmissions, length of stay and direct costs of care were determined from BHCS administrative data; 30-day mortality was determined from the Social Security Death Master File [18]. Order set use is recorded as a required field in MIDAS+ for heart failure patients, with the options ‘BHCS standard order set used’, ‘no standard order set used’, ‘other standard order set used’ and ‘BHCS order set not applicable.’ The order set was regarded as ‘not applicable’ if:

  1. the diagnosis of heart failure was made after the admission orders were written by the attending physician (not the Emergency Department physician);

  2. both heart failure and pneumonia were present on admission and the physician used the BHCS pneumonia order set; or

  3. the patient was admitted for an elective implantable cardioverter defibrillator placement and other orders sets were in place for the specific procedure.

Statistical analysis

Some patients included in the study population had multiple admissions during the study period. For these patients a single admission record was randomly selected as initial admission in the analysis.

For the unadjusted analyses, binomial tests, t-tests, chi-square tests and robust analysis of variance tests were used to determine association between patient characteristics and order set use, or differences seen in outcomes with order set use.

This being an observational study, patients were not randomized into the intervention group. A propensity score adjustment approach was used to reduce the impact of selection bias on the association between order set use and outcomes of interest. The propensity score, the conditional probability of a patient being treated with the order set given the patient characteristics [age, gender, race, risk of mortality/severity of illness, type of physician (hospitalists vs. non-hospitalists), facility and month of discharge], was determined from a multivariable logistic regression model, and then used as a covariate in the adjusted effect model. The risk of mortality and severity of illness were used independently depending on the outcome evaluated. For clinical outcomes, for example mortality, the risk of mortality variable was included in the model while for resource utilization, for example the cost of care, severity of illness variable was included.

Propensity score adjusted effects of the order set on mortality and core measures compliance were modeled using multivariable logistic regression model. Length of stay was modeled using a multivariable robust regression due to high skewness in its distribution. Readmission was modeled using competing risk regression analysis [19] implemented in R software [20]. All patients who died during their hospitalization were excluded from the readmission analysis. Death after hospitalization was considered a competing risk event for readmission.

Our approach addresses two important aspects of cost data that are non-normal: (i) they are typically right skewed (with a thick tail) as some patients (beyond simply outliers) incur substantial health-care costs compared with the mean); and (ii) a substantial number of patients have no readmissions and associated costs. This two part-model with a logarithmic distribution on the conditional (re-admitted patient) costs appropriately addresses the lack of normality and specific distribution of the cost data.

Direct costs were modeled using a two-step estimation method. In the first step, the probability of readmission was modeled based on a propensity score adjusted logistic regression model; in the second step, direct costs were modeled based on the log-link function with the gamma distribution, given that the readmission had occurred [19, 21]. The product of these two steps creates the estimate of the readmission costs. Subsequently, the method of recycled predictions [21] was employed to appropriately consider underlying differences between the order set and non-order set groups and essentially create a balanced design. Initial admission direct costs and readmission direct costs were predicted from the modeled equations based on two scenarios: (i) every patient received the heart failure order set; and (ii) every patient did not receive the heart failure order set. The difference between these two predictions constitutes the predicted mean differences in initial admission and readmission costs.

Total heart failure direct costs were estimated as part of a three-step process: we generated 1000 bootstrap samples and created recycled predictions to estimate differences with and without order set use in direct costs during initial admission (Step 1); probability of all-cause readmission (30-day and 1-year) (Step 2); and, for each bootstrap sample where a readmission actually occurred for the patient, direct costs for the hospital readmission (Step 3). For this last estimate, we created recycled predictions for the entire bootstrap sample, not just for the individuals who had readmissions. Total direct heart failure costs were then estimated as: Embedded Image where mean direct costs estimated in Step 1 represented the order set effect for index admissions; the mean of the readmission probabilities estimated in Step 2, the order set effect for readmissions and the mean direct costs of readmission estimated in Step 3 to estimate the order set effect for total direct costs including readmissions. The standard error for each of these statistics was calculated as the standard deviation of the combined 1000 bootstrap samples. This methodology accounts for two potential effects use of the order set during initial admissions may have on readmissions: the potential impact on probability of readmission and the potential impact on the direct cost of the readmissions themselves.

Analyses were done using SAS 9.2 (Cary, NC) and R version 2.9.0 (2009–04-17) statistical software. P-values less than 5% were considered statistically significant.


A total of 2633 eligible patients were discharged from the 10 included BHCS hospitals between December 2007 and March 2009. Age ranged from 22 to 103 years with a mean age (standard deviation) of 69.1 (15.3) years. Approximately half (50.3% or 1324 of 2633) were females; and 37.5% were non-white (988 of 2633; see Table 1).

View this table:
Table 1

Descriptive summary statistics of heart failure patients discharged from Baylor health care system (BHCS) hospitals from December 2007 to March 2009

There was significant variation in order set use at all hospitals combined by month (P < 0.01), ranging from 15 to 80%, with use generally increasing over time (see Fig. 1). Table 2 shows the results of the unadjusted comparison of order set by patient demographics and all Patient Refined Diagnosis Related Group (APR-DRG) risk of mortality and severity of illness. The use of the order set differed significantly by severity of illness class (P= 0.02)—with patients in the highest severity class being least likely to receive the order set—and approached significance by risk of mortality class (P = 0.07) (Fig. 2).

View this table:
Table 2

Unadjusted comparison of BHCS order set use by APR-DRG risk of mortality and severity of illness for heart failure patients discharged from BHCS hospitals from December 2007 to March 2009

Figure 1

Proportion of heart failure patients who received the Baylor health care system (BHCS) Standardized Order Set by month among patients discharged from BHCS' eight acute care hospitals and two specialty heart hospitals from December 2007 to March 2009.

Figure 2

Percentage of heart failure patients treated with the Baylor health care system-heart failure (BHCS-HF) Standardized Order Set by facility from December 2007 to March 2009 (sorted by facility total number of cases).

Table 3 shows the unadjusted results of comparisons between patients who did and did not receive the BHCS heart failure order set on in-hospital and 30-day mortality, 30-day readmission, core measures compliance, length of stay and direct cost. There were 183 (7.0%) deaths within 30-days of admission (either in-hospital of following discharge) and 60 (2.3%) in-hospital deaths. Statistically significant differences were observed for in-hospital mortality, core measures compliance and direct costs, with patients receiving the order set having lower mortality rates, smaller direct costs and greater core measures compliance. Following risk adjustment, the effect of the standardized heart failure order set on in-hospital mortality and core measures compliance was retained (see Table 4). The reductions in 30-day mortality and 30-day readmission bordered on significance [odds ratio (95% confidence interval) = 0.81(0.58; 1.13)].

View this table:
Table 3

Unadjusted comparison of BHCS order set vs. no order set on mortality, readmission, core measures compliance and length of stay for heart failure patients discharged from BHCS hospitals from December 2007 to March 2009

View this table:
Table 4

Unadjusted and adjusted effect of BHCS-HF order set on mortality, readmission, core measures compliance, length of stay and direct cost for heart failure patients discharged from BHCS hospitals from December 2007 to March 2009

The decrease in length of stay with order set use approached but did not achieve significance. All measures of direct costs examined except 30-day and 1-year readmission direct costs showed significant decreases with order set use.


Summary of results

During the first year of implementation, the use of the BHCS heart failure order set reached 60% of eligible cases systemwide and increased to >70% by March 2009. In unadjusted comparisons, the use of the BHCS order set was associated with significantly lower in-hospital mortality, smaller direct costs and significantly higher core measures compliance. Importantly, following risk adjustment, differences in in-hospital mortality and core measures compliance remained statistically significant. These findings demonstrate that wide deployment of a standardized evidence-based heart failure order set is associated with improved in-hospital outcomes. Based on the observed mortality rates with and without order set use, for every 85 heart failure patient encounters in which the BHCS order set is used, one in-hospital death is prevented. Significant direct cost savings, overall (initial admission + re-admissions) and when initial admissions were examined in isolation, were also observed with order set use. Decreases in 30-day mortality and 30-day readmission among patients receiving the order set bordered on significance.


As this was an observational study rather than a randomized trial, it is possible that order set use was influenced by unmeasured patient or hospital characteristics, potentially under- or over-estimating the impact of order set use. Counterbalancing this limitation is the fact that observational studies based on real-world clinical data provide a realistic estimate of the effect that may be expected in similar real-world settings. To account for the differences between the patients that did and did not receive the order set, we performed risk-adjusted analyses based on a propensity score approach, which also accounted for the variation in order set use by facility and time observed in this study. However, risk adjustment is limited in that it can only account for measured confounders and does not ensure a balanced distribution of all covariates among the study subjects such as randomization provides. Since only administrative data were available, we were unable to adjust for patient's clinical characteristics and we cannot eliminate the possibility of selection bias, with ‘healthier’ patients who fit most easily within standardized treatment guidelines being more likely to receive the order set. Such selection bias leads to overestimation of the intervention's effects. On the other hand, since all clinicians had access to the order set and may have used it for some heart failure patients but not others, it is also possible that treatment of the ‘no order set’ group was influenced or contaminated by the clinicians’ exposure to and incorporation of the items included in the order set. If so, the results demonstrated here may underestimate the full benefit of order set implementation.

Our study population represents the subset of heart failure patients eligible for the CMS heart failure core measures and excludes any patient who was admitted with both heart failure and another diagnosis for which a BHCS standardized order set was available (pneumonia or sepsis) and the physician used the order set for that diagnosis. Additionally, all data were from a single hospital system: data regarding readmissions to other facilities were not available, and the estimates of mortality and cost savings may not be broadly applicable to other systems, hospitals or geographical regions.

Comparison to the literature

Adoption of the BHCS heart failure order set is comparable to that reported for order sets implemented in the Grey Bruce Health Network in Canada (∼40% systemwide after 6 months) [22] and mirrored the 72% adoption of an emergency department heart failure clinical pathway 14 months post-implementation at a single suburban tertiary care hospital [23].

Similarly, previous studies have shown similar improvements in heart failure outcomes following implementation of hospital-based tools to increase standardization. The Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure (OPTIMIZE-HF) provided 259 participating hospitals with evidence-based practice algorithms, critical pathways, standardized orders, discharge checklists and other tools to assist with the management of heart failure. The OPTIMIZE-HF intervention resulted in substantial increases in the provision of complete discharge instructions, assessment of left ventricular function, adherence to both ACEI/ARB and β-blocker therapy, and smoking cessation instructions. This process of care improvement strategy was associated with a decrease in risk-adjusted in-hospital deaths, 90-day mortality and a decrease in the combined endpoint of readmission and mortality [10]. Clinical pathways for heart failure have also been associated with an ∼70% increase in the eligible patients receiving ACEIs [24]; and significant increases in delivery of heart failure clinical care measures, with concomitant decreases in total admissions, readmissions and inpatient mortality [25].

Our study adds to the strength of the evidence favoring implementation of standardized order sets to increase compliance with clinical guidelines for heart failure and positively impact clinical outcomes. Improved heart failure outcomes have previously been demonstrated with tools to increase the use of evidence-based orders [10, 26]. Narrowing treatment choices to reduce variation in care through the use of evidence-based medicine simultaneously supports smooth workflow and safe practices, and reduces risk of errors [27, 28]. A recent publication emphasizes that outcome improvements can be achieved when clinicians practice in accordance with clinical guidelines [29].

The study results are also consistent with the observation that collection of performance data and feedback to physicians and other decision-makers is an essential part of quality improvement [30]. Monitoring of order set utilization and feedback of usage using academic detailing and monthly reporting to hospital leadership was an essential component of promoting order set adoption in our study.

The decrease seen in the direct cost measures reflects observations in previous reports. For example, implementation of a heart failure clinical pathway in a 376-bed community hospital lowered median costs of hospitalization from $4500 to $2,250 following stratification of patients into moderate and severe comorbidity groups [25]. Given that inpatient hospitalization costs account for 50–70% of the health costs of patients with heart failure in Western industrialized nations [1, 31, 32], tools that decrease hospitalization costs likely have substantial impact on the overall cost of heart failure care.


Improving heart failure outcomes and decreasing costs associated with heart failure care are of pressing importance as the aging of the population and the decreasing mortality rates for ischemic cardiac events are combining to increase both incidence and prevalence of this condition [33]. In our study, analysis of administrative data showed improved clinical and financial outcomes in a large integrated health system associated with the deployment of a standardized heart failure order set. In addition to possible clinical benefits of reduced inpatient mortality for heart failure patients, the potential cost savings demonstrated are of timely importance given the current market challenges, and the growing demands to control escalations in the cost of care. Given the recently demonstrated disconnection between performance on the CMS core measures for heart failure and patient outcome measures [34], hospitals should consider broader-reaching quality improvement tools that span the full spectrum of care during the patient's hospitalization. Our results suggest that implementation of evidence-based standardized order sets may be an accessible tool whereby hospitals can improve performance in a relatively short period of time.


Our study provides evidence that health systems with committed leadership and strategically dedicated resources can successfully deploy and foster the rapid adoption of a standardized order set across multiple facilities. Order sets should be based on evidence and developed to be compliant with international guidelines for the management of heart failure. Engagement of front-line providers, including physician champions and other local leaders at each facility, is essential to ensuring application of the order set in the majority of appropriate cases.

While recognizing the limitations of our observational study, we believe that the use of a standardized heart failure order set for all of the 1 million heart failure admissions in the USA annually [14] could result in a significant improvement in heart failure care while reducing costs. Based upon an average in-hospital mortality rate of 2.97% estimated from a large sample of university hospitals (data from the University HealthSystem Consortium) [35], the 51% decrease in risk of in-hospital mortality and $1909 decrease in total direct cost (initial hospitalization plus 1-year all-cause readmission costs) associated with use of a standardized heart failure order set use within BHCS translate into annual savings of 15 147 in-hospital deaths and $1.9 billion dollars nationally. Further research using clinical data is needed to confirm our results, and ideally, this intervention should be tested in a multicenter randomized controlled trial.


View Abstract