OUP user menu

A comparison of quality of care indicators in urban acute care hospitals and rural critical access hospitals in the United States

M. Nawal Lutfiyya, Deepa K. Bhat, Seema R. Gandhi, Catherine Nguyen, Vicki L. Weidenbacher-Hoper, Martin S. Lipsky
DOI: http://dx.doi.org/10.1093/intqhc/mzm010 141-149 First published online: 18 April 2007


Objective Two recent Institute of Medicine reports highlight that the quality of healthcare in the US is less than what should be expected from the world's most extensive and expensive healthcare system. This may be especially true for critical access hospitals since these smaller rural-based hospitals often have fewer resources and less funding than larger urban hospitals. The purpose of this paper was to compare quality of hospital care provided in urban acute care hospitals to that provided in rural critical access hospitals.

Design Cross-sectional study analyzing secondary Hospital Compare data. T-test statistics were computed on weighted data to ascertain if differences were statistically significant (P = 0.01).

Setting Centers for Medicare and Medicaid Services hospitals.

Participants US Acute Care and Critical Access hospitals.

Main outcome measures Differences between urban acute care hospitals and rural critical access hospitals on quality care indicators related to acute myocardial infarction, heart failure and pneumonia.

Results For 8 of the 12 hospital quality indicators the differences between urban acute care and rural critical access hospitals were statistically significant (P = 0.01). In seven instances these differences favored urban hospitals. One indicator related to pneumonia favored rural hospitals

Conclusions Although this study focused on only three disease states, these are among the most common clinical conditions encountered in inpatient settings. The findings suggested that there may be differences in quality in rural critical access hospitals and urban acute care hospitals and support the need for future studies addressing disparities between urban acute care and rural critical access hospitals.

  • disparities in hospital care
  • quality care
  • quality indicators
  • rural hospital care
  • urban hospital care
  • US hospital care


Two recent Institute of Medicine reports [12] highlight that the quality of healthcare in the US is less than what should be expected from the world's most extensive and expensive healthcare system. Other studies also point out that the quality of American healthcare might be deficient [34] and that many patients do not receive care consistent with the latest scientific knowledge or accepted best practice [5].

These reports and study findings stimulated interest for health agencies, consumers, and physicians to assess and improve healthcare quality. The response to date has focused primarily upon increased measurement in the form of reporting clinical indicator data in the US [68]. Despite efforts to use this information to gauge quality and to improve outcomes, there is still uncertainty about the quality of hospital care and how they compare to each other. This may be especially true for the approximately 1200 rural critical access hospitals in the US because these smaller rural-based hospitals often have fewer resources and less funding than larger urban hospitals. Skilled personnel may also be an issue since only 10% of physicians serve rural populations [9] and less than one-third the number of specialists per capita practice in rural settings versus urban settings [10]. Compounding these factors are the challenges presented in caring for rural residents who tend to be older than urban dwellers, have higher rates of chronic illnesses, and exhibit poorer health behaviors, such as higher rates of smoking and obesity and lower rates of exercise than their urban counterparts [11]. Given these combinations of circumstances it is hard not to speculate that the quality of rural hospital care might not compare favorably to urban hospitals [12].

Previous studies focusing on one disease state [1317] such as acute myocardial infarction (AMI) supports the contention that healthcare quality in rural settings might be inferior to that found in urban settings [13]. Researchers using indicators such as personnel, equipment, organizational systems and quality improvement activities [13] found that patients experiencing an AMI in rural hospitals in Kansas were less likely to receive standard care and tended to have worse outcomes compared with their urban counterparts [14]. Another study examining rural hospital Medicare patients with an AMI had similar findings with a significantly higher adjusted 30-day post AMI death rate than those in urban hospitals [15]. In contrast, other studies have documented superior outcomes in rural settings for common procedures such as low-risk obstetrics [18] and a lower incidence overall of medical errors or injuries [19] suggesting that there may not be a disparity in rural hospital care.

Unfortunately, most previous studies exploring the differences in outcomes of patients hospitalized in rural hospitals are limited because they either focused on one geographical region, a single disease state, a particular segment of the population, or examined rural regions alone with no comparisons to urban or metropolitan populations [1417, 2021]. Other studies used surrogate measures for quality of care such as staffing, organizational systems and admission rates which have not always been proven to be accurate measures of clinical outcomes.

The purpose of this research was to compare the quality of hospital care provided in urban acute care hospitals to that provided in rural critical access hospitals using a national database. This study builds upon existing work by comparing quality of care indicators that have been proven to decrease morbidity and mortality for AMI, heart failure and pneumonia. The disease states, cardiovascular disease and pneumonia, examined in this study were of particular interest because some of the most effective and immediate treatments are equally accessible in rural as well as urban hospitals, making them a useful standard for assessing quality of care [14].


Data source

The data for this project were generated by the Centers for Medicare and Medicaid Services, an agency of the US Department of Health and Human Services along with the Hospital Quality Alliance. The Hospital Quality Alliance is a public–private collaboration established to promote reporting on hospital quality of care. The Hospital Quality Alliance consists of organizations that represent consumers, hospitals, physicians, employers, accrediting organizations and federal agencies. The Hospital Quality Alliance: improving care through Information was created in December 2002. The major instrument for achieving this goal is the consumer-oriented Hospital Compare website [22].

The data made available by Hospital Compare are provided voluntarily by short-term and largely urban acute care hospitals and rural small, remote critical access hospitals. Urban acute care hospitals are defined as short-term hospitals that provide inpatient medical care and other related services for surgery, acute medical conditions or injuries (usually for a short term illness or condition). All but 2.6% of urban acute care hospitals are located in urban areas as defined by the US Department of Agriculture's Economic Research Service's rural-urban continuum codes [23] and the urban acute care hospitals were considered as urban for the purpose of this analysis. In contrast, to be designated as a critical access hospital, a hospital must be located in a rural area, provide 24-hour emergency services; have an average length-of-stay for its patients of 96 hours or less; be located >35 miles (or >15 miles in areas with mountainous terrain) from another hospital or be designated by its State as a ‘necessary provider’ and have no more than 25 beds. In this instance rural areas comprise open country and settlements with fewer than 2500 residents [23].

Beginning with 2004 discharges, eligible short-term urban acute care hospitals could elect to report quality data in return for an incentive payment. To obtain increased payment, the provision required eligible hospitals to report on an initial set of 10 quality performance measures and to agree to have their data publicly available. According to the Hospital Quality Alliance, the majority of these hospitals agreed to participate in the Hospital Quality Alliance voluntary reporting initiative. Beginning with discharges in the second quarter of 2004 (April–June), hospitals participating in the Hospital Quality Alliance could also elect to submit data on seven additional measures in three conditions (see Table 1). Because the payment incentive was related only to the submission of the initial 10 measures, Hospital Quality Alliance-participating hospitals could chose whether to submit the data, and once submitted, whether or not to have the data made publicly available. Data on hospital compare were available for both first and second quarters of 2004 for the initial set of 10 measures. For the seven additional measures, Hospital Compare data were available only from the second quarter of 2004.

View this table:
Table 1

Hospital quality indicators 2005 US Department of Health and Human Services Hospital Quality indicators for AMI, heart failure and pneumonia

Rural critical access hospitals were not eligible for an incentive payment. These hospitals could elect to submit data for any or all of the 17 measures in the measurement set and could elect to report data but not have it publicly available. All 423 rural critical access hospitals reporting data were used for analysis. This represents an estimated 35% participation rate since there are approximately 1200 rural critical access hospitals.

The data collection approach for submitting data to the website was primarily retrospective. Required data elements included both administrative data and medical record documents. Some hospitals preferred to collect their data concurrently by identifying patients in the population of interest. Some hospitals reported data retrospectively. This approach provided opportunity for improvement at the point of care or service. However, complete documentation included the principal and other ICD-9-CM diagnoses and procedure codes, which required retrospective data entry. More detailed information about data quality assurance is available elsewhere [22].

Study design

For the purposes of this study, we examined data from 41 states (see Table 2)—all states submitting data to Hospital Compare that had both acute care and critical access hospitals. A subset of quality measures endorsed by the National Quality Forum process was selected by Hospital Quality Alliance. Table 1 displays the performance measures and their descriptions for the quality indicator measure set. This quality indicator set includes eight measures related to heart attack care, four measures related to heart failure care and five measures related to pneumonia care.

View this table:
Table 2

Hospital type by state included in the analysis 2005 US Department of Health and Human Services Hospital Quality indicators for AMI, heart failure and pneumonia

Raw numbers were abstracted from the publicly available information [22] and entered into a customized database for analysis. All analyses were executed in Statistical Package for Social Scientists (SPSS) 14.0. After examining the raw data, five of the hospital quality measures were discarded because of insufficient amounts of data. These discarded measures were: AMI patients given angiotensin converting enzyme (ACE) inhibitor for left ventricular systolic dysfunction (LVSD), AMI patients receiving thrombolytic agent within 30 minutes of hospital arrival, AMI patients receiving percutaneous transluminal coronary angioplasty (PTCA) within 90 minutes of hospital arrival, AMI patients given smoking cessation advice/counseling, and heart failure patients receiving discharge instructions.

For the remaining 12 hospital quality measures, the data were aggregated by urban setting (urban acute care hospitals) and rural setting (rural critical access hospitals) within each state. The data were then aggregated by urban versus rural for each hospital quality indicator combining all states. To account for differences in number of patients eligible for the indicator by hospital, all observations were weighted in proportion to the total number of eligible patients. A weighting variable was calculated by dividing total number of eligible patients for the indicator in an individual hospital by the mean sample size for the indicator. After applying the weighting variable, a two-tailed t-test for independent samples was calculated for each hospital quality indicator to compare the two independent proportions of rural and urban hospitals. To ensure that comparisons were not significant by chance, alpha was set at 0.01.


Table 1 describes all 17 hospital quality indicators for the three disease states—AMI, heart failure and pneumonia—of which 12 of these were analysed. Of the 4203 hospital studied, 89% (3780) were urban acute care hospitals while 423 or 11% were rural critical access hospitals. Table 2 lists the type of hospital, urban acute care hospital or rural critical access hospital, by state for hospitals reporting data for the 12 quality indicators examined in this study. Of the urban acute care hospitals reporting data on the hospital quality indicators, 85% were accredited by a national organization like the Joint Commission on Accreditation of Healthcare Organizations JCAHO) or the American Osteopathic Association. In comparison, the same could be said of only 35% of the rural critical access hospitals that reported data on the quality measures. Data were not available on specific accreditation organization by hospital for either hospital type.

Table 3 presents the results of the comparison of aggregated hospital quality indicators by urban acute care and rural critical access hospitals. For 8 of the 12 hospital quality indicators the differences between urban and rural hospitals, as tested by t-test were statistically significant (P = 0.01). In seven instances these differences favored urban hospitals and the remaining one, rural hospitals. Indicators favored by urban acute care hospitals included all disease states, while the one indicator favored by the rural critical access hospital setting related to pneumonia. Among the 12 indicators tested for comparison, urban hospitals scored above 90% in four instances, while the only category over 90% for rural hospitals was in the percent of pneumonia patients assessed for oxygenation. Hospital Compare sets the target for all quality measures at 100%.

View this table:
Table 3

Comparison of hospital quality indicators by hospital type 2005 US Department of health and human services hospital quality indicators for ami, heart failure and pneumonia

All differences in the care of an AMI favored urban hospitals. For instance, patients in urban settings were more likely to receive aspirin upon arrival and at discharge and beta-blockers upon arrival. Furthermore, urban hospitals were significantly more likely to assess left ventricular function and to provide smoking cessation education for heart failure patients. In both instances, the percent differences between urban acute care hospitals and rural critical access hospitals were marked.

In contrast to cardiovascular care, rural critical access hospitals performed as well as or better than urban hospitals in four of the five pneumonia-related indicators. Rural critical access hospitals were equally likely to collect a blood culture specimen before administering the first dose of antibiotics and to assess oxygen levels as their urban counterparts. Rural critical access hospitals performed significantly better than urban acute care hospitals on the measure regarding receipt of antibiotics within the first 4 hours of arrival to hospital. In general, all hospitals in all states studied did a poor job at screening for and administering pneumococcal vaccine as needed. The same was true for all hospitals in terms of giving adult smoking cessation advice to pneumonia patients (63.8% for urban acute care hospitals and 53% for rural critical access hospitals). Nevertheless, the rates for providing smoking cessation advice or counseling were statistically different (P ≤ 0.01).


This study found that urban acute care hospitals tended to perform better in more quality indicators than rural critical access hospitals, although rural hospitals were superior in one measure related to pneumonia. In general, heart failure and AMI appear to be more effectively managed in the urban acute care hospital setting compared to critical access hospitals. For example, urban acute care hospitals performed significantly better than rural hospitals in assessing left ventricular function for patients with heart failure. This finding matches previous studies which also demonstrated that rural patients with heart failure did not have left ventricular function assessed as frequently [2429].

This study also revealed that urban acute care hospitals were more likely to provide smoking-cessation counseling for heart failure patients. Since rural residents are more likely to smoke and counseling is a low tech intervention, this seems an area that should be of particular interest for rural hospitals to address and improve. One caution is that the findings may reflect better documentation of smoking cessation counseling rather than having more effective counseling outcomes. Unlike the delivery of aspirin or beta-blockers which can be more confidently assessed, smoking cessation is an indicator that hospitals can be credited for by using check-off boxes on discharge instruction sheets. Since quality scores for urban acute care hospitals are more likely to have financial consequences, perhaps there is a greater incentive for these hospitals to manipulate the system through the use of check off forms for patients discharged with target conditions.

Although there was no significant difference in ACE inhibitor usage for LVSD in heart failure patients, all the examined quality measures for AMI favored urban hospitals. Specifically, patients in urban hospitals were more likely to receive aspirin upon arrival and at discharge and beta-blockers upon arrival, results consistent with the previous studies [13, 14]. The percent difference between urban and rural hospitals was highest on the quality measure regarding prescribing beta-blockers at discharge with compliance fairly high for urban acute care hospitals (91.2%) and much lower at rural critical access hospitals (78.4%).

Several possible explanations exist for why urban acute care hospitals perform better than rural critical access hospitals for cardiovascular care. One reason may be that many critical access hospitals do not have 24-hour coverage by an emergency medicine specialist in their emergency departments and that access to cardiologists may be more limited. Studies show that cardiologists are more likely to prescribe aspirin and beta-blockers than generalists and differences in care may represent an increased likelihood that generalists rather than cardiologists provide care in rural hospitals. In addition, rural hospitals may transfer many of their cardiovascular patients to urban hospitals. There is a well documented connection between volume and quality and it may be that rural hospitals treat too few heart failure or AMI patients to be as proficient as their urban counterparts [16].

In contrast to cardiovascular care, rural hospitals scored better in comparison to urban hospitals in one of five indicators related to pneumonia. Unlike patients with heart disease, patients with pneumonia may be less likely to be transferred resulting in a sufficient volume of pneumonia patients to develop good quality. At the same time rural hospitals deal with fewer total patients in comparison to their urban counterparts, perhaps allowing the medical staff in rural hospitals a greater opportunity to evaluate and treat patients (e.g. timely administration of antibiotics) and adhere to quality measures. In contrast, a busier urban hospital dealing with multiple patients at the same time may compromise their ability to administer antibiotics promptly. Similarly, rural hospitals did somewhat better than urban hospitals in providing pneumococcal vaccination, but the difference did not achieve statistical significance. However, both scores were poor and may reflect the time constraints of the inpatient setting causing providers in both settings to defer this measure for a follow-up outpatient visit.

Of concern is that the indicators examined in this study such as aspirin, beta-blockers, smoking cessation counseling and left ventricular function assessment should be well within the scope of the services provided by rural critical access hospitals. The findings support the need to address these disparities. One possible method to improve quality of care for AMI and heart failure is for critical access hospitals and urban hospitals to conjointly develop guidelines that ensure standard and swift delivery of care [13, 14]. A recent example of this can be found in Minnesota, where rural and urban physicians, community hospitals and emergency medical services partnered with a tertiary care hospital to adopt a common, community-wide protocol [30]. Likewise, there are a number of ways to improve pneumonia care in urban hospitals such as standing immunization orders prior to discharge [31] or implementation of a pneumonia practice guideline. Such guidelines have been associated with a reduction in 30-day mortality among elderly patients with pneumonia [32].

There are several potential limitations to our study. For example, this study only focuses on three disease states and may not reflect overall care. However, these are three of the most common clinical conditions encountered in the inpatient setting and account for about 15% of total admissions. Also, the outcome measures in our analyses were process measures which, although associated with outcomes, could differ from actual patient outcomes. Another potential limitation is that the data are based on voluntarily self-reported data from hospitals which could affect the reported results. Some data were not available especially for rural hospitals perhaps due to under-reporting or willfully not reporting data. There are additional data limitations that extend beyond the self-reported nature of the data. For instance, Hospital Compare (the website providing public access to the hospital quality measure data) is relatively new, becoming accessible to the public in April 2005. Hence, despite considerable time and study devoted to creating both a clinically valid and reliable public hospital performance reporting system, there are insufficient data to analyse trends or comparison performance differences over time. Also, even though Hospital Compare audits submitted data in order to assess whether they are consistent with defined parameters such as sample size, outliers, and missing data to verify that they are consistent and reproducible; only a very small sample of medical records are used for this data check and validation process. Specifically, for each quarter of data submitted, only a random sample of five medical records across all topics are selected for each hospital, regardless of the number of cases submitted. This small of a sample may be insufficient for validation of the data. Finally, this study specifically looked at critical access hospitals—a subset of rural hospitals. This subset of rural hospitals, although substantial, may not reflect the care provided at all rural based hospitals. Since critical access hospitals only reflect a portion of the hospitals serving the rural population, future studies looking at non–critical access rural based hospitals will be important in assessing the general quality of inpatient care in rural settings.

The results of this study also raise several other questions. For example, why are the rates of smoking cessation counseling and the rates of pneumococcal vaccination poor in both the urban and rural settings? Also, the reasons why there are disparities between urban and rural settings remain unclear and could include differences in public health systems, funding, and or the use of protocols. Further study examining this issue is indicated. Finally, some differences in quality indicators between rural and urban settings might be acceptable if it meant retaining a service locally and making it more accessible. Research exploring this issue might be valuable in determining acceptable standards for smaller, rural hospitals and also for determining what types of conditions are better managed in larger urban settings.


View Abstract