OUP user menu

The importance of place of residence in patient satisfaction

Carey Levinton, Jeremy Veillard, Arthur Slutsky, Adalsteinn Brown
DOI: http://dx.doi.org/10.1093/intqhc/mzr048 495-502 First published online: 3 August 2011

Abstract

Objective To determine the effect of patients' place of residence on their evaluations of care, and to explore related policy implications.

Study Design We used a conditional regression analysis of stratum matched case controls to examine whether place of residence of patients living in the Greater Toronto Area (GTA) or in Ontario outside of the GTA affects patient satisfaction with their experiences during hospitalization.

Setting One hundred and six acute care hospitals located in the province of Ontario, Canada.

Participants A total of 101 683 Ontario residents hospitalized as inpatients between 1 October 2002 and 30 June 2004.

Main Outcome Measures Patient satisfaction indicators publicly reported in Ontario comprising patient perceptions of consideration, responsiveness, communication, and overall impressions, scored on a continuous scale from 1 to 100.

Results Patients who lived outside Toronto were consistently more satisfied than patients who lived inside Toronto when both types of patients were hospitalized in Toronto (P < 0.0001). In contrast, patients who lived inside Toronto were usually and substantially more satisfied than patients who lived outside Toronto when they were hospitalized in facilities outside Toronto (P < 0.02). These findings were consistent after adjustment for several patient-level predictor variables: age, sex, self-assessed health status and number of hospital stays in the last 6 months.

Conclusion Findings suggest that where patients live has a small but potentially important impact on how they rate their care. Residence may therefore be considered when designing public reporting systems and pay-for-performance programs. Further attention to patient-level factors may be important to accurate and useful public reporting of patient satisfaction.

  • patient satisfaction
  • risk adjustment
  • statistical methods
  • quality measurement
  • hospital performance

Introduction

Patient satisfaction and patients' evaluations of their care are an important and common component of public performance reporting and pay-for-performance programs by Health Maintenance Organizations (HMOs) and other payers [1]. In line with a large volume of work on conceptual models for patient satisfaction [2], and to ensure that these comparisons are fair, reports of patients' evaluations of care are typically risk adjusted for patient-level factors over which providers have no control, such as patients' age, sex or severity of illness [3]. Although risk adjustment can benefit from sophisticated statistical approaches such as hierarchical modeling [4], there may be a number of important factors that can affect patients' evaluations of care that are not captured in common conceptual models [5].

One potentially important factor is the location of the provider. In Ontario, report cards for emergency departments and acute care hospitals have consistently shown a lower level of patient satisfaction across Toronto (for this study, Toronto designates the Greater Toronto Area (GTA) which had a population of 5.5 millions inhabitants in the 2006 census) hospitals compared with Ontario hospitals in other, smaller urban areas and in rural communities. This persistent finding has led chief executives at Toronto hospitals to argue for Toronto-specific risk adjustments. The most recent hospital report cards have used hierarchical modeling to adjust for both patient-level characteristics (such as age and self-rated health) and the hospital corporation's peer group [4, 6]. However, it is difficult to determine whether there is a need to risk-adjust for the location of the hospital or for the place of residence of its patients [7, 8]. This question is difficult to answer using regression methods. Further, any inquiry is complicated by issues common in large provinces and states: hospitals in Toronto tend to be larger than elsewhere in Ontario, they are more likely to be teaching hospitals, and they serve the most heterogeneous communities. These factors may affect patients' perceptions of care without any need to rely on place-specific arguments. At the same time, the patients resident in these communities may have different expectations of care because of a choice among hospitals, because they lack social connections to the workers in their local hospital that might exist in smaller communities, or because there is greater exposure to media reports on problems with care in hospitals, particularly if larger newspapers tend to focus on negative stories compared with smaller newspapers [9]. Other studies have suggested similar differentials in satisfaction in the USA, with rural patients reporting ‘better care’ than urban patients [10, 11]. If patients' place of residence does affect patients' evaluations, then performance reports and pay-for-performance programs may need to take place of residence into account.

This study aims to disentangle the effects of where patients live and where patients receive care on their evaluations of care to determine whether or not the lower scores seen in Toronto hospitals are more related to the care received or to patient characteristics. It uses conditional regression analysis of stratum matched case controls [12] to examine whether place of residence affects patient satisfaction with hospitalization. Specifically, the study focuses on differences in perceptions between patients who live in Toronto and patients who live elsewhere in Ontario when they are hospitalized either in Toronto hospitals or in hospitals outside of Toronto.

Methods

Data

We used 101 683 surveys (45.8% response rate) collected by the NRC Picker Group Canada as part of a hospital report card project from Ontario residents hospitalized as inpatients in 106 Ontario hospitals between 1 October 2002 and 30 June 2004. These surveys follow the Picker Institute's Patient-Centered Care philosophy and ask questions about patients' experiences with their hospital stay. These response rates are consistent with those seen in the USA and Europe for such broad surveys and with historical experience in Canada, but Table 1 shows that response rates do vary by category (peer group) of hospital. It should be noted that hospital categories (small, community and teaching hospitals) are defined based on a number of different factors such as hospital size, acuity of cases treated, activities carried out, isolation and geographic location [13]. Tables 2 and 3 show that there are a number of statistically significant differences between populations based on where they live and where they are hospitalized that could be sufficient to account for differences in reported evaluations of Toronto and non-Toronto hospitals. These differences are associated with differences in patient satisfaction scores and are adjusted for in our analyses. We also excluded responses from patients who were under 18, who were hospitalized at the two Ontario specialized stand-alone pediatric hospitals, and responses completed by someone other than the patient because of difficulty in applying risk-adjustment models to these groups. We scaled the responses to questions using a similar approach to that employed by the Picker Institute, Europe (lowest score = 0; highest score = 100) and calculated indicators of patient-centered care using the data from these surveys using indicators that broadly correspond to the collapsed sets of the dimensions originally developed by the Picker Institute and that are used for public reporting in Ontario. Details of the questions, derived indicators and adjustment methods are described elsewhere [iv,vi].

View this table:
Table 1

Response rates where hospitals are located and where patients live

Hospital in TorontoSmallCommunityTeaching
Patient from TorontoN/A43.8% (n = 54 228)43.9% (n = 5422)
Patient from outside TorontoN/A56.2% (n = 2110)59.5% (n = 555)
Hospital outside TorontoSmallCommunityTeaching
Patient from Toronto41.7% (n = 36)51.1% (n = 603)52.2% (n = 224)
Patient from outside Toronto52.7% (n = 11 755)51.6% (n = 71 640)52.9% (n = 28 720)
View this table:
Table 2

Characteristics of patients hospitalized in Toronto, depending on where they live

VariableResidence concordant with hospital locationResidence non-concordant with hospital location
General health status
 Poor1061 (5.1%)43 (3.9%)P < 0.0001
 Fair3812 (18.3%)176 (12.6%)
 Good6866 (32.9%)492 (35.3%)
 Very good6176 (29.6%)485 (34.8%)
 Excellent2942 (14.1%)197 (14.1%)
Including this stay, how many times in the last 6 months have you been in a hospital overnight or longer?
 Only this time15 689 (75.2%)1027 (73.4%)P = 0.34
 This time and one other time3443 (16.5%)245 (17.5%)
 This time and more than one other time1746 (8.4%)127 (9.1%)
Education level
 Public school2186 (10.9%)115 (8.6%)P < 0.0001
 High school6567 (32.8%)515 (38.4%)
 College, trade or technical school5503 (27.5%)421 (31.4%)
 Undergraduate degree3488 (17.4%)169 (12.6%)
 Graduate education2306 (11.5%)123 (9.2%)
Sex
 Female13 276 (61.1%)789 (55.1%)P < 0.0001
 Male8455 (38.9%)644 (44.9%)
Age57.85 (0.13) (n = 21731)57.40 (0.44) (n = 1433)P = 0.33
Length of stay5.51 (0.05) (n = 21 715)4.66 (0.16) (n = 1433)P < 0.0001
During the past month, how many days did illness or injury keep you in bed all or part of the day?1.8 (0.02) (n = 20 562)1.5 (0.08) (n = 1379)P < 0.0001
View this table:
Table 3

Characteristics of patients hospitalized outside of Toronto, based on where they live

VariableResidence non-concordant with hospital locationResidence concordant with hospital location
General health status
 Poor22 (3.0%)3276 (5.7%)P < 0.0001
 Fair113 (15.3%)11683 (20.3%)
 Good227 (30.8%)18347 (32.0%)
 Very good250 (33.9%)16157 (28.1%)
 Excellent126 (17.1%)7966 (13.9%)
Including this stay, how many times in the last 6 months have you been in a hospital overnight or longer?
 Only this time547 (73.3%)41 083 (71.8%)P = 0.62
 This time and one other time127 (17.0%)10 192 (17.8%)
 This time and more than one other time72 (9.7%)5987 (10.5%)
Education level
 Public school45 (6.3%)9368 (17.1%)P < 0.0001
 High school243 (33.9%)20 378 (37.2%)
 College, trade or technical school208 (29.1%)15 256 (27.9%)
 Undergraduate degree140 (19.6%)5939 (10.9%)
 Graduate education80 (11.2%)3808 (7.0%)
Sex
 Female471 (60.4%)37 046 (61.9%)P = 0.40
 Male309 (39.6%)22 851 (38.25%)
Age54.1 (0.69) (n = 780)58.2 (0.08) (n = 59 912)P < 0.0001
Length of stay4.9 (0.27) (n = 755)6.7 (0.88) (n = 58 891)P = 0.05
During the past month, how many days did illness or injury keep you in bed all or part of the day?1.6 (0.11) (n = 732)1.8 (0.01) (n = 56 629)P = 0.27

Statistical analysis

This study employs a case-control study design matching cases (n) and controls (m) in order to compare: (i) the perceptions of care by people living in the Great Toronto Area (designated as Toronto in this study for convenience) and receiving care in Toronto against those of people living in Ontario outside Toronto but receiving care in Toronto matched by hospital and (ii) the perceptions of care by people living in Toronto but receiving care outside Toronto against the satisfaction of people living outside Toronto and receiving care outside Toronto.

In this study, cases n are patients residing in the GTA and controls m are patients residing in Ontario outside the GTA. The dummy variables are similarly matched to cases (patients residing in the GTA) and controls (patients residing in Ontario outside the GTA). In addition, all cases and controls are matched to their hospital of treatment via a strata statement. Analysis is carried out through a conditional logistic regression method (SAS 8.2, SAS, Cary, NC) [1416]. By using the hospital as the stratification or matching variable, we effectively set up a series of two-by-two tables that enabled us to compare patient scores by where the patient resided and where the patient sought care. This means that we are making comparisons only for patients inside and outside Toronto who were seen at the same hospital within the time period of the study. We compared the satisfaction scores across these different groups of patients using hazard ratios to approximate odds ratios and also calculated means and standard errors of average satisfaction scores for each of the indicators.

The structure of the model is presented in Table 4.

View this table:
Table 4

Model structure

DesignCase control using conditional logistic regression
Strata (matching n:m), i.e. variable number of cases to controlsHospital facility (acute care)
Model application1. Patients hospitalized in GTA
2. Patients hospitalized outside GTA
Outcome (dependent variable)GTA patient vs. non-GTA patient (dummy coded 0/1)
Patient characteristics (independent variables)Age (continuous)
Sex
Education (see Tables 2 and 3)
Length of stay (days)
General Health Status
Bed days due to illness or
Injury in the past month
Number of hospitalizations
In the last 6 months
Facility characteristicsPeer group (Table 1) (excluded from final analysis)
Effect of interestPatient satisfaction with hospital experience: (continuous)
 Responsiveness
 Dignity
 Communication
 Overall satisfaction

The methods used have a number of advantages over typical unconditional models, which were particularly useful for this study. First, we could match any number of patients in the GTA to those outside the GTA: thus, all the data contributed to the analysis, unlike propensity scoring where we have a 1:1 or 1:n design. Second, the methods allowed for the development of exact estimates and testing of the fit of the models themselves. Finally, the same conditional likelihood applies whether the design is prospective or case control [17].

In order to see if the results presented below were due to some small amount of border crossing by residents living close to one side or the other of the heavily populated edges of the GTA, we recalculated the results focusing on the historic center of Toronto, a much smaller geographic and population area with fewer but, on average, larger hospitals. This did not have any substantial impact on the direction, significance or sizes of the differences reported below. In order to see if the results were due to the type of hospital (e.g. patients hospitalized in Toronto were more likely to be hospitalized in large or teaching hospitals), we also recalculated the results by peer group (small, large community or teaching) of hospitals. Although the type of facility or peer group has been found to correlate strongly with a variety of patient satisfaction measures [vii], this did not have any substantial impact on the direction of the differences reported below, although some comparisons were impossible, i.e. there are no small hospitals in Toronto.

Results

Tables 13 show that there are substantial differences in response rates and in demographic characteristics between patients living inside and outside Toronto when they are hospitalized in Toronto and outside Toronto. Most importantly, patients tend to report being healthier when they are hospitalized away from home and tend to stay for shorter durations when hospitalized away from home. These differences are adjusted for in our analyses. However, this finding is consistent with the fact that most people are hospitalized close to home in Ontario and that most emergency care happens close to home [18].

A comparison of Tables 5 and 6 shows that scores for patients' perceptions of care tend to be lower for Toronto hospitals, regardless of where patients lived. This confirms past findings [iv]. However, Tables 5 and 6 also show that perception scores were consistently higher for patients living outside Toronto and hospitalized in Toronto, than for patients living in and hospitalized in Toronto. All hospitals in Toronto received patients from outside the GTA (19/19). For each indicator, patients from outside Toronto scored significantly higher or better (P < 0.0001) than Toronto patients when hospitalized in the same Toronto hospitals, as reflected in risk ratios that were significant. Even after adjusting for age, sex, self-assessed health status and number of times the patient was hospitalized in the last 6 months, scores for patients living outside Toronto were considerably higher, with P values < 0.0001 across all indicators. This means that patients from outside Toronto rated the care they received in Toronto hospitals more positively than patients residing in Toronto and receiving care in the same hospitals. In addition, it is worth noting that patients from outside Toronto had comparable results in their evaluation of care received when comparing for hospitals inside and outside Toronto.

View this table:
Table 5

Average patient evaluations of care in Toronto hospitals, based on where they live

OutcomeOdds ratioP-valueResidence concordant with hospital location, mean (SE)Residence non-concordant with hospital location, mean (SE)
Responsiveness
 Unadjusted1.018 (1.014, 1.022)<0.000177.93 (0.12) (n = 21 075)82.73 (0.42) (n = 1402)
 Adjusted1.015 (1.011, 1.019)<0.0001
Dignity1.021 (1.017, 1.025)<0.000180.54 (0.12) (n = 21 210)85.51 (0.38) (n = 1413)
1.019 (1.015, 1.023)<0.0001
Communication1.009 (1.006, 1.012)<0.000175.13 (0.17) (n = 19 413)79.67 (0.60) (n = 1305)
1.006 (1.003, 1.009)<0.0001
Overall satisfaction1.012 (1.009, 1.015)<0.000180.83 (0.14) (n = 21 225)85.29 (0.47) (n = 1411)
1.009 (1.006, 1.013)<0.0001
View this table:
Table 6

Average patient evaluations of care in hospitals outside of Toronto, based on where they live

OutcomeOdds ratioP-valueResidence non-concordant with hospital location Mean (SE)Residence concordant with hospital location Mean (SE)
Responsiveness
 Unadjusted1.007 (1.002, 1.012)0.00583.58 (0.55) (n = 747)82.73 (0.06) (n = 57 957)
 Adjusted1.006 (1.001, 1.012)0.02
Dignity1.010 (1.005, 10.16)0.000386.47 (0.52) (n = 750)84.89 (0.06) (n = 58 950)
1.008 (1.002, 1.014)0.008
Communication1.002 (0.998, 1.006)0.3179.21 (0.81) (n = 680)79.47 (0.09) (n = 54 083)
1.001 (0.997, 1.005)0.74
Overall satisfaction1.008 (1.003, 1.013)0.000686.52 (0.62) (n = 752)84.76 (0.07) (n = 58 238)
1.007 (1.002, 1.012)0.005

However, Table 6 shows that (with one exception) patients from Toronto consistently and significantly (P < 0.02) rated their care more positively when hospitalized outside Toronto, when compared with non-Toronto patients. The one exception is their ratings of communication. Patients from outside Toronto rated the communication indicator more positively but not significantly so compared with patients from Toronto when hospitalized outside Toronto (mean score of 79.47 vs. 79.21, P = 0.75). This difference is much smaller than the nearly five times as large difference observed when comparing patients from inside and outside Toronto hospitalized in Toronto. However, it is important to note that the difference associated with living inside and outside Toronto when hospitalized in Toronto was larger than the mean difference between the historically lower scoring Toronto hospitals and hospitals located elsewhere in the province. This means that although the difference attributable to place of residence is small, it is large enough to change the provincial rankings of hospital satisfaction if included in statistical risk adjustment.

Discussion

This analysis suggests that where people reside can have some impact on their perceptions of care. In Ontario, when hospitalized in Toronto hospitals, patients residing in Toronto rate their care more poorly than patients residing outside of Toronto. Another finding of this study is that when hospitalized outside of Toronto, patients residing in Toronto largely rate their care better than patients residing outside of Toronto. Finally, we found that patients residing outside of Toronto rated the care they received consistently, whether they were hospitalized inside or outside Toronto. This was not the case for patients residing in Toronto. Taken together, these results suggest that patients' place of residence has some impact on their satisfaction with care received. However, this is not sufficient to explain all of the observed differences in patients' evaluations of care between Toronto and non-Toronto hospitals.

There are many reasons why Toronto patients may rate Toronto hospitals more poorly than their peers from elsewhere in Ontario. First, these patients may have higher expectations of care because of a perception of choice. The large number of hospitals in Toronto may lead patients to think that there may be hospitals providing better care in their city. In effect, expectations may be higher for patients who perceive a choice leading to lower scores according to the expectation disconfirmation model of satisfaction [19, 20]. In a previous report we noted that patients resident in Toronto and hospitalized in Toronto were less likely to report that they would return to the same hospital for care when compared with patients resident outside of Toronto and hospitalized outside of Toronto who reported the same overall levels of satisfaction [viii]. Second, patients resident in Toronto may be less satisfied with their care experience because they expect to be less satisfied. A number of media reports and some scholarly work have pointed to profound limits on the capacity of Toronto hospitals with resulting overcrowding [21]. An alternative explanation for these variations would be that in many instances when patients receive care outside of their place of residence they are doing so in order to obtain services that are specialized and/or not readily available nearby; patients may be more inclined to report higher satisfaction under such circumstances.

However, none of these potential explanations suggest why Toronto residents may be relatively happier with their care outside of Toronto. Toronto residents may receive less intense care or may be on holiday when receiving care outside of Toronto, but it is hard to argue that people would be pleased or have different expectations in such a situation. Risk adjustment for self-reported health status might have handled the first possibility. To investigate the second possibility, we examined whether the Christmas period had any effect on the results, to see whether or not patients viewed hospitalization over holidays more positively. There was no effect. Likewise day of the week in which a patient was hospitalized was statistically insignificant, suggesting that weekend hospitalizations during trips to cottages did not lead to better evaluations of care (data available on request).

It is important to emphasize that even if the differences accounted for by place of hospitalization and residence may be small in absolute terms (all odds ratios reported are significant but close to 1.0), they are consistent and significant. Because patient satisfaction scores are so tightly clustered for many hospitals, they would be sufficient to change a ranking of hospitals by patient satisfaction scores. Thus, the results could have several important implications for using comparative reports on patients' evaluations of their care. First, when using comparative scores to guide benchmarking and quality improvement exercises, hospitals should be careful to select hospitals that have similar environments. Not surprisingly, large urban hospitals may want to benchmark against other large urban hospitals and not against suburban hospitals. Second, comparative report cards that include patients' evaluations of care may want to consider whether some risk adjustment for place of residence using hierarchical models is necessary. An argument against risk adjustment for place of residence would be that such risk adjustment could be detrimental to efforts underway to improve patient satisfaction in hospitals with lower patient satisfaction rates. Third, the unadjusted use of comparative scores as part of pay-for-performance schemes may be problematic [22]. Although large urban hospitals are not likely to reject local residents because of concerns over their evaluations of care—a common concern over the impact of other types of pay-for-performance programs [23]—large urban hospitals may perceive comparative analyses that do not adjust for patient place of residence to be unfair.

These data also suggest that comparisons of patient satisfaction scores across jurisdictional boundaries should be approached carefully and that more focused comparisons between clinical groupings of patients hospitalized in similar settings may be more appropriate ways of generating cross-jurisdictional comparisons of performance [24]. Alternatively, comparisons of adjusted scores across time within a jurisdiction may be a safer approach to comparative performance reporting of patient perceptions of care, that is comparing the amount of improvement, and one that is more compatible with improvement-oriented theoretical models like patient-centered care [25].

A number of limitations of this study should be pointed out. One limitation is the absence of risk adjustment based on clinical judgment of severity of illness. We relied upon patient self-report of medical status. Even if precautions were taken to avoid bias as mentioned in the Methods section, in the future it would be helpful to link clinical records and patient satisfaction surveys in a way that is respectful of privacy policies and regulations but can also support more precise adjustments of satisfaction based on health status at admission to hospital. It should also be noted that the response rate among Toronto residents hospitalized in Toronto hospitals is lower than among patients living outside of Toronto. Like the difference in satisfaction scores, the overall difference in response rates is small but statistically significant. If the likelihood of responding was inversely related to satisfaction with care, this difference could account for the observed differences in scores between Toronto and non-Toronto hospitals. However, limited evidence suggests that non-responders are actually less satisfied with care [26]. Notwithstanding this argument, response rates do deserve attention in further research.

The fundamental point of this paper is that there is a complex set of factors that affect patient satisfaction and patients' perceptions of care that may be independent of the technical quality of care provided in a hospital. These factors are important for risk adjustment and accountability uses of the data but they are also important for understanding the best way to target improvement strategies centered on patient care. Depending on the nature of unmeasured risk factors, it may or may not be realistic or credible to hold clinicians or other providers fully accountable for performance differences [27] and policy-makers will need to weigh the competing arguments when deciding on the appropriate use of risk-adjusted data.

Future research could examine the impact of patients' place of residence on the effectiveness of different strategies designed to improve patient perceptions of care; or could consider a more in-depth exploration of how expectations and other patient factors differ based on the number of hospitals in a community, perceived competition among hospitals, and a priori perceptions of the quality of care in a hospital. Most importantly, the improvement in clinical data systems and new possibilities for linking clinical data with survey data will open new opportunities to re-examine the range of useful predictor variables in risk-adjustment models and enable fair and meaningful comparisons [xxiv].

Funding

This study was supported through the Hospital Report Project, a joint initiative of the Ontario Hospital Association and the Ontario Ministry of Health and Long-term Care.

Acknowledgements

The authors wish to thank Dr Michael Murray (University of Toronto, Canada) and Prof Niek Klazinga (University of Amsterdam, Netherlands) for their time and thoughtful suggestions in preparation of the manuscript.

References

View Abstract