International Journal for Quality in Health Care Advance Access originally published online on June 10, 2005
International Journal for Quality in Health Care 2005 17(5):439-445; doi:10.1093/intqhc/mzi056
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Are family members suitable proxies for transitional care unit residents when collecting satisfaction information?
Health Policy & Management, University of Pittsburgh, Pittsburgh, PA, USA
Objective. To examine the agreement and association of elders’ responses with family member proxy responses using the same, previously validated satisfaction instrument on both groups of respondents.
Methods. Satisfaction data came from transitional care unit residents and family members (N = 462 paired responses) from one facility and were collected between 1999 and 2000. The satisfaction questionnaire consisted of 17 items evaluating the art of care, technical quality, efficacy, amenities of the care environment, and global satisfaction. Bias indexes and intraclass correlation coefficients were used to examine the satisfaction scores.
Results. In general, proxy satisfaction ratings were higher than ratings of residents. The results also show that proxy ratings varied less from resident ratings for the amenity items, which were considered the most concrete items. Proxy ratings were much higher for the art of care and efficacy domain items, which were considered the least concrete items.
Conclusion. The results of this investigation show that proxy ratings do not necessarily substitute for resident ratings.
Keywords: family, proxies, transitional care unit
Address reprint requests to Nicholas Castle, Assistant Professor, Department of Health Policy & Management, University of Pittsburgh, GSPH, Health Policy & Management, 130 DeSoto Street, Pittsburgh, PA 15261, USA. E-mail: castlen{at}pitt.edu
Accepted for publication April 30, 2005.
Many authors have recently described the substantial benefits inherent in collecting satisfaction information in long-term care settings [1,2]. These benefits include promoting quality initiatives, consumer choice, and improved care, to name just three. Despite these potential benefits, when using residents as respondents, some barriers to effective implementation of these initiatives in these settings exist. Residents may be too ill to complete a questionnaire. Potential difficulties in obtaining survey information from long-term care residents include problems with response rates [3], low cognition [4], acquiescent response bias [5], and lack of response variability [6]. Some of these difficulties may be due to poor survey administration practices. Nevertheless, possibly because of these potential difficulties in obtaining survey information from residents, several authors have also used satisfaction initiatives with family members serving as a proxy for the resident [2,7–10].
When using proxy respondents, the barriers to obtaining sound satisfaction information are substantially lower (also sometimes called surrogate [11] or collateral [12] sources) compared with those encountered when using resident satisfaction surveys. These proxies are defined as ‘a family member, a friend, or a staff caregiver who knows the patient well, and reports what he or she believes to be true for the individual being examined’ [13]. With proper administration of the questionnaire, response rates from proxies may be high and cognition is less problematic. Moreover, costs of mail surveys that may be used with proxies are significantly lower than those inherent in in-person interviews of long-term care residents. However, using proxy respondents comes with its own potential problems, including item non-response and missing values [14].
Of most significance as a potential problem is that proxy reports may not be truly representative of resident reports. Magaziner and associates [15] have noted that proxy reports often do not agree with the self-reports of cognitively intact patients. This is important because proxy responses, if used as a substitute for resident responses, might lead to under- or overestimations of the ratings and reports, creating biased point estimates [14]. Such errors, in turn, may lead to invalid conclusions and the implementation of inappropriate improvement strategies.
Several studies have examined the extent of agreement between proxy responses and patients’ responses [16]. This includes comparing the responses of patient–proxy pairs on instruments measuring various domains of health status and quality of life [17,18]. Novella and Jochum [19] in reviewing this literature reconciled seemingly conflicting findings across studies by concluding that ‘the accuracy of proxy ratings is higher when the information sought is concrete and observable’. This conclusion now seems well accepted in the literature, especially in the area of quality of life. However, few studies have examined proxy ratings of performance measures such as resident satisfaction in long-term care [16].
In nursing homes, Lavizzo-Mourey et al. [20] found very little association between residents and surrogates (i.e. proxies). Norton et al. [21] and Becker and Kaldenberg [22] found resident satisfaction ratings to be substantially higher than the satisfaction ratings of family, whereas, in general, Duffy et al. [23] found family satisfaction to be higher than resident satisfaction. More recently, Gasquet et al. [24] found proxy scores to be generally higher than those of residents; however, in several satisfaction domains, resident and proxy responses were highly correlated.
A limited number of studies have examined resident–proxy agreement in long-term care settings, and these studies provide few clues to whether proxies represent suitable alternative satisfaction information for residents. This article reports on the agreement and association of elders’ responses with proxy responses (primarily from family members) by using the same, previously validated satisfaction instrument on both groups. All elderly respondents had stayed in a transitional care unit before completing the satisfaction instrument.
The recent study of elders by Gasquet et al. [24] had a large sample size and most directly addressed this issue of proxy respondents. Therefore, from this prior study, we hypothesize (H1) that, in general, proxy satisfaction measures will be higher (i.e. more satisfied) than satisfaction measures for residents, but overall resident–proxy responses will be highly correlated. However, no studies have examined whether resident–proxy responses for satisfaction measures vary according to the nature of the question (i.e. more observable versus less observable), and this variation may partially explain why the few long-term care studies in this area had such different findings. Following robust findings from the general proxy literature, we hypothesize (H2) that the agreement of proxy ratings will be higher for more observable and concrete satisfaction measures and lesser for less observable and less concrete satisfaction measures.
 |
Methods |
|---|
 Top Methods ResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
Data source and sample selection
Data came from transitional care unit residents in one facility and were collected between 1999 and 2000. A transitional care unit consists of hospital beds designated for subacute care. Residents in transitional care units are most often the elderly recovering from in-patient surgery and in need of complex medical services [25].
Satisfaction data were self-reported by residents. In addition, using a similar satisfaction instrument, we collected responses from family members acting as proxies for the residents. The only difference between these questionnaires was that the family questionnaire asked the respondents to answer from the perspective of the elder. Staff may also be used as resident proxies, but this information was not available for this investigation. We also note that staff do not necessarily provide the same information as family members [19].
Census records indicate that 576 residents were discharged from the transitional care unit during the period of data collection. Four hundred and sixty-two paired resident–proxy questionnaires were collected, giving us a respectable response rate of 80%.
Instrument development and administration
We previously developed and tested the survey instrument. The development of this questionnaire followed previous studies, by including three dimensions of resident satisfaction—art of care, technical quality, and efficacy [26]. The art of care domain assesses provider characteristics; this includes courtesy and comfort in asking questions. The technical quality of care domain assesses competence of caregivers; this includes the knowledge and explanations given by physicians, nurses, and social workers. The efficacy domain assesses the degree to which residents feel they were helped by the care provided. In addition, four questions regarding the amenities of the care environment and one global item were included [27]. For all 17 satisfaction questions, responses were rated on a five-point scale anchored by excellent (5) and poor (1). The psychometric properties of the instrument were previously reported as highly satisfactory [27]. Cronbach’s alphas for these scales were all higher than the usually recommended level of 0.70 [28].
To ensure that responses were representative of all the time spent in the transitional care unit, we administered surveys to residents on the day of discharge. The questionnaire and instructions were given to the resident as part of the discharge package. Residents were provided with instructions to deposit the completed questionnaire in a locked box in the lounge area. Staff were instructed not to help residents complete the survey and not to be in proximity when they were doing so.
Staff also gave a similar questionnaire to resident’s family (or friends) while they were waiting for elders to be discharged. Family members were provided with similar instructions to deposit the completed questionnaire in a locked box in the lounge area and were assured of anonymity. A numeric code on the questionnaire was used to create the resident–family dyad. That is, if a resident was given questionnaire number 10A, the family member was given questionnaire number 10B.
The questionnaires also included an introductory section that was intended to collect basic background and context information. This information included self-reported gender, age, perceived physical health, and number of visits to the transitional care unit. Physical health was measured on a six-point scale anchored by excellent (6) and very poor (1). We also asked elders whether this was their first stay in this transitional care unit and whether they were admitted from home or another institution. We asked proxies to identify their relationship with the elder as family, friend, or other.
Following our hypothesis [(H2) that the agreement of proxy ratings will be higher for more observable and concrete satisfaction measures and lesser for less observable and less concrete satisfaction measures], it is proposed that the amenity items are the most concrete and observable, followed by technical quality of care and art of care. The efficacy domain items were considered the least concrete and observable questions. These opinions of the domains came from a five-person research team, consisting of experts and practitioners in gerontology, geriatrics, and long-term care.
Analyses
Analyses used were bias indexes and intraclass correlation coefficients. The first bias measure used was the mean directional difference between the resident and proxy scores. Positive values indicate that family scores overestimate resident satisfaction and vice versa. Student’s paired t tests were used to determine whether the differences between the resident and family scores were significantly different from zero. Significant t scores indicate that systematic bias is present. The second bias measure examines the magnitude of any systematic bias. This second bias index is the difference between family and resident scores, divided by the standard deviation of the resident scores. Following Cohen’s effect sizes [29] (and as used by others, e.g. Gasquet et al. [24] and Sneeuw et al. [30]), we defined absent, small, moderate, and large over- or underestimates as values of <0.2, 0.2–0.5, >0.5–0.8, and >0.8, respectively.
Also following the approach recently used by Gasquet et al. [24], we used intraclass correlation coefficients to examine the agreement between resident and proxy responses. Owing to chance alone, a certain degree of agreement between scores can be expected. The intraclass correlation coefficients are a chance-corrected index of agreement [30]. This intraclass correlation coefficient measure ranges from 0 to 1. A score of 0 indicates no agreement beyond chance alone. Poor agreement is at levels of 0.40 and below, whereas excellent, good, and moderate agreement scores are at levels of >0.80, 0.60–0.80, and 0.41–0.59, respectively [24].
Scores were transformed to a common 0–100 scale, with higher scores reflecting greater satisfaction. Item responses in the 1–5 categories were recoded as 0, 25, 50, 75, and 100, respectively. This transformation was used because mean scores using the 0–100 scale were more intuitive than scores using the 1–5 scale.
Family members’ experience with each domain of satisfaction evaluated in the survey is highly dependent upon their interaction with the transitional care unit and residents. Therefore, we excluded from the analyses family members with less than five visits. This exclusion was not based on any theoretical or statistical information. Future work should determine whether a more appropriate and statistically sound cutoff score for the number of visits could be used. In our case, this resulted in the exclusion of only 27 pairs of residents and proxies, as the modal number of visits (n = 11) made by proxies was relatively high.
|
Results |
|---|
|
TopMethodsResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
Table 1 summarizes descriptive statistics of the resident and proxy respondents. As expected, residents were older, with an average age of 72 years, whereas proxies had an average age of 58 years. Most proxies (66%) identified themselves as family members, only 16% identified themselves as friends, and 18% identified themselves as other.
|
Table 2 summarizes the results from the statistical analyses used to examine the satisfaction scores between resident and proxy responses. The second and third columns of Table 2 summarize the mean scores and standard deviations for each domain and each satisfaction question. For the most part, the scores fall into the 70–80 range. Thus, as with most satisfaction data, the distributions are skewed to the higher end of the scale. It is also worth noting that the scores were stable during the 2 years of data collection, and sensitivity analyses using only 1999 data were highly similar to those using only data from 2000.
|
The fourth column of Table 2 summarizes the first bias measure, which is the mean directional difference between the patient and proxy scores. These values range from 7 to 11. Most scores indicate that family overestimate elders’ satisfaction, as shown by the positive scores. Only two questions in the technical quality domain were rated higher on average by residents than proxies. Student’s t tests show that systematic bias was present for nine item scores and two domain scores. No significant t scores were found for any of the amenity questions.
The second bias measure is given in column 6 of Table 2 and examines the magnitude of any systematic bias. We find the bias to be absent in eight questions, small in seven, moderate in two, and large in no questions (as shown within parentheses in column 6).
The intraclass correlation coefficients are summarized in column 7 and examine the agreement between resident and proxy responses. Agreement was excellent for four questions, good in eight, and moderate in three questions (as shown within parentheses in column 7). For all but one of the amenity items, the agreement was excellent, and for all the efficacy items, the agreement was poor.
|
Discussion |
|---|
|
TopMethodsResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
Sprangers and Aaronson [31] have voiced some concern that small sample sizes, differences in the timing of administering the questionnaire, and differences in the proxy version of questionnaires served to confound the results of patient–proxy studies. This investigation addresses all of these concerns by using a relatively large sample size, consisting of 462 resident–proxy pairs, and by using an identical questionnaire for both parties, administered at approximately the same time. This gives us a considerable degree of confidence in the results reported.
We first hypothesized (H1) that, in general, proxy satisfaction measures would be higher (i.e. more satisfied) than satisfaction measures for residents, but overall resident–proxy responses would be highly correlated. The results show that for 14 of the questionnaire items, proxies were significantly more satisfied than residents. For the remaining three questions, proxies had lower scores than residents; however, invariably, these were not statistically significant. In addition, for 14 of the questions, resident–proxy responses were significantly correlated. Thus, our findings follow this hypothesis.
We also hypothesized (H2) that the accuracy of proxy ratings would be higher for more observable and concrete satisfaction measures and less consistent for less observable and less concrete satisfaction measures. The intraclass correlation coefficient results show that proxy ratings varied little from resident ratings for the amenity items, which were considered the most concrete items. Proxy ratings were much higher for the art of care and efficacy domain items, which were considered the least concrete questions. Thus, our findings follow this hypothesis too.
The results for individual question items were very robust and in most cases followed our hypotheses. However, a few of the individual questions did not follow the relationships proposed. For example, comfort in asking questions of doctors item had a greater intraclass correlation coefficient than expected, but in all other cases the domain scores followed our hypotheses. These multi-item domain scores are theoretically more reliable than the scores from the individual questions. Therefore, we recommend that if proxy satisfaction information is to be collected and used, then objective and concrete questions should be asked.
|
Limitations of the study and suggestions for further research |
|---|
|
TopMethodsResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
Sneeuw et al. [30] point out that one methodological issue with studies examining patient–proxy responses is that a questionnaire has to be completed by both parties. Yet inferences cannot be made for patients with the most need for proxy respondents (e.g. Alzheimer’s patients) because they are unable to participate in such studies. We have a similar concern with our study, as 20% of residents were not included in the analytic file, and they were likely the sickest residents.
Nineteen percent of residents indicated that they previously had a stay in the transitional care unit. These residents may give different satisfaction responses from other residents. Moreover, our data probably contain more than one response from the same resident–family dyad. Given that our question asked whether the resident had a prior stay in the transitional care unit without specifying a time frame and that our data came from only a 2-year interval, we speculate that less than 19% of cases were duplicate resident–family dyads. Indeed, data from subsequent years would suggest that duplicate responses likely totaled approximately 11% of cases. Results from sensitivity analyses excluding these residents (and family) who reported a previous transitional care unit experience were highly similar to those presented.
We asked residents and proxies to complete the questionnaire at the time of discharge. This approach was used primarily because of convenience. However, the site of questionnaire administration may influence responses. We suggest that additional work examining respondents asked to complete questionnaires at home will be useful.
Most of the satisfaction score distributions were skewed to the higher end of the response scales. This was not unexpected, because a high satisfaction response bias is often found in satisfaction surveys [1,27]. Some of this bias may be due to acquiescence bias—the propensity for respondents to provide favorable ratings. When comparing the scores of different individuals (i.e. family and resident), this may be problematic. Differences may occur because one group is more prone to acquiescence bias.
The results presented also suffer from the shortcoming of poor generalizability. Firstly, given that our findings come from one facility, our results may not be generalizable to other transitional care units. The transitional care unit we used was typical concerning bed size and staffing levels. It was also located in a ‘typical’ hospital, of medium size and not-for-profit status. Nevertheless, our study should be replicated using a larger number of providers. Secondly, our findings may not be generalizable to other long-term care settings. Transitional care is part of the long-term care continuum of services; however, lengths of stay and resident characteristics are unlike most other areas of long-term care. Given the short length of stay, residents/family are less likely to attribute any negative outcomes to the facility. Although our sensitivity analyses examining different resident lengths of stay did not identify any significant differences from the results reported, these analyses were limited by small sample sizes as resident length of stay was relatively homogenous.
|
Conclusion |
|---|
|
TopMethodsResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
We should consider the substitutability of resident and proxy ratings. The results of this investigation show that proxy ratings do not necessarily substitute for resident ratings. Proxy ratings varied little from resident ratings for the most concrete satisfaction items but were much higher for the least concrete questions. There currently is much interest in measuring satisfaction in nursing homes [1]. Thus, there is still a need to replicate this study with nursing home residents/families.
|
References |
|---|
|
TopMethodsResultsDiscussionLimitations of the study...ConclusionReferences |
|---|
- Cohen-Mansfield J, Ejaz F, Werner P. Satisfaction Surveys in Long-Term Care. New York: Springer Publishing Company, 2000.
- Ejaz FK, Straker JK, Fox K, Swami S. Developing a satisfaction survey for families of Ohio’s nursing home residents. Gerontologist 2003; 43: 447–458.
[Abstract/Free Full Text] - El-Guebaly N, Toews J, Leckie A, Harper D. On evaluating patient satisfaction: methodological issues. Can J Psychiatry 1983; 28: 24–29.[Web of Science][Medline]
- Simmons SF, Schnelle JF, Uman GC, Kulvicki AD, Lee KO, Ouslander JG. Selecting nursing home residents for satisfaction surveys. Gerontologist 1997; 37 (4): 543–550.[Abstract]
- Ross C, Steward C, Sinacore JM. A comparative study of seven measures of patient satisfaction. Med Care 1995; 33: 392–406.[CrossRef][Web of Science][Medline]
- Pascoe G, Attkisson CC. The evaluation ranking scale: a new methodology for assessing satisfaction. Eval Prog Plann 1983; 6: 335–347.
- Kleinsorge IK, Koenig HF. The silent customers: measuring customer satisfaction in nursing homes. J Health Care Manag 1991; 11: 2–13.
- Mostyn MM, Race KE, Seibert JH, Johnson M. Quality assessment in nursing home facilities: measuring customer satisfaction. Am J Med Qual 2000; 15 (2): 54–61.
[Abstract/Free Full Text] - Steffen TM, Nystrom PC. Organizational determinants of service quality in nursing homes. Hosp Health Serv Adm 1997; 422: 179–191.
- Wakefield B, Buckwalter KC, Collins CE. Assessing family satisfaction with care for persons with dementia. Balance 1997; 1 (1): 16–17, 40–42.
- McCusker J, Stoddard AM. Use of a surrogate for the Sickness Impact Profile. Med Care 1984; 22: 789–795.[CrossRef][Web of Science][Medline]
- Teri L, Wagner AW. Assessment of depression in patients with Alzheimer’s disease: concordance among informants. Psychol Aging 1991; 6 (2): 280–285.[CrossRef][Web of Science][Medline]
- Hickey EM, Bourgeois MS. Health and quality of life (HR-QOL) in nursing home residents with dementia: stability and relationships among measures. Aphasiology 2000; 14 (5/6): 669–679.[CrossRef][Web of Science]
- Shaw C, McColl E, Bond S. Functional abilities and continence: the use of proxy respondent in research involving older people. Qual Life Res 2000; 9 (10): 1117–1126.[CrossRef][Web of Science][Medline]
- Magaziner J, Zimmerman SI, Gruber-Baldini AL, Hebel JR, Fox KM. Proxy reporting in five areas of functional status. Comparisons with self-reports and observations of performance. Am J Epidemiol 1997; 146: 418–428.
[Abstract/Free Full Text] - Neumann PJ, Araki SS, Gutterman EM. The use of proxy respondents in studies of older adults: lessons, challenges, and opportunities. J Am Geriatr Soc 2000; 48: 1646–1654.[Web of Science][Medline]
- Magaziner J, Simonsick EM, Kashner TM, Hebel JR. Patient-proxy response comparability on measures of patient health and functional status. J Clin Epidemiol 1988; 41: 1065–1074.[CrossRef][Web of Science][Medline]
- Rubenstein LZ, Schairer C, Wieland GD, Kane R. Systematic biases in functional status assessment of elderly adults: effects of different data sources. J Gerontol 1984; 39: 686–691.
[Abstract/Free Full Text] - Novella AM, Jochum C. Agreement between patients’ and proxies’ reports of quality of life in Alzheimer’s patients. Qual Life Res 2001; 10 (5): 443–452.[CrossRef][Web of Science][Medline]
- Lavizzo-Mourey RJ, Zinn JS, Taylor L. Ability of surrogates to represent satisfaction of nursing home residents with quality of care. J Am Geriatr Soc 1992; 40: 39–47.[Web of Science][Medline]
- Norton PG, van Maris B, Soberman L, Murray M. Satisfaction of residents and families in long-term care: I. Construction and application of an instrument. Qual Manag Health Care 1996; 4 (3): 38–46.[Medline]
- Becker BW, Kaldenberg DO. Factors influencing the recommendation of nursing homes. Mark Health Serv 2000; 20 (4): 22–28.[Medline]
- Duffy JA, Duffy M, Kilbourne WE. A comparative study of resident, family and administrator expectations for service quality in nursing homes. Health Care Manage Rev 2001; 26 (3): 75–85.[Web of Science][Medline]
- Gasquet I, Dehe S, Gaudebout P, Falissard B. Regular visitors are not good substitutes for assessment of elderly patient satisfaction with nursing home care and services. J Gerontol A Biol Sci Med Sci 2003; 58: 1036–1041.
- Kovner AR, Jonas S. Health Care Delivery in the United States. New York: Springer Publishing Company, 1999.
- Ware JE, Davies-Avery A, Stewart AL. The measurement and meaning of patient satisfaction. Health Med Care Serv Rev 1978; 1: 1–15.
- Castle NG. Resident satisfaction in a Transitional Care Unit. J Appl Gerontol 2004; 23 (4): 411–428.[Abstract]
- McHorney CA, Ware JE, Lu R, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994; 32 (1): 40–66.[Web of Science][Medline]
- Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press, 1977.
- Sneeuw KCA, Aaronson NK, Osoba D et al. The use of significant others as proxy raters of the quality of life of patients with brain cancer. Med Care 1997; 35 (5): 490–506.[CrossRef][Web of Science][Medline]
- Sprangers MA, Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: a review. J Clin Epidemiol 1992; 45: 743–760.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||