International Journal for Quality in Health Care Advance Access originally published online on March 23, 2005
International Journal for Quality in Health Care 2005 17(3):259-267; doi:10.1093/intqhc/mzi032
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The association between quality of care and technical efficiency in long-term care
1 Centre for Health Economics at STAKES CHESS, Helsinki, Finland, 2 Chydenius Institute, University of Jyväskylä, Kokkola, Finland
Objective. To analyse the association between quality of care and technical (productive) efficiency in institutional long-term care wards for the elderly.
Setting. One hundred and fourteen public health centre hospitals and residential homes in Finland.
Study design. Wards were divided into two categories according to their rank in the quality distribution, considering 41 quality variables separately. The technical efficiency scores of the good- and poor-quality groups were compared using cross-sectional data.
Methods. Data envelopment analysis was used for calculating technical efficiency. The MannWhitney test and correlation coefficients were used to explore the association between quality and efficiency.
Results. The wards where quality indicators indicated less pro-active (passive) nursing practice and more dependent patientsfor instance, in terms of very high prevalence of bedfast residents or very high prevalence of daily physical restraintsperformed more efficiently than the comparison group.
Conclusion. The results suggest that an association may exist between technical efficiency and unwanted dimensions of quality. Hence, the efficiency and quality of care are essential aspects of management and performance measurement in elderly care.
Keywords: data envelopment analysis, long-term care, quality, technical efficiency
Address reprint requests to Juha Laine, STAKES, Lintulahdenkuja 4, PO Box 220 FIN-00531 Helsinki, Finland. E-mail: juha.laine{at}stakes.fi
The association between economic performance and quality of care is an essential aspect of the production of health services. According to Newhouse [1], quantity and quality are two commodities to which the decision-maker can allocate resources, and that a quantityquality trade-off is present when resources are constrained. It is appealing to think that increasing quality may require additional labour and capital resources, whilst a tendency towards efficiency improvements and cost containment can lead to a poorer performance in quality. On the other hand, better quality can be associated with better economic performance and lower production costs, i.e. better efficiency [2]. Deterioration in quality can cause voiced complaints (voice) and possible withdrawal (exit) of customers, which force the management of the unit to react and to prevent economic loss [3]. If there is an association between economic performance and quality of care in health services, it definitely has implications relevant to policy and management.
Technical (productive) efficiency is one possible way to evaluate the economic performance of the units. It is a measure of the proportion of input resources (e.g. labour) to the output (e.g. in-patient days). When units are compared, efficient units produce the same output with fewer inputs than inefficient units, or alternatively, produce more output with the same input as inefficient units [4]. Björkgren et al. [5] and Laine et al. [6] have shown that the average technical inefficiency is high in Finnish long-term care units. The variation in clinical quality of care and staff per resident ratio within these units has also been shown to be marked [6,7]. Therefore, particular attention should be paid to these differences in performance and the potential association between efficiency and quality of care. Differences in efficiency may result in differences in the quality of care produced and the number of nursing staff.
Defining and measuring the quality of care in long-term care facilities is a multidimensional and complex issue with several pitfalls [8]. Firstly, the majority of the residents suffer from cognitive decline [7] and thus, the voice of the client might not reach the management. Decline in cognition may also indicate that residents cannot necessarily evaluate quality, which signifies an asymmetry in information between residents and management. Secondly, Kane [9] argues that the goals of care among the residents are far from uniform. A minority of this population consists of individuals with good prognosis and discharge potential. Kane [9] further states that high-quality care, in these settings, may also mean that the patient does less poorly than otherwise expected.
There are few studies of the relationship between staffing and quality in elderly care that show consistently that better staffing is associated with better quality of care [10]. In Finland the present discussion concerning public services is related mostly to productive efficiency or productivity. Unfortunately, there are not yet clear messages or tools for policy-makers, since the empirical evidence on the association between efficiency and quality in elderly care seems to be scarce and somewhat contradictory [1113].
The aim of this study was to consider the association between efficiency and quality of care in long-term care wards in Finland. Due to the multidimensional nature of quality, 41 quality indicators were used. In the first stage, instead of using traditional staffing figuressuch as the staff per resident ratiowe calculated the productive efficiency of the units. A units efficiency score comprises the following essential components: the number of staff, the amount of capital, the total number of in-patient days, and the case-mix. In the second stage, units were divided into poor- and good-quality groups and the efficiency scores of these groups were compared.
| Materials and methods |
|---|
|
|
|---|
Data
The observation unit in this study was a long-term care ward. For the study we needed information on the staff, the number of the beds, the case-mix, and the quality of care in the wards. Complete research data pertaining to 114 wards in 41 facilities were gathered from 1 January 2002 to 31 December 2002 from several sources. There were approximately 3300 patients on the wards.
Client-level assessments have been done using the Minimum Data Set 2.0 (MDS 2.0) questionnaire for nursing facilities [14]. In Finland, each patient is assessed by the staff with the assistance of the residents relatives both at admission and semi-annually, or when a distinct change in the patients condition takes place. Resource Utilization Groups (RUG-III) were used to represent the case-mix [15]. It can be constructed from the MDS together with clinical quality indicators indicating prevalence and incidence of adverse outcomes and care processes [14,16,17].
The RUG-III classification is internationally the most widely used and validated method for case-mix adjustment [1820]. In the USA, the RUG-III system forms the basis of prospective payment of publicly reimbursed nursing home care. RUG-III is based on information in the Resident Assessment Instrument (RAI) and its MDS, such as patients physical functioning and cognition. The RAI is a standardized assessment instrument for improving care planning and quality of care. In Finland, a 22-group version of RUG-III classification is used [19]. In the RUG-III/22 patients are placed into one of the seven clinical categories, which are divided into 22 subcategories. The subcategories measure how much labour resources a patient needs per day. Each of the subcategories is cost weighted, and the cost weight denotes the average proportional staff time per 24-hour period per patient. The cost weight for the average patient is set at 1.0. The cost weights for different patient groups vary from 0.48 to 2.52 in RUG-III/22, which denotes that there is a six-fold difference in average nursing time per 24-hours between patients. In this study we used a case-mix index, which was aggregated for the ward level from individual-level data.
Clinical quality indicators at ward level were calculated as mean values of measurements made during two MDS assessment periods (cross-sections) in 2002, in accordance with Zimmermann et al. [17]. The pooling together (calculated averages) of two cross-sections ensures more consistent quality measures and diminishes measurement errors and randomness. The prevalence and incidence of the quality indicators in relative terms were calculated by including all patients on the ward, or including only low-risk patients, or only high-risk patients. These clinical quality indicators are available for routine reporting from our RAI benchmarking database. A questionnaire filled in by the head nurses provided data on staff resources and structural quality measures, such as the number of staff per resident and the proportion of single rooms. The descriptive statistics of the quality variables are given in Table 1.
|
An average number of three different full-time equivalent staff categories and capital (unit size) were used as input. The in-patient days of the wards were obtained from the finance offices. We used the RUG-III/22 classification for case-mix adjustment [19], which was applied to the output variable, i.e. to the number of in-patient days. By applying the case-mix adjustment the wards are comparable in terms of the output measure, which is crucial in efficiency measurement. The descriptive statistics for the output and input variables are given in Table 2.
|
Statistical methods
Calculating efficiency. The non-parametric approach, data envelopment analysis, is a traditional method for calculating (in)efficiency [4,21]. Technical inefficiency in this context means that a technically efficient producer could produce the same output with less input when compared with the inefficient unit. Alternatively, a technically efficient producer could produce more output with the same input. These correspond to an input- or output-oriented data envelopment analysis model perspective, respectively. An illustrative example of the data envelopment analysis is given in Figure 1.
|
For example, consider a production technology of two inputs and one output. The units A, B, C, and D in Figure 1 produce the same output with different combinations of the inputs. The efficiency scores are determined by the ratio of the sum of output to the sum of corresponding input. The efficient units in the frontier (A, B, D) have a score of 1.0. The unit C is inefficient, because it uses more input for the same output compared with the other units. The efficiency score for unit C is about 0.75 (the ratio OC/OC), which denotes 25% inefficiency. Unit C could produce its output with 25% less input resources.
The frontier, isoquant Q1, in Figure 1 denotes the best practice production frontier. Linear programming is used in the data envelopment analysis to construct an empirical piece-wise linear efficient frontier which serves as a reference in the evaluation of efficiency. Data envelopment analysis can also be used for calculating allocative and cost efficiency, which also takes input mix and prices into account. Technical efficiency describes the efficiency of production, which is a necessary but not sufficient precondition for cost efficiency and lowest possible unit costs. In the framework of this study a ward is judged to be efficient if it is operating on the best practice production frontier (see Figure 1). Data envelopment analysis efficiency scores were calculated using the DEAP program written by Tim Coelli. Input price variables (wages) were not available for this study, so an analysis was restricted to exploring technical efficiency, using three categories of labour and capital variables as input, and case-mix adjusted in-patient days as output. Compared with traditional staffing figures, the efficiency score also takes case-mix and the amount of the output into account.
Quality measures as correlates of inefficiency.
We applied a totally non-parametric approach in the analysis of quality measures as correlates of technical inefficiency. We divided wards into two categories according to their rank in the quality distribution, considering all quality variables separately. If a ward was ranked in the zero to 85th percentiles in the quality distribution, it was placed in the group of good or acceptable quality. In contrast, wards that were above the 85th percentile threshold, in which the prevalence and incidence of the specific quality indicator was the highest, were placed in the group of poor quality. There may be severe quality problems in the group above the 85th percentile threshold, and therefore we pay particular attention to these wards in this study.
The traditional MannWhitney U-test was used to test whether the technical efficiency scores of the two sampled populations were equivalent in rank. The analysis was conducted with pooled data of health centre hospitals and residential homes (n = 114). In the sensitivity analysis we changed the thresholds from 085th percentiles (good quality) and above the 85th percentile (poor quality) to 075th percentiles (good quality) and above the 75th percentile (poor quality). In addition, we also combined the two groups and tested the correlation between efficiency and continuous quality indicators in the whole sample.
| Results |
|---|
|
|
|---|
In Figure 2 the Salter diagram illustrates the efficiency distribution. The width of the bars denotes the number of beds in the wards, which varied from 8 to 74. The mean technical efficiency in pooled data was 0.72. Hence, on average, wards could produce in-patient days with 28% less input resources. Four units out of 114 were fully efficient, i.e. efficiency scores were 1.0, which determined the production frontier. There was no clear association between unit size and technical efficiency.
|
Table 3 presents the results of the analyses tests. The third column denotes the mean rank in efficiency in the comparison groups. In the fourth column the asymptotic significance of the MannWhitney test is reported. The correlation coefficients between efficiency score and continuous quality indicators in the pooled data (n = 114) are presented in the fifth column. In the last column a positive sign denotes that the wards where the quality seems to be poorest operate more efficiently than wards where quality is good or adequate, according to clinical quality indicators and structural quality measures. A negative sign denotes that poor-quality wards are also less efficient than good-quality wards. The sign +/ denotes no clear association between quality and efficiency.
|
Positively significant clinical quality indicators in Table 3 are associated with clinically complex and very dependent patients. For instance, the mean rank for bedfast residents was 51 in the good-quality group compared with 92 in the poor-quality group, which indicates clearly that better efficiency is associated with nursing practices which lead to poorer quality in terms of a high number of bedfast residents. However, indicators describing less dependent patients, i.e. patients who walk, who have behavioural problems, or who are more medicated are negatively or non-significantly associated with efficiency. Furthermore, the lower the proportion of registered nurses and proportion of single rooms, the better the efficiency; whereas wards operated more efficiently when they had a higher number of rooms with en-suite toilets. The correlation coefficients between efficiency and quality indicators in the whole sample indicated consistent results with the MannWhitney tests. Hence, there also seems to be a linear association between quality indicators and efficiency scores.
In the sensitivity analysis we changed the threshold for poor quality from the 85th percentile to the 75th percentile. The differences in mean ranks between the groups were consistent (the results are not reported in the tables). The only exceptions were a prevalence of depression with no treatment and a proportion of rooms with their own toilet, in which there were no differences in efficiency between the groups. Neither of the correlations were significant.
| Discussion and conclusions |
|---|
|
|
|---|
There was great variation in efficiency and quality between the wards. Mean technical inefficiency was 28%, which was higher than was estimated in earlier studies [22]. The variation in quality also showed great differences between wards. Our findings showed that in 13 of 41 areas, poor quality was associated with high productive or technical efficiency (see Table 3), indicating also an association between costs, staffing, and quality of care; whereas in six areas the poor quality was related to low efficiency. In the remaining 22 areas no clear association was found.
Efficient wards had quality problems in the areas that indicate time-consuming nursing procedures. These procedures were not in place or they were inadequately performed because of a low staffing ratio. A high prevalence of bedfast residents, the use of physical restraints, a high percentage of inactive patients, and a lack of nursing rehabilitation are all different aspects of mobility restriction, a known risk factor in the disablement process [23]. Use of physical restraints has previously been shown to be associated with low staffing levels in the USA [24]. Grade 14 pressure ulcers can be seen as a consequence of both mobility restrictions [25] and incontinence. In addition, a high occurrence of pressure ulcers reflects an absence of or inadequate repositioning programmes in dependent patients. During the disablement process, acceleration of incontinence occurs. A high prevalence of occasional or frequent bladder or bowel incontinence without a toileting plan was associated with high efficiency, though the statistical significance level varied in the non-parametric analysis.
Efficient wards showed a low prevalence of falls. Physical restraints are usually used to prevent falls or injuries [26,27]. Conversely, it has been pointed out that the use of these devices has itself been a source of injuries [26]. In our study, a low prevalence of falls might simply reflect the highest possible level of disability of patients in the efficient wards: bedfast residents are less likely to fall [7].
Low efficiency was found to be associated with poor quality in the areas of disruptive behaviour and the use of antipsychotic medication in the absence of psychotic conditions. The presence of neurodegenerative and/or psychiatric disease is most often the source of disruptive behaviour in these settings [28,29]. In spite of physical disability, wards specializing in dementia care or in psychiatric careespecially those with severe behavioural disturbances as eligibility criteriaare traditionally provided with a higher staff ratio than ordinary wards [30]. In these areas the basic question of quality is whether the use of antipsychotic medication actually improves the individuals quality of life in the long run; a question still open to debate.
Quality measures related to nutrition were not associated with efficiency at all. This might reflect the allocation of nursing time as much as to prioritizing the most important areas: whatever the staffing ratio, issues of feeding and hydration are considered more important than rehabilitation. Neither was the high prevalence of anti-anxiety and hypnotic medication associated with efficiency. One explanation might be the wide use of such medication in all conditions and their unclear relationship with time-consuming care processes.
Our results revealed a clear association between efficiency and quality in the prevalence-type of quality indicators in the time-consuming areas only. However, incidence-type indicatorssuch as the incidence of decline in the late loss of activities of daily living and the incidence of decline in the range of motionwere the most important indicators to support our theory that caregivers adapt non-pro-active nursing patterns in the absence of adequate staffing ratios with regard to patients needs. In the analysis, incidence-types of indicator, however, failed to show significant relationships, conceivably due in part to the small number of observations or a weak relationship.
Due to the cross-sectional nature of our study, the observed relationships are associative. The efficiency estimates would also have been more consistent in panel data and the longitudinal follow-up of patients might have revealed causal relationships. The clinical quality indicators used in this study may also be associated with each other. For instance, there may be a positive correlation between the prevalence of bedfast residents and pressure ulcers. Although our non-parametric approach enabled the use of all quality indicators in the analysis separately, one might consider using factor analysis for combining different quality indicators. However, for the time being in Finland these clinical quality indicators are routinely used for benchmarking purposes. Our results showed that efficiency was associated with certain types of quality indicator or quality dimension, which also favours the possibility of reducing the number of quality dimensions. The results also indicate that the selection of quality variables may affect the interference if certain quality dimensions are omitted. The strength of the non-parametric approach is that several quality indicators can be used in the analysis.
There seems to be clear improvement potential both in productive efficiency and quality of care, which may result in better outcomes and a better utilization of labour input and cost containment in the long run, although the population is ageing and probably more resources will be needed for the care of older persons. The results support the fact that efficiency and quality of care are essential aspects of management and performance measurement. If there is an intention to increase productive efficiency in elderly careas there is in Finlandthen quality has to be taken into account, for instance by using quality monitoring systems, although the empirical evidence on the association between quality and efficiency is still not fully consistent [11,12]. The association between efficiency and poor quality of care was detected in some areas associated with non-pro-active nursing practice and the consequences of such patterns. Further studies with a greater sample of wards and with longitudinal construction are warranted to show whether or not the results of this study indicate a true trade-off between productive efficiency and care.
| Acknowledgements |
|---|
|
|
|---|
This study has been funded by the Academy of Finland (Academy decision notification number 70034), the National Research and Development Centre for Welfare and Health (STAKES), and the Doctoral Programme in Social and Health Services Management and Economics of the University of Kuopio. Acknowledgement is also given to interRAI.
| References |
|---|
|
|
|---|
- Newhouse J. Toward a theory of nonprofit institutions: an economic model of a hospital. Am Econ Rev 1970; 60: 6474.[Web of Science]
- Weech-Maldonado RJ, Neff G, Mor V. Does quality of care lead to better financial performance? The case of the nursing home industry. Health Care Manage Rev 2003; 28: 201216.[Web of Science][Medline]
- Hirschman AO. Exit, Voice, and Loyalty. Cambridge, MA: Harvard University Press, 1970.
- Farrell MJ. The measurement of productive efficiency. J R Stat Soc 1957; A120: 253281.
- Björkgren M, Häkkinen U, Linna M. A comparison of methods for determining efficiency of Finnish long-term care units. Sos Laaketiet Aikak 2000; 37: 216222 (in Finnish).
- Laine J, Linna M, Häkkinen U, Noro A. Measuring the productive efficiency and clinical quality of institutional long-term care for the elderly. Health Econ 2004; 14(3): 245256.
- Noro A, Finne-Soveri H, Björkgren M et al. Introduction of the RAI-System and Experiences of Benchmarking of Long-term Care for the Elderly. [RAI-tietojärjestelmän esittely ja kokemuksia pitkäaikaishoidon benchmarkingista.] RAI -raportti 1/2000. Helsinki: Stakes, 2001 (in Finnish).
- Berg K, Mor V, Morrris J, Murphy KM, Moore T, Harris Y. Identification and evaluation of existing nursing homes quality indicators. Health Care Financ Rev 2002; 23: 1936.[Medline]
- Kane R. Assuring quality in nursing home care. J Am Geriatr Soc 1998; 46: 232237.[Web of Science][Medline]
- Dellefield ME. The relationship between nurse staffing in nursing homes and quality indicators. J Gerontol Nurs 2000; 27: 1428.
- Knox KJ, Blankmeyer EC, Stutzman JR. Organizational efficiency and quality in Texas nursing facilities. Health Care Manag Sci 2003; 6: 175188.[CrossRef][Medline]
- Rosko M, Chilingerian J, Zinn J, Aaronson W. The effects of ownership, operating environment, and strategic choices on nursing home efficiency. Med Care 1995; 33: 10011021.[Web of Science][Medline]
- Chou S-Y. Asymmetric information, ownership and quality of care: an empirical analysis of nursing homes. J Health Econ 2002; 21: 293311.[CrossRef][Web of Science][Medline]
- Morris JN, Hawes C, Fries BE et al. Designing the National Resident Assessment Instrument for nursing homes. Gerontologist 1990; 30: 293307.[Abstract]
- Fries BE, Schneider DP, Foley WJ, Gavazzi M, Burke R, Cornelius E. Refining a case-mix measure for nursing homes: Resource Utilization Groups (RUG-III). Med Care 1995; 32: 668685.
- Hawes C, Morris J, Phillips C, Fries B, Murphy C, Mor V. Development of the nursing home Resident Assessment Instrument in the USA. Age Ageing 1997; 26 (Suppl 2): 1925.
- Zimmermann DR, Karon SL, Arling G et al.. Development and testing of nursing home quality indicators. Health Care Financ Rev 1995; 16: 107140.[Medline]
- Carpenter I, Perry M, Challis D, Hope K. Identification of registered nursing care of residents in English nursing homes using the Minimum Data Set Resident Assessment Instrument (MDS/RAI) and Resource Utilisation Groups version III (RUG-III). Age Ageing 2003; 32: 7.
- Björkgren M, Häkkinen U, Finne-Soveri H, Fries B. Validity and reliability of Resource Utilization Groups (RUG-III) in Finnish long-term care facilities. Scand J Public Health 1999: 27: 228234.[CrossRef][Web of Science][Medline]
- Carrillo E, Garcia-Altes A, Peiro S, Portella E. System for the classification of patients in mid and long-term care facilities: Resource Utilization Groups, version III. Validation in Spain. Revista de Gerontologia 1996; 6: 276284.
- Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units. Eur J Oper Res 1978; 2: 429444.[CrossRef]
- Björkgren M, Häkkinen U, Linna M. Measuring efficiency of long-term care units in Finland. Health Care Manag Sci 2001; 4: 193200.[CrossRef][Medline]
- Stuck AE, Walthert JM, Nikolaus T, Bula CJ, Hohmann C, Beck JC. Risk factors for functional status decline in community-living elderly people: a systematic literature review. Soc Sci Med 1999; 48: 445469.[CrossRef][Web of Science][Medline]
- Castle NG. Nursing homes with persistent deficiency citations for physical restraint use. Med Care 2002; 40: 868878.[CrossRef][Web of Science][Medline]
- van Marum RJ, Ooms ME, Ribbe MW, van Eijk JT. The Dutch pressure sore assessment score or the Norton scale for identifying at-risk nursing home patients? Age Ageing 2000; 29: 6368.
[Abstract/Free Full Text] - Evans LK, Strumpf NE. Tying down the elderly: a review of the literature on physical restraint. J Am Geriatr Soc 1989; 37: 6574.[Web of Science][Medline]
- Capezuti E, Strumpf NE, Evans LK, Grisso JA, Maislin G. The relationship between physical restraint removal and falls and injuries among nursing home residents. J Gerontol A Biol Sci Med Sci 1998; 53: M4752.[Abstract]
- Volicer L, Hurley AC. Management of behavioral symptoms in progressive degenerative dementias. J Gerontol A Biol Sci Med Sci 2003; 58: M837845.
[Abstract/Free Full Text] - Martin H, Slyk MP, Deymann S, Cornacchione MJ. Safety profile assessment of risperidone and olanzapine in long-term care patients with dementia. J Am Med Dir Assoc 2003; 4: 183188.[CrossRef][Medline]
- Vaarama M, Luomahaara J, Peiponen A, Voutilainen P. The Whole Municipality Working Together for Older People. Report. Helsinki: Stakes, 2001.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

