International Journal for Quality in Health Care Advance Access originally published online on December 11, 2007
International Journal for Quality in Health Care 2008 20(2):95-104; doi:10.1093/intqhc/mzm061
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comparing patient reports about hospital care across a Canadian-US border
1 Department of Health Policy, Management, and Evaluation, University of Toronto, 155 College Street, Suite 425, Toronto, ON, Canada M5T 3M6
2 Niagara Health Quality Coalition, 5820 Main Street, Suite 501, Williamsville, NY 14221, USA
Address reprint requests to: Bruce Boissonnault, Niagara Health Quality Coalition, 5820 Main Street, Suite 501, Williamsville, NY 14221, USA. Tel: +1-716-250-6472; Fax: +1-716-250-4329; E-mail: bruce{at}nhqc.com
| Abstract |
|---|
|
|
|---|
Objective. To compare patient reports about hospital care between western New York State and southern Ontario using a random intercept model.
Method. Cross-sectional survey of 3923 patients who received medical or surgical care between August and October 2004 at 28 hospitals (14 hospitals per jurisdiction). Thirty-five questions were combined to calculate eight indicators with scores ranging from 0 to 100 (best care experience). For each indicator, a model was built where the region (western New York vs. southern Ontario) was included as a fixed effect with hospital as random within region. A number of patient characteristics were also included as fixed effects.
Results. The effect of the region was statistically significant (P < 0.05) only for the models predicting the continuity and transition, involvement of family and physical comfort indicator scores. The differences were 10.66, 4.05 and –3.23 points, respectively. In all three models, the random intercepts were not statistically significant, indicating that the differences above did not vary by hospitals. The model predicting overall impression scores, however, showed a random intercept statistically significant (P = 0.026). The individual-level explained proportion of variance ranged from 5.68 to 11.22%, and the hospital-within-region-level explained proportion of variance ranged from 2.19 to 52.28%.
Conclusion. The difference observed on the continuity and transition indicator might be the only one somewhat meaningful, and might be explained by health maintenance organization reimbursements' mechanisms and hospital quality improvement initiatives available in western New York, as well as by the fact that occupancy rates in western New York border the 60% compared with the 95% in southern Ontario.
Keywords: cross-jurisdictional comparisons, hospital performance indicators, perceptions of hospital care
Patient satisfaction or patient care experiences are an increasingly common and important component of a comprehensive assessment of the quality of care [1–9]. Evidence also shows that satisfied patients are more likely to cooperate with their treatment [5, 9–11], which, in turn, is associated with better clinical outcomes [9, 10]. International comparisons of performance measures, such as patients' perceptions of their care or their satisfaction, may provide an important opportunity for (i) identifying best practices relevant to several decision-makers such as hospital managers, insurance payers, and government policy-makers and (ii) promoting policy debates and discussions about the effectiveness of differing health systems. Differences in demographics, culture, as well as issues with the translation of surveys, may reduce the value of these analyses to decision-makers [4]. However, if differences between two groups of patients are controlled for (including expectations if possible to measure), then some of the variation not accounted for might be due to real differences in perceived quality between health systems. The present study contrasts patient reports about hospital care between two jurisdictions with distinctly different health systems: western New York State (USA) and southern Ontario (Canada). Canadian hospitals, organized as public or not-for-profit organizations, operate under annual budgets largely set by provincial and regional health authorities, while US hospitals operate within decentralized, competitive delivery systems [12]. Again, adjusting patient perceptions of their care for patient and institutional characteristics [6, 12–15] can enhance the comparability across countries by reducing the impact of factors that might not be within the control of healthcare organizations. In this study and slightly different from current international comparisons [4, 5, 16, 17], we used multilevel (or hierarchical) models to investigate differences in patient ratings about the hospital care and a number of additional patient characteristics including sex, age, self-reported health status, days spent in bed during the last month, number of times in hospital overnight or longer in the last six months, education and a square age term. A few reasons became visibly appealing when this study was conceived: (i) both jurisdictions have been using the same Picker based-family survey for a number of years now contributing to readily available data, (ii) both jurisdictions are separated only by a river but the way healthcare is delivered and organized is distinctly different, (iii) both jurisdiction's data were large enough to perform statistical analyses and came from random samples and (iv) last, but not least, western New York hospitals have been involved in a large number of quality improvement activities during the last 10 years; some of them as a result of the use of the Picker survey. This experience might be instrumental when promoting collaborative quality improvement activities across jurisdictions.
| Methods |
|---|
|
|
|---|
Two databases containing the results of an inpatient survey were obtained from the Hospital Report Research Collaborative [18] in Toronto, Ontario and from the Niagara Health Quality Coalition [19] in Buffalo, New York. Both organizations regularly release public reports on the performance of hospitals including measurements built from patient survey data. The data included patients that received care services at 14 hospitals in western New York State (4 teaching and 10 non-teaching) and at 14 hospitals in southern Ontario (4 teaching and 10 non-teaching) between August and October 2004. In both jurisdictions, the survey process was administered by NRC + Picker (Lincoln, NE & Markham, ON) using a two-wave approach. Random samples were drawn from records containing medical or surgical patients 18 years of age or older with a length of stay of 1 day or more. In western New York, eligible patients totaled 5619 and in southern Ontario, 2509. The first wave of surveys was mailed between 7 and 9 weeks after patients had been discharged. Approximately 22 days after, there was a second wave of mailings. Returned questionnaires totaled 2945 in western New York and 1211 in southern Ontario representing a response rate of 52.4 and 50.4%, respectively. In both jurisdictions, there were no statistically significant differences in sex or length of stay between respondents and non-respondents, however, respondents were older than non-respondents. A total of 233 records were excluded because of missing values. The final samples included 2749 records from western New York and 1174 records from southern Ontario (total N = 3923 patients). Table 1 shows patient characteristics from both jurisdictions along with non-parametric tests for differences.
|
Hospital bed sizes ranged from 70 to 911 (median: 195) in western New York and from 21 to 567 (median: 208) in southern Ontario. Western New York State and southern Ontario are contiguous geographic areas separated only by the Niagara River. Each of these regions has a population of nearly 1.2 million and includes at least one large city; Buffalo in western New York and Hamilton in southern Ontario. The ratio of beds per 100 000 habitants is 343 in western New York and 232 in southern Ontario.
The questionnaire used in both jurisdictions was developed by the Picker Institute and has been extensively used in the USA since its development in the 1980s. In Canada, this instrument has been modified, pilot tested and validated by the NRC + Picker Canada (Markham, ON), and has been in use since 2002. The basic Picker instrument contains 40 standard items and a number of additional questions which are utilized depending on the country in which they are used and the requirements of individual hospitals [3]. The development of the basic Picker instrument involved consultation with experts, a systematic literature review and an organization of patient focus group and in-depth interviews [3]. The 40 standard items were grouped on the basis of their face validity into eight dimensions: (i) continuity and transition, (ii) coordination of care, (iii) emotional support, (iv) information and education, (v) involvement of family and friends, (vi) physical comfort, (vii) respect for patient preferences and (viii) overall impression. The items included in each dimension can be found in the study published by Jenkinson et al. [3].
In this study, the questionnaires obtained from each region included 53 and 68 closed-ended items in southern Ontario and western New York, respectively. Thirty-five items were identical and were retained for the analysis. These 35 items are those standard items developed by the Picker Institute and covered all eight dimensions published by Jenkinson et al. [3]. The dropped items were those specific to the jurisdictions and hospitals. For example, what health insurance plan do you use to cover most or all of your medical care? was used in western New York but not in southern Ontario. Owing to a lack of tendency to retain items for comparisons, we did not introduce any bias in the selection of the items used. The Hospital Report Research Collaborative however, moved five items from the original overall impression dimension. Three were moved to the dimension respect for patient preferences and two to the dimension coordination of care. The names of these two dimensions were slightly modified. This re-arrangement was based on feedback from patient focus groups, hospital staff surveys and discussion among team members [20]. We used the aforementioned 35 identical questions to calculate 8 indicators (dimensions) for each patient. An indicator score was calculated only if at least half of the items used to build that indicator had valid responses. All responses were transformed into a 0–100 scale with higher scores indicating more favorable experiences with hospital care. A five-point scale including ratings of 1, poor; 2, fair; 3, good; 4, very good and 5, excellent were converted to the scores of 0, 25, 50, 75 and 100, respectively. The three- and seven-point scales were converted to scores of 0, 50 and 100, and 0, 16.7, 33.3, 50, 66.7, 83.3 and 100, respectively. The scores of the items under each indicator were averaged to build the indicator score. This approach of scoring is different from that used by the Picker Institute, in which each item is coded as a dichotomous problem score, indicating the presence or absence of a problem [3]. We decided to use a 0–100 scale since it is more sensitive to differences (e.g. improvement) than the Picker Institute approach which does not seem to capture entirely gradations of patients' evaluations. Table 2 presents the eight dimensions (will be referred to as indicators hereinafter), the items used to build each indicator, their scales, response options, patient-level item and indicator scores and a test for differences.
|
To test the effect of the region (that is, western New York vs. southern Ontario) on the indicator scores, we used the random intercept model [21]. For each of the eight indicators, a model was built where the region was included as a fixed effect with hospital as random within region. A number of patient characteristics recommend to be used as covariates were also included as fixed effects [8]. All variables used in this study are presented in Table 3. The random intercept model was built according to the procedure described by Snijders and Bosker [21] and utilized by Sjetne et al. [8] and Sixma et al. [9]. Only statistically significant fixed effects were retained in the final models. The level-one (patient characteristics and region) and level-two (hospital within region) explained proportion of variance were estimated. For the latter, we used the harmonic mean of 47. A P-value of 0.05 was used to assess significant effects. We also weighted each record in the sample using 4009 beds in western New York (range 70–911) and 2408 acute beds in southern Ontario (range 10–510). Acute beds were available in southern Ontario, thus they were used instead. Weighted data were used in all analyses. Statistical analyses were performed using SPSS 16.0 for Windows (SPSS Inc., Chicago, Illinois, USA).
|
| Results |
|---|
|
|
|---|
Table 4 presents the results of the three random intercept models where there was a region effect. All complete eight models are available upon request to the authors. The effects of patient characteristics on the indicator scores can be summarized as follows: older, male, less educated and healthier (self-rated) patients tended to rate the care provided by the hospital higher (i.e. more favorable care experiences) than younger, female, more educated and sicker (self-rated) patients. Similarly, patients who had fewer hospital stays during the last 6 months and fewer days in bed at home in the last month tended to evaluate their hospital experiences better than those who had more hospital stays and spent more days in bed. The effect of the region (western New York vs. southern Ontario) was statistically significant at P < 0.05 only for the models predicting the continuity and transition, involvement of family and physical comfort indicator scores (Table 4). The differences were 10.66, 4.05 and –3.23 points, respectively. In all three models, the random intercepts were not statistically significant. This indicates that the differences above did not vary by hospitals. The model predicting overall impression scores, however, showed a random intercept statistically significant a P = 0.026 with a variance component of 16.20 (model not presented in Table 4). Although the region effect was not statistically significant in this model, two similar patients (one from each hospital-region) might still show a difference depending on the hospital providing care. The individual-level explained proportion of variance ranged from 5.68% (physical comfort indicator) to 11.22% (coordination of care and access indicator). The hospital-within-region-level explained proportion of variance ranged from 2.19% (emotional support indicator) to 52.28% (continuity and transition indicator). These values indicate that we were able to reduce a fair amount of error when predicting an individual score as well as a hospital mean score.
|
Additional analyses showed that there was a positive hospital outlier in western New York (data not shown). That is, one hospital scored consistently higher than all other western New York hospitals in all indicators. This situation did not occur among southern Ontario hospitals. There were no negative hospital outliers. To test whether the positive hospital outlier in western New York was affecting the differences between the two areas, we removed the cases from that hospital (N = 173) and performed the random intercept models again. No changes were observed in any of the variables. This positive hospital outliner was highly focussed on cancer patients, younger treated patients and those patients whose length of stay was longer than those treated in the other 13 western New York hospitals.
| Discussion |
|---|
|
|
|---|
Despite some slight differences, the results of this study show that, overall, patients' evaluations of the hospital care received were similar between western New York and southern Ontario. Some of the slight differences observed, however, deserve some comments. First, the differences of 4.05 and –3.23 points on the involvement of family and physical comfort indicators are smaller than a small effect size defined by Cohen [22]. Thus, these differences are not meaningful and further support our overall conclusion. Second, in the model predicting overall impression scores, the region was not statistically significant. However, we might still see a difference between a patient treated in a western New York hospital and one treated in a southern Ontario hospital, even with similar characteristics. This is because, in this model, the random effect with a variance component of 16.20 (SD = 4.02) was statistically significant. And third, the difference of 10.66 points on the continuity and transition indicator could be classified as a small-medium effect size under Cohen's definition of effect sizes [22]. As such, this difference might be more meaningful from a practical perspective and therefore, part of the discussion in this study will be focussed on explaining this difference.
Health care systems in the USA and Canada differ widely in health care expenditures (15 vs. 10% of gross domestic product) and government expenditures on health (45 vs. 70% of total health expenditures) [23]. Besides these system-level markers, western New York and southern Ontario have also other distinctive differences. In 2004, the medical-surgical occupancy rates were in the range of 60% in western New York (Erie and Niagara Counties) [24]. In southern Ontario (counties surrounded Hamilton) the occupancy rates bordered the 95% in 2004 [25]. In 2002, Buffalo in western New York presented a health maintenance organization penetration rate of 64% which was the fourth highest nationwide [26]. Buffalo was also found to be a fairly competitive market with a high degree of quality incentives in use [26]. In southern Ontario (and Canada overall) there is a rather different dynamic. There are no such health maintenance organizations, close to 85% of hospital operating revenues comes from funds provided by the Ministry of Health on an annual basis [25], and competition is quite limited. In fact, a 2004 survey conducted in Ontario among hospitals' chief executive officers showed that less than 10% of hospitals compete with other health care organizations for patients, and that 90% of teaching hospitals engage in collaboration (data not published but available from the Hospital Report Research Collaborative—University of Toronto, Canada) [18].
The differences in health systems above might be a key aspect when performing cross-jurisdictional comparisons since they most likely shape patient expectations, and in turn, patient perceptions of the hospital care received. If there are differences in patient expectations, then they might be one source of variation of patient ratings across jurisdictions. Although we did not measure patient expectations, we believe it is unlikely they account for the small-medium difference observed on the continuity and transition indicator. In fact, Blendon et al. found that Americans had modestly higher expectations for their health-care arrangements than Canadians had, and that differences in patient expectations were in general not large [27]. Assuming that patient expectations played a small role in explaining the differences observed in the continuity and transition indicator, then some other sources of variation can be hypothesized as follows. First, the existence of health maintenance organizations, which focus on high levels of productivity and reductions in health service utilization [28], might have played a role. As mentioned earlier, Buffalo was among the geographic sites with the highest health maintenance organization penetration nationwide in 2002, and was defined as a market ripe for incentive programs [26]. If patients are well informed at the time discharged, it seems reasonable to argue that they will be less likely to be readmitted due to a better self-care management at home. This can be seen in Table 2 where the item staff explained when to resume usual activities (work, drive etc.) showed a difference of 18.2 points between the two jurisdictions. Therefore, health maintenance organizations might have some quality incentive programs in place to avoid patient readmissions. To our knowledge, health maintenance organizations do pay for performance in western New York State (internal communications with Niagara Health Quality Coalition). For example, hospitals with lower avoidable readmission rates receive higher reimbursements from health maintenance organizations. Also, western New York hospitals whose patient survey scores improve receive higher reimbursement from health maintenance organizations. Second, another explanation of the difference in the continuity and transition indicator might relate to some internal executive performance bonus programs for senior hospital administrators available in western New York hospitals (internal communications with Niagara Health Quality Coalition). For example, the head of nursing at each hospital receives performance bonuses based on improvements in patient survey scores. These internal hospital-based incentives will be soon implemented widely to more hospital staff across western New York hospitals. In this line, it is also expected that the federal government (i.e. Centers for Medicaid and Medicare Services) will eventually base their payments on patient survey results. And third, another explanation of the difference in the continuity and transition indicator could relate to the fact that in western New York hospitals, where occupancy rates border the 60% (compared to 95% in southern Ontario), staff have more time to explain transition issues such as the purpose of the medicines to be taken at home, when to resume usual activities etc. We believe that the three main sources of variation hypothesized above are reasonable ones and measuring them in future research should be considered. Table 5 summarizes overall health system differences between western New York and southern Ontario, as well as the results of this study.
|
Of interest, other studies comparing patient satisfaction across nations have shown similar results [3, 4, 16, 17]. That is, differences were similar to those found in this study at the indicator and item levels. Particularly, patients from the USA scored consistently higher on similar themes to those captured in our continuity and transition indicator and consistently lower on those captured in our physical comfort indicator. These similarities, especially the former, reinforce our hypothesis regarding the influence of health maintenance organizations on hospital performance.
The low response rate in this study (close to 50%) is a potential limitation; however, it does not seem to jeopardize the generalizability of our results since the difference between respondents and non-respondents on age was modest. Furthermore, age and other socio-demographic characteristics are minor predictors of satisfaction with low explanatory power [29]. Therefore, while a larger sample might have affected the statistical significance of the differences, it would not necessarily have impacted the effect sizes. Finally, our data came only from patients who received care between the months of August and October of 2004. This might be a potential limitation for generalizability, if seasonality is a factor that explains patient satisfaction. Despite these limitations and other difficulties with international comparisons [4], our findings reinforce those of others reported in the literature and highlight opportunities to debate about the effectiveness of different health-care systems. We have shown that patients from two distinctly different health systems have reasonably similar perceptions on the hospital care received. The only somewhat meaningful difference found on the continuity and transition indicator might be the result of health maintenance organization reimbursements' mechanisms and internal hospital-based quality improvement initiatives available in western New York, as well as the fact that occupancy rates in western New York border the 60% compared with the 95% in southern Ontario.
From a quality improvement perspective, it would be important to focus efforts on the two items that showed the highest difference between the two jurisdictions: staff explained when to resume usual activities (work, drive etc.), and staff explained danger signals about the illness or operation to watch at home. Collaboration across the border seems to be a reasonable next step. Niagara Health Quality Coalition has been improving quality through collaboration for a number of years now, and to our knowledge, there are more than 100 quality improvement activities in place (internal communications with Niagara Health Quality Coalition). In addition, the score of the continuity and transition indicator in southern Ontario is far below the scores of the other seven indicators, and patient satisfaction has been identified as a challenge or major challenge by nearly 80% of Ontario hospitals [30]. The low performance on issues related to provider–patient communication about discharge planning is not new. In 1994, Charles et al. [31] reported provider–patient communication about discharge planning as one of the most problematic areas as reported by patients in Canadian hospitals.
| Acknowledgments |
|---|
We thank Paula Blackstien-Hirsch and Brian D'Arcy for their valuable contributions on previous versions of this manuscript.
| References |
|---|
|
|
|---|
- Calnan M, Katsouyiannopoulos V, Ovcharov VK, et al. Major determinants of consumer satisfaction with primary care in different health systems. Fam Pract (1994) 11:468–78.
[Abstract/Free Full Text] - Bredart A, Robertson C, Razavi D, et al. Patients' satisfaction ratings and their desire for care improvement across oncology settings from France, Italy, Poland and Sweden. Psychooncology (2003) 12:68–77.[CrossRef][Medline]
- Jenkinson C, Coulter A, Bruster S. The picker patient experience questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care (2002) 14:353–8.
[Abstract/Free Full Text] - Coulter A, Cleary PD. Patients' experiences with hospitals care in five countries. Health Aff (2001) 20:244–52.
[Abstract/Free Full Text] - Larsson BW, Larsson G, Chantereau MW, et al. International comparisons of patients' views on quality of care. Int J Health Care Qual Assur (2005) 18:62–73.[CrossRef]
- Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: effects of demographic and institutional characteristics. Med Care (2000) 38:325–34.[CrossRef][Web of Science][Medline]
- Murray CJ, Frenk J. A framework for assessing the performance of health systems. Bull World Health Organ (2000) 78:717–31.[Web of Science][Medline]
- Sjetne IS, Veenstra M, Stavem K. The effect of hospital size and teaching status on patient experiences with hospital care. A multilevel analysis. Med Care (2007) 45:252–8.[CrossRef][Web of Science][Medline]
- Sixma HJ, Spreeuwenberg PMM, Van Der Pasch MAA. Patient satisfaction with the general practitioner. A two-level analysis. Med Care (1998) 36:212–29.[CrossRef][Web of Science][Medline]
- Walker MS, Ristvedt SL, Haughey BH. Patient care in multidisciplinary cancer clinics: does attention to psychosocial needs predict patient satisfaction? Psychooncology (2003) 12:291–300.[CrossRef][Medline]
- Pascoe GC. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann (1983) 6:185–210.[CrossRef][Medline]
- Blendon RJ, Schoen C, DesRoches CM, Osborn R, Zapert K, Raleigh E. Confronting competing demands to improve quality: a five-country hospital survey. Health Aff (Millwood) (2005) 23:119–35.[CrossRef]
- Rahmqvist M. Patient satisfaction in relation to age, health status and other background factors: a model for comparisons of care units. Int J Qual Health Care (2001) 13:385–90.
[Abstract/Free Full Text] - Hargraves JL, Wilson IB, Zaslavsky A, et al. Adjusting for patient characteristics when analyzing reports from patients about hospital care. Med Care (2001) 39:635–41.[CrossRef][Web of Science][Medline]
- Finkelstein BS, Singh J, Silvers JB, Neuhauser D, Rosenthal GE. Patient and hospital characteristics associated with patient assessments of hospital obstetrical care. Med Care (1998) 36((8 Suppl)).
- Davis BA, Bush HA. Patient satisfaction of emergency nursing care in the United States, Slovenia, and Australia. J Nurs Care Qual (2003) 18:267–74.[Web of Science][Medline]
- Blendon RJ, Schoen C, DesRoches CM, Osborn R, Scoles KL, Zapert K. Inequities in health care: a five-country survey. Health Aff (Millwood) (2002) 21:182–91.
[Abstract/Free Full Text] - Hospital Report Research Collaborative, Toronto, Ontario. (27 February 2007, date last accessed.
- Niagara Health Quality Coalition. (27 February 2007, date last accessed). Williamsville, New York.
- Loreti M, Murray M. Hospital Report 2005: Acute Care Patient Satisfaction Technical Summary (2005) (27 February 2007, date last accessed. Toronto, ON.
- Snijders T, Bosker R. Multilevel Analysis: An Introduction to Basic and Advance Multilevel Modeling (1999) London: Sage Publications Ltd.
- Cohen J. Statistical Power Analysis for the Behavioral Sciences (1998) 2. New Jersey: Lawrence Erlbaum Associates.
- The World Health Organization. (27 February 2007, date last accessed. Geneva, Switzerland.
- Commission on Health Care Facilities in the 21st Century, New York. (27 February 2007, date last accessed. New York.
- Ministry of Health and Long-Term Care, Finance and Information Management Branch, Ontario Government, Toronto, Ontario. (27 February 2007, date last accessed.
- National Health Care Purchasing Institute. Recommended Health Care Markets for Provider Incentive Demonstrations (2002) (31 June 2006, date last accessed. Washington, DC.
- Blendon RJ, Benson J, Donelan K, et al. Who has the best health care system? A second look. Health Aff (1995) 14:220–30.[Abstract]
- Cutler DM, McClellan M, Newhouse JP. How does managed care do it? Rand J Econ. (2000) 31:526–48.
- Hall JA, Dornan MC. Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis. Soc Sci Med (1999) 30:811–8.[CrossRef]
- Brown AD, Alikhan LM, Sandoval GA, Seeman N, Baker GR, Pink GH. Acute care hospital strategic priorities: perceptions of challenges, control, competition and collaboration in Ontario's evolving healthcare system. Healthc Q (2005) 8:36–47.[Medline]
- Charles C, Gauld M, Chambers L, O'Brien B, Haynes RB, Labelle R. How was your hospital stay? Patients' reports about their care in Canadian hospitals. CMAJ (1994) 150:1813–22.[Abstract]
![]()
CiteULike
Connotea
Del.icio.us What's this?
This article has been cited by other articles:
![]() |
O Groene, N Klazinga, K Walshe, C Cucic, C D Shaw, and R Sunol Learning from MARQuIS: future direction of quality and safety in hospital care in the European Union Qual. Saf. Health Care, February 1, 2009; 18(Suppl_1): i69 - i74. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
