OUP user menu

Semi-customizing patient surveys: linking results and organizational conditions

Erik Riiskjær, Jette Ammentorp, Jørn Flohr Nielsen, Poul-Erik Kofoed
DOI: http://dx.doi.org/10.1093/intqhc/mzr001 284-291 First published online: 9 February 2011

Abstract

Objective The study investigated the needs and consequences of semi-customizing patient satisfaction surveys to low organizational levels and explored whether patient satisfaction was correlated with local organizational conditions.

Design From 1999 to 2006, the County of Aarhus carried out 398 surveys during four rounds in eight hospitals. To explain differences between the wards, data on the 40 wards with the best and the 40 wards with the worst evaluations (identified by patient surveys) were compared with the data from job satisfaction surveys and management information systems.

Setting Eight public hospitals in a Danish county.

Participants 32 809 inpatients and 1842 nurses on 84 wards.

Main Outcome Measure Optimal organizational level for measuring patient satisfaction and correlations between overall patient satisfaction and organizational context.

Results In all, 71.4% of the departments chose to have the survey results specified at the subunit level or for specific diagnostic groups. Substantial differences in patient satisfaction between wards are illustrated. On the wards with the highest improvement potential, we found significantly higher occupancy rates, acute rates, rates of sickness absenteeism, staff perceptions of high workload and low experience of professionalism.

Conclusions The study confirmed that departments desired individual, detailed descriptions of the results. Differences in patient satisfaction were associated with differences in organizational conditions. Establishing a link between patient satisfaction and organizational variables broadens the quality development focus to include more than simply analysis of specific questions. Semi-customizing patient surveys are recommended.

  • patient satisfaction
  • perceived usefulness
  • evaluation
  • organizational conditions

Introduction

Patient satisfaction surveys are claimed to have two goals: (i) to create transparency and (ii) to improve patient care. Experience indicates that these goals are seldom realized [1, 2]. In particular, the use of patient surveys as a tool for change is neglected to a problematic degree [3, 4]. One reason for that might be that patient surveys are often collected and reported at high organizational levels (e.g. hospital level) [46]. Results at the hospital level are difficult to follow up on, because hospitals are heterogeneous organizations with many small, relatively autonomous units. In this way patient surveys might miss their contribution to making clinical practice more patient centred [7].

Only a few studies have tried to determine the optimal organizational level for patient satisfaction surveys [8, 9]. These analyses indicate that low organizational levels are a source of more variation in patient satisfaction than higher organizational levels, and that even the team level might be of particular interest [8]. These finding are in accordance with more normative recommendations for quality development that is supposed to take place at low organizational levels (e.g. ward level) [10].

In accordance with surveys at high organizational level, the methodological literature has primarily focused on creating surveys that are valid in a narrow technical sense [2, 6]. Lowering the organizational level of focus and increasing focus on follow-up, strengthen the importance of perceived usefulness [1113]. In that context, the semi-customization of patient surveys to small organizational units would seem to be key, because it would create ownership of and responsibility for results and follow-up on the part of the staff supposed to use the results [14].

Lowering the organizational focus in patient surveys also challenges the question about what factors influence the patients’ perceptions. At the hospital level, size of hospital, hospital status and patient characteristics have been given attention, but lowering organizational focus makes it possible to draw attention to more local organizational antecedents for patient satisfaction, e.g. occupancy rates, sickness absenteeism and different dimensions of job satisfaction [1517]. Also with regard to job satisfaction surveys, the ward level has been recommended as the relevant organizational level [18].

To improve the perceived usefulness of patient satisfaction surveys and to relate results to lower organizational levels, the County of Aarhus in 1999 created a new patient survey system that differed from others by: (i) allowing all participating departments to determine how detailed they wanted the results to be (e.g. organized at the ward level or for specific diagnosis or patient subgroups), (ii) offering all departments extensive patient comments and (iii) providing all departments with their own semi-customized reports, including comments and figures at the departmental and customized subunit levels. The design was based on methods to cope with different types of patients, different requests from different specialties and different traditions in organizing the work across departments.

In a previous article, we examined the effects of using detailed patient surveys in a time series perspective for initiating change processes [14]. In this article, we will examine the needs and consequences for customizing patient satisfaction surveys and we will explore whether patient satisfaction is correlated with local organizational conditions with special attention to inpatients at the ward level.

Methods

Patient surveys were carried out four times at 2 year intervals (from 1999 to 2006) in eight public hospitals.

Included patients

Inpatients, outpatients, day surgery patients and medical day-care patients were included. For each of the four patient subgroups, up to 400 patients in each department received a questionnaire for periods of up to 3 months. A department could use more than one type of questionnaire. The questionnaires were sent by the departments to patients selected at random from the patient registration system using the Danish 10-digit person identification number. The surveys were anonymous and no reminders were sent.

Questionnaires to the patients

Four questionnaires were developed, one for each patient subgroup, i.e. inpatients, outpatients, day surgery patients and medical day-care patients. All questions were based on well-known problems identified from the existing literature on patient satisfaction surveys. In particular, a Danish qualitative study about lack of communication, coordination and continuity seen from the patient perspective was used in compiling the preliminary drafts [19]. These were prepared in cooperation with four to five staff from two departments and with two consultants. The questionnaires were validated through interviews with 66 patients and discussed and approved in a steering group consisting of doctors, nurses and managers. Psychometric properties were tested after the first round of surveys using explorative factor analysis, e.g. revealing three dimensions for the inpatients. This was only used for validation purposes, because the steering group had decided from the beginning that reporting should take place at single item level for pedagogical reasons.

Responses to questions about overall satisfaction were rated on a 5-point satisfaction scale ranging from ‘excellent’ to ‘unacceptable’. In addition, the questionnaires contained 10–14 specific questions, answered using a 4-point scale, i.e. ‘Yes’, ‘Both or neither’, ‘No’ and ‘Don't know’. The 4-point scale is especially suited for uncovering perceptions as opposed to satisfaction, which calls for more detailed scales. Furthermore, the 4-point scale was designed to encourage respondents to add qualitative comments to their score for each item. The idea was to counteract the tendency of patient surveys to deliver overly positive results [2, 20], and allow for more individualized feedback from the patients [7].

Variables were added concerning patient age, education, gender and form of admission.

Specification of subunits and groups of diagnosis/patients

With the help of the consultants, each department determined which of the four questionnaires was appropriate for its patients. This took place at a meeting with the management of each department. Furthermore, the department determined how the patients should be divided into subunits and into robust diagnosis and patient subgroups. One department, for example, decided to distinguish between patients with and without a cancer diagnosis. Another department wanted to focus on possible differences between patients transferred to another hospital and those discharged to return home.

Reporting of patient satisfaction

Two to 3 months after the departments had sent questionnaires to their patients, the departments received a semi-customized report for each patient subgroup. An automatic IT system gathered data and transcribed comments from the patients into a report, supplemented with the results of former surveys and benchmarked with other departments.

Data on job satisfaction

To investigate the correlation between nurse job satisfaction and patient satisfaction, we used job satisfaction data from a survey of 1842 nurses on 84 wards in the County of Aarhus carried out from April 2002 to April 2005.

The survey consisted of an overall question answered using a 10-point scale and 40 questions answered using a 4-point scale. These questions were classified into six dimensions based on theoretical considerations [21] (Table 1). The response rate was 81.0%. The job satisfaction data were gathered at the subunit level and analysed together with the corresponding patient surveys completed between August 2003 and April 2004.

View this table:
Table 1

Questions and dimensions in the job satisfaction surveya

Decision autonomy, Cronbach's alpha = 0.77 (0.69)
 I find that staff are involved appropriately when decisions are made.
 I am satisfied with my opportunity to influence the planning of my work.
 I have the information I need to perform my work.
 I know what is expected from my work.
Management, Cronbach's alpha = 0.96 (0.91)
 Management is motivating and inspiring.
 Management switches appropriately between listening and talking.
 Management has a good grasp of the workplace.
 It pays to talk to management about difficulties in my work.
 Conflicts are solved appropriately at the workplace.
 The manager is capable of decisive action.
 Management gives me favourable and critical comments in a way that motivates me to improve my work.
 My effort is appreciated.
Skill discretion, Cronbach's alpha = 0.90 (0.84)
 I am satisfied with the challenges I get through my work.
 The work gives me the opportunity to exercise my personal and professional skills.
 My job is exciting.
 My work is appropriately varied. My workplace gives me the opportunity for personal and professional development.
Cooperation, Cronbach's alpha = 0.94 (0.85)
 I feel comfortable with the rhetoric and social conventions at the workplace.
 I find that the workplace is characterized by a cooperative spirit.
 We treat each other as equals and with respect.
 We support each other in difficult situations at work.
 At the workplace, we give each other favourable and critical comments in a way that motivates me to improve my work.
 In my experience, there is equality of opportunity at my workplace (e.g. across age, gender and ethnicity).
 Everybody can express his or her opinion freely at my workplace.
Workload, Cronbach's alpha = 0. 83 (0.67)
 I can combine work demands with a good private life.
 I am satisfied with my amount of daily work.
Professionalism, Cronbach's alpha = 0.92 (0.86)
 The products of my workplace are of high quality.
 There is agreement between our daily work and our values and goals.
 In my experience, there is agreement concerning the values and goals of the workplace.
 At the workplace, we are preoccupied with improving the quality of the work.
 Time and energy are deployed for the right purposes.
 My workplace enjoys respect among users and collaborators.
 If something does not work, it is addressed appropriately.
  • aTheoretical consideration formed six dimensions on the basis of 40 items. Seven items are not included. Items are measured on a 4-point scale. Cronbach's alpha (CA) is estimated from that part of the data that pertains to the hospital area (6401 answers). Data were gathered from April 2002 to April 2005. CA is shown in parentheses for all county employees (11 544 answers).

Data from the management information system

To investigate the correlation between patient satisfaction and selected organizational variables, we used data from the county's management information system (MIS), which cover length of stay, number of beds, bed occupancy rates and staff sickness absenteeism. MIS data selected for each ward covered the same period as covered by the patient surveys.

Data selection and handling

In analysing correlations between patient satisfaction and organizational conditions, we only included the inpatients, because the most critical respondents and the most units with data from both patients and staff were found among the inpatient units.

In order to be able to show robust correlations at ward level and because data from the patient survey were positively skewed the data were dichotomized. ‘Yes’ were categorized as ‘answer without potential for improvement’, while ‘Both or neither’ and ‘No’ were categorized as ‘answers with potential for improvement’. Similarly, the answers ‘Excellent’ and ‘Good’ to the overall question were categorized as ‘answers without potential for improvement’, and ‘Good and bad’, ‘Bad’ and ‘Unacceptable’ were categorized as ‘answers with potential for improvement’.

Based on overall patient satisfaction, the wards were divided into two groups, one comprising the two best quintiles as identified by the patients and another comprising the two worst quintiles. Organizational data and data on nurse job satisfaction were described and analysed according to these two groups of wards. Differences between the two groups of wards according to selected organizational variables were found by t-test.

The organizational level of analysis, chosen by departmental management, is compared with the data clustering according to three potentially relevant organizational levels [22], i.e. the subunit/ward, department and hospital levels. We performed a multilevel analysis using SPSS version 16.0. The parameter of interest is the intra-class correlation (ICC), which yields the percentage of variance in patient satisfaction scores due to differences in explained variance between subunits/wards, departments or hospitals.

Empty models were estimated for each relevant organizational level. In this context, we are only interested in differences in explained variance between organizational units, because variance due to patient characteristics is less interesting from a design perspective [23].

Results

A total of 398 surveys were distributed at the departmental level from 1999 to 2006; 75 769 patients answered the surveys, giving a response rate of 57.0%. Of these, 32 809 respondents were inpatients, corresponding to a response rate of 53.5%.

Overall, 71.4% of the departments decided to have the answers specified at the subgroup level (Table 2). Most departments wanted the answers to be specified at the subunit level (45.7%) or according to diagnostic group (42.4%). Specification both at the subunit level and based on diagnosis was most frequently requested for inpatient surveys. The desired level of specification remained constant across all four periods. The subunits included wards with an average of 15.7 beds (CI: 14.8–16.2).

View this table:
Table 2

Percentages of departments in the County of Aarhus that wanted their results reported at the departmental level (no specification), at the subunit level or on the basis of diagnosisa

NNo specificationaSpecified at subunit levelaSpecified on diagnosisa
Inpatients
 20004524.460.042.3
 20024522.268.945.5
 20043815.871.147.4
 20063920.564.151.3
 Total inpatients16721.065.846.1
Outpatients
 20004233.326.242.9
 20024341.930.334.9
 20043528.637.237.2
 20063522.937.742.9
 Total outpatients15532.331.338.4
Day-patients
 20002035.030.050.0
 20021947.336.820.0
 20041936.826.442.1
 20061833.322.350.0
 Total day-patients7638.128.840.6
All surveys39828.645.742.4
  • aDo not total 100%, because 16.8% of departments wanted specifications based on both subunits and diagnosis.

The multilevel analysis of data clustering according to the three organizational levels reveals ICC coefficients of 4.5% in relation to the hospital level, 5.6% for the departmental level and 7.8% for the more detailed ward level chosen by the managers. Although not statistically significant, the tendency indicates that a substantial part of the variance is at the patient level and that the detailed organizational level explains more than the departmental and the hospital levels. The analysis is only performed for the overall question.

The potential for improvement among inpatients is presented in Table 3, which presents the average scores for the whole county and for the wards with the best and worst scores. To avoid outlier phenomena, the best- and worst-evaluated wards are calculated as the average scores for the three worst and three best wards, respectively, across the four rounds of surveys. The biggest difference between the best and worst wards is seen for the question about ward facilities, and the smallest difference for the question about how patients were received at the ward. An average ward produced 82 answers.

View this table:
Table 3

Potential for improvement among inpatients

Questions from the patient satisfaction surveyAverage at the county levelaBest-evaluated wardsbWorst-evaluated wardsb
Percentages of ‘Both or neither’ and ‘No’ answers
 Did you receive a good welcome at the department?13.30.036.1
 Are you satisfied with the treatment of your illness?13.41.138.7
 Did the doctors listen to you with interest when you said something?13.70.041.2
 Did you get the personal support you needed from the staff during your admission?14.41.139.8
 Did you receive careful nursing during your admission (from all the staff you were in contact with)?16.12.338.7
 Were your examination and treatment well planned during your contact with the hospital (‘a main thread’)?19.43.143.1
 Did you get the information you needed during your admission (e.g. about your illness, examinations, treatments and side effects)?20.04.041.7
 Was there a clear coherence in what you were told, when you talked to various staff in the department?23.65.053.3
 Was the accommodation adequate (e.g. bath, toilet and patient sitting room)?24.03.950.5
 Were you allowed to stay at the department until you felt ready to leave?25.04.851.9
 Did you get the information you needed before leaving the department (e.g. medicine and good advice)?25.10.672.3
 Has the collaboration between your GP and the department about your illness been satisfactory (e.g. referral and follow-up)?25.68.952.5
Percentages of ‘Yes’ answers
 Did you have to talk to too many doctors during your admission (only latest admission)?14.62.743.6
 Did you have to talk to too many staff during your admission (only latest admission)?18.31.352.1
Percentages of ‘Good and bad’, ‘Bad'or ‘Unacceptable’ answers
 What is your overall impression of the ward?16.80.043.2
  • Results from 399 wards for the four rounds of surveys performed, 1999–2006. Total number of patients included was 32 809.

  • aAverages at the county level are calculated without weighting for differences in ward size.

  • bBest- and worst-evaluated wards are calculated as the average of the three best- and three worst-scoring wards, respectively, across the four rounds of surveys. Only wards with >20 answers are included. An average ward has 82 answers.

The 40 wards with the highest scores on the overall patient satisfaction question (the two highest quintiles) are characterized by low acute rates (P < 0.01), low bed occupancy rates (P < 0.01), high experience of nurse professionalism (P < 0.01), low workload (P < 0.01) and low sickness absenteeism (P < 0.05) (Table 4).

View this table:
Table 4

Best- and worst-evaluated wards identified by patients,a correlated with patient characteristics, organization and nurse job satisfaction, 2003–04

Number of wardsThe 40 best evaluated wards, average in % (CI)cThe 40 worst evaluated wards, average in % (CI)cP-valueb
Patient satisfaction
 Percentage of satisfied patients (overall)8091.5 (90.3–92.6)73.4 (71.6–75.1)<0.01
Patient characteristics
 Percentage of patients with higher education8031.8 (27.7–35.9)25.3 (21.6–29.1)<0.05
 Percentage of patients >70 years of age8026.4 (19.8–33.0)35.4 (28.9–41.8)
 Percentage of women8053.2 (46.0–60.4)55.7 (51.8–59.5)
 Percentage of acute patients8042.6 (32.8–52.4)68.5 (61.4–75.7)<0.01
Organization
 Number of available beds7612.6 (10.6–14.6)15.6 (12.7–18.4)
 Occupancy rate7283.4 (78.4–88.3) 96.2 (89.7–102.8)<0.01
 Length of stay (days)76 4.7 (3.9–5.5) 6.3 (5.1–7.5)<0.05
 Sickness absenteeism66 5.3 (4.6–5.9) 6.4 (5.5–7.2)<0.05
Job satisfactiond
 Decision autonomy6874.2 (71.2–77.1)71.9 (69.5–74.4)
 Management6869.1 (64.3–73.8)68.5 (64.2–72.7)
 Skill discretion6879.7 (77.7–81.8)79.5 (77.6–81.4)
 Cooperation6878.9 (75.7–82.1)78.6 (75.8–81.4)
 Workload6870.0 (66.1–73.9)62.8 (59.3–66.3)<0.01
 Professionalism6872.1 (69.1–75.2)65.2 (61.8–68.7)<0.01
 Overall job satisfaction687.5 (7.2–7.9)7.2 (6.8–7.5)
  • aThe two groups were segregated based on patient answers to the overall question. The 20 wards in the middle quintile are not shown in the table.

  • bT-test.

  • cMean is computed without weighting for differences in ward size.

  • dIndex from 0 to 100. The items are presented in Table 1.

  • Complete data on job satisfaction and patient satisfaction from 84 of 100 wards.

  • Complete organizational and patient satisfaction data from 76 of 100 wards.

The percentage of women, elderly patients, as well as the ward size, is not correlated with patient satisfaction at the ward level.

Discussion

The study indicates that hospital departments need patient feedback at a specific, detailed and individually chosen level below the departmental level. Results collected according to these organizational specifications indicate substantial variation in patient satisfaction between subunits. The variation also seems to be correlated with a range of variables traditionally collected in hospitals. By establishing a link between patient satisfaction and organizational variables, the focus for follow-up is broadened, including not only analyses of specific questions, but also consideration of resources, job satisfaction and the nature of the specialty.

In this study, the organizational level preferred by managers is identical to the level that explains the greatest variation in patient satisfaction, indicating that the ward level is an attractive organizational level for the analysis of patient satisfaction. On the other hand, semi-customizing patient surveys at the subunit level, is more resource intensive, because it calls for larger samples at each level to secure reliable quantitative results.

Previous studies at low organizational levels demonstrate a positive correlation between overall patient satisfaction and overall job satisfaction, though the correlation is surprisingly weak [16] or even non-existent [24]. Corroborating these results, our study documents only a weak correlation between overall patient satisfaction and overall job satisfaction. However, we find interesting correlations between overall patient satisfaction and specific dimensions of job satisfaction, workflow perception and professionalism, corresponding to the results of a recent large study of the subject [17]. However, it is impossible to assess the direction of any potential causal effect.

Other studies have confirmed high occupancy rates as an important organizational variable affecting mortality [15] and adverse events [25]. The present study found occupancy rates to be correlated with patient satisfaction from a cross-sectional perspective. Another study, in a Danish department, documented a correlation between the timely intake of acute patients and patient satisfaction [26]. Many of the organizational variables seem to be interrelated, e.g. high acute rates might cause high occupancy rates.

When using patient surveys at high organizational level, it is traditionally recommended that the results of patient surveys are corrected for differences in patient mix before comparing different organizational units [9, 27]. Some research, however, has objected to this recommendation, because it can ‘hide’ the actual number of unsatisfied patients [23] because a department, as part of a patient-centred strategy, can only refer to the actual mix of patients and not to an average computed abstraction. In fact, the adaptation of consultations to different patients is the true measure of patient centredness [7]. The present study focusing on organizational subunits at a low level indicates that it might be more productive to focus on differences in subunit organizational conditions in order to supplement the agenda in the follow-up process with relevant organizational variables.

In this study, analysing data from various sources caused problems of comparability, as the relevant data were not coordinated at low organizational levels, for example, the measurement periods and the definitions and demarcations of the small organizational units. We estimated that this would weaken the correlation between job satisfaction and patient satisfaction, but without creating any bias, because no systematic time lags between patient surveys and job satisfaction surveys were found. However, these problems highlight an administrative obstacle to routine organizational analysis at low organizational levels.

The overall response rate of the surveys was 57%, better than for most patient surveys [28]. Still, this response rate leaves a potential for bias. In this study, bias could have influenced the level of satisfaction, probably in the critical direction, as there are more answers from women and younger people. However, such a possible bias is expected to influence each ward in the same direction and is thus less relevant when wards are compared.

By requesting detailed answers, the departments signal that they want patient surveys to be more precise and informative and not only produce information requested at the departmental and hospital levels. It seems obvious to conclude that the lack of follow-up on patient satisfaction surveys can partly be ascribed to the lack of precision and consequent lack of usefulness of the feedback obtained. Without customization attempts, patient surveys tend to be treated as rituals, rather than as quality improvement tools.

The trend of using generic questionnaires designed by non-clinicians and focusing on results to be reported on high organizational levels might waste evaluation capacity. Semi-customizing can be seen as a reaction toward this trend and is a tool to secure some kind of interest in the clinic.

Choosing patient survey methods calls for balancing the interests of politicians, managers, professionals and patients. Different parts of the health care system might have different ideas as to what constitutes an ideal patient survey system. So far, politicians and managers seem to have had the biggest influence on the design of patient surveys, reflecting the changing logic in health care from the values of professionals to the values of managers [29]. In this way, patient surveys have become more a part of marketing and less a quality improvement tool for the clinic. This also means that patient perspectives are hidden behind high reported levels of satisfaction [2].

The approach of this study was to make patient surveys more patient centred [7], allowing patients to express their individual experiences, focusing on the relevant small organizational units and from the perspective of selected diagnostic groups. As health-care systems become more specialized, fragmented and standardized, it becomes a challenge to design patient survey systems in ways that uncover the patient perspective and make it possible for clinicians to relate to the results. Our response to this challenge is to semi-customize generic patient surveys. Another response is to question the concept of generic patient surveys by creating local initiatives reflecting locally relevant themes of interest to professionals and patients. New methods using electronic questionnaires have facilitated the tailoring of questions to subunits and specific patient categories [26].

In conclusion, this study demonstrates that detailed, individually designed patient surveys are requested by departmental managers, and that patient satisfaction differs considerably at the ward level. Patient perception of care seems to be correlated with classical environmental conditions in the departments, indicating that follow-up may benefit from a broader organizational approach. We recommend considering more customization when planning generic patient satisfaction surveys in order to increase the probability that such surveys will have useful effects. This approach will be suitable for other settings as well. However, the success depends on the motivation of the leaders and other key persons, the engagement in quality development in general and the culture of the setting.

Funding

This work was supported by Region Midt, Trygfonden [Jnr. 7547-07], Momsfonden [Jnr. 11.20.00 G01].

Acknowledgement

We thank Ugeskrift for Læger for allowing us to use Tables 1–4 from an article in their journal (in Danish).

References

View Abstract