International Journal for Quality in Health Care Advance Access originally published online on August 12, 2005
International Journal for Quality in Health Care 2005 17(6):465-472; doi:10.1093/intqhc/mzi067
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Development and validation of an in-patient satisfaction questionnaire
1 Research Unit, Hospital of Galdakao, 2 Research Unit, Hospital of Basurto, Bilbao, Bizkaia, 3 Research Unit, Hospital of Txagorritxu, Vitoria-Gasteiz, Alava, Spain, 4 Department of Politics and The Graduate School in Social and Political Studies, University of Edinburgh, Edinburgh, United Kingdom, 5 Respiratory Service, Hospital of Galdakao, Galdakao, 6 Quality Unit, Hospital of Cruces, Cruces, and 7 Psychiatry and Psychology Service, Hospital of Basurto, Bilbao, Bizkaia, Spain
Objective. To develop a psychometrically sound, hospital patient satisfaction questionnaire to be administered to patients discharged from medical and surgical services.
Design. Cross-sectional survey in Spanish.
Setting. Four acute care general hospitals of the Basque Health Service.
Study participants. Random samples of 650 discharged patients from each hospital during February and March 2002. A total of 1910 patients responded to the questionnaire (73.5%).
Main outcome measures. Overall perceived quality of health care and perceived health improvement.
Results. No sociodemographic differences were found between respondents and non-respondents. Six dimensions were identified from the factor analysis, explaining 50% of the variance. All items, except two, revealed loadings above 0.4. Cronbach’s alpha exceeded 0.7 for all dimensions, except privacy. Comfort was the dimension with the lowest level of patient satisfaction, whereas privacy was the most satisfactory. The interscale correlations never exceeded the internal consistency of each scale. The analysis of the dimensions with two items of global assessment showed a positive correlation.
Conclusions. The results obtained from the development and validation of the questionnaire provide evidence of its psychometric properties, although it would be useful to carry out further analyses to assess time-based properties of reliability. We found a positive relation between the degree of patient satisfaction and overall evaluation of the quality of health care, providing evidence of the ability of the questionnaire to correlate with other concepts. The in-patient satisfaction questionnaire could become a useful instrument in quality-of-care assessment.
Keywords: patient satisfaction, psychometric properties, quality in health care, validity
Address reprint requests to Nerea González, Research Unit Hospital of Galdakao, BºLabeaga, S/N 48960 Galdakao, Spain. Tel: +34944007105, Fax: +34944007132. E-mail: uinves2{at}hgda.osakidetza.net
Accepted for publication July 6, 2005.
Traditionally, both direct and indirect outcome measures have been developed to assess the results of health care from the viewpoint of health care professionals. However, more recent studies have emphasized the need for assessment techniques to measure patient perceptions of health care quality, given that their perceptions can differ from those of professionals. Thus, patient perceptions have become a major indicator in the evaluation and improvement of quality in health care [1–5].
Patient satisfaction is one of the most commonly used outcome measures, as shown in the increased number of tools created to assess satisfaction in recent years. Some authors go so far as to consider patient satisfaction as one of the primary outcomes of health care [6].
Several approaches have been developed to survey patient perceptions of health care, such as suggestion boxes, formal complaints, qualitative methods, or audits. However, satisfaction questionnaires are undoubtedly the most commonly used method [7].
The patient satisfaction questionnaires must fulfill, among other requirements, certain psychometric properties, especially if the aim is to generalize the information to the target population [8]. Over the last 30 years, numerous questionnaires have been developed to measure patient satisfaction with hospital care [3,8–15]. However, not all the questionnaires developed so far have been evaluated regarding their psychometric properties, which reduces the usefulness of the results. The last 5 years have seen a change in this trend, with most studies now attempting to ensure that the validity of the instruments is well grounded [4,14,15]. However, each of those questionnaires has been developed in other countries, which makes it difficult to adapt them to our needs because of differences in culture or health systems. Also, some questionnaires for evaluating patient satisfaction with hospitalization have been developed in Spanish, but we could not use them in our study, because they were still under development during our study [16], they employed a different conceptual approach [17], or their psychometric properties had not been evaluated. These reasons made us decide to develop our own questionnaire.
The goals of this study were to develop an in-patient satisfaction questionnaire to be used in our health system, from which the underlying dimensions could be derived and individual patient scores calculated, and to evaluate some of the questionnaire’s psychometric properties, such as its content, criterion and construct validity, and internal consistency, as a measure of reliability.
 |
Materials and methods |
|---|
 Top Materials and methods ResultsDiscussionReferences |
|---|
Questionnaire development
Various sources and methods were used to determine the questions to be included in the questionnaire. First, a literature search was undertaken between January and April 2000, using MEDLINE and PSYCLIT databases, that aimed to analyse the instruments that had been devised so far to evaluate in-patient satisfaction at the national and international levels. The keywords used were patient satisfaction, questionnaires, quality in health care, psychometric properties, and validity.
Second, eight focus groups were conducted with patients and two with health care professionals to explore opinions about the most positive and negative aspects of care received during a hospital stay. These focus groups were geared toward gathering information and expressions that could be used to develop questions to be included in the questionnaire. The meetings were guided by two trained moderators, who transcribed and analysed the discussions of the participants, to obtain a list of the aspects considered most important.
Third, several areas were identified within the previous information and the research team developed a pool of items, in relation to these areas, to be included in the questionnaire. These items were shown to a group of patients and professionals, who provided their opinions about their appropriateness and comprehension and evaluated the content and face validity of the questions. An initial version of the questionnaire was then created, which was evaluated in one pilot study, to analyse the comprehensibility and clarity of the items and features related to the psychometric properties of the instrument. The results of the pilot study led us to make changes in the questionnaire that resulted in the instrument used during the fieldwork described in this study. These changes included the elimination of some items or the rewording of others because of a high non-response rate, response scales that showed little variability, or instances in which some patients chose more than one option.
The final questionnaire included 34 questions, which follow in chronological order the steps from the time the patient is admitted to the hospital until discharge. The questionnaire also contained sociodemographic variables, including age, sex, educational level, professional status, and marital status. The response scale that we used had a varied number of options, ranging from three to six.
Study participants
The study was conducted among patients admitted to any of four acute care general hospitals in the Basque Health Care Service. The hospitals were selected because of their different geographical locations.
Adult patients 18 years and older were included if they had been in the hospital for longer than 48 hours. Patients admitted to the Neurology Department were excluded because a high percentage of them had pathologies of the central nervous system (such as cerebrovascular disease) that could hinder or prevent their participation in the study. Patients with serious physical or mental pathologies, such as terminal disease and psychosis, which could make the comprehension and completion of the questionnaire difficult, were also excluded, as were patients whose destination after hospital release was different from their usual residence, given the difficulties associated with locating them.
Survey
Random samples of 650 patients who had been discharged between February and March 2002 were contacted in each hospital. Two weeks after discharge, the selected patients received the questionnaire with a prepaid return envelope. A covering letter was also attached that explained the reasons for conducting the survey, encouraged their participation, and guaranteed data confidentiality. A follow-up letter was sent to non-respondents 2 weeks later. If they still had not returned the questionnaire 15 days after the first reminder, they received a third letter with a new copy of the questionnaire.
The response rate obtained by this method was 73.5%. Of the 1910 patients who answered, 52.4% responded to the first mailing, 27.2% after receiving the first reminder, and 20.4% after receiving the second mailing. No statistically significant differences were found in sex and age among those who answered the questionnaire and those who did not. However, there was a difference in the hospital service, with a higher percentage of respondents having been admitted for surgery. In addition, there were differences among the four hospitals taking part in the study, in that one had a significantly lower rate of response (66.6%; P < 0.0001) than the others.
Statistical analysis
Descriptive statistics included the frequencies and percentages of the categorical variables and the means and standard deviations of the continuous variables. Principal component exploratory factor analysis was carried out using varimax rotation to determine the number of dimensions and to assess the construct validity of the questionnaire. Items were included in the dimensions if they revealed loadings greater than 0.4. In the case of multiple loading of an item on several factors, it was included in the factor that had more conceptual relationship. Factor loadings and communalities were calculated at the item level and eigenvalues and explained variance at the scale level.
When individual items from a domain were missing, the simple mean imputation method was used. This method consists of estimating the missing values from the mean value for those items that are available, but it can only be used if the respondent has completed at least half of the items in that domain. Once each of the questionnaire factors was identified, scores were calculated by adding the values attributed to the answers to all the items in each of the factors. A linear transformation was then carried out, so that the scoring scale for each dimension was standardized between 0 and 100, with a score of 100 indicating the highest level of satisfaction.
The factor analysis and the scores calculation were performed to test the main aim of the study. The hypothesis was that it is possible to obtain meaningful, independent dimensions to evaluate patient satisfaction.
For each of the dimensions, we calculated Cronbach’s alpha coefficient [18] to assess the internal consistency and the correlations with the other dimensions.
Finally, the scores for each of the six dimensions were compared with two items of global assessment, to probe the hypothesis that there is a positive relationship between each dimension of patient satisfaction and their overall evaluation of the quality of health care and their health improvement. The first item evaluated the perception of the global quality of care during the hospital stay, with a response scale ranging from ‘poor’ to ‘excellent’. The second item analysed the patients’ perceived health improvement in relation to the problem that led to their hospital admission, with a response scale ranging from ‘none’ to ‘totally’. The analysis of variance with Tukey’s test for multiple comparisons and the Kruskal–Wallis test was used.
Patient responses were entered into a database prepared using Microsoft Access, and data analysis was subsequently performed using SAS for Windows version 8.2 [19].
|
Results |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
Table 1 summarizes the sociodemographic data for the respondents and the non-respondents.
|
All 34 items were included in the factor analysis, from which we derived six dimensions that explained 50% of the variance. The dimensions identified were information and medical care (12 items), nursing care (8 items), comfort (6 items), visiting (4 items), privacy (2 items), and cleanliness (2 items). All items showed loadings above 0.4, except for two variables that concerned ‘nurses’ explanations about the disease’ (0.364) and the patient ‘wake-up time’ (0.395) (Table 2).
|
Cronbach’s alpha coefficient results were above 0.7 for all dimensions except privacy, where a coefficient of 0.60 was obtained from its two items. For cleanliness, however, the coefficient was 0.74, despite only two items, which revealed the high consistency between them (Table 2).
The mean scores and the medians show that comfort was the dimension that presented the lowest level of satisfaction, whereas privacy had the highest scores. The floor and ceiling effects corroborated the high degrees of satisfaction observed in the mean scores and the medians. Floor effect percentages were low in all dimensions, but the same was not the case for ceiling effects, since a percentage of 77.35% was found for privacy and 64.18% for cleanliness (Table 3).
|
Regarding inter-scale correlations, they were never higher than the internal consistency of each scale. However, a high correlation of 0.62 was found between information and nursing care (Table 4).
|
We found a positive relation between the six dimensions of the questionnaire with the item that measures the patients’ perceptions of the global quality of health care during the hospital stay. Regarding information, nursing care, comfort, and cleanliness, statistically significant mean differences were found among all categories of each global variable. Regarding the visiting and privacy dimensions, differences were found only between the patients who rated the attention most negatively compared with the rest (Table 5).
|
In relation to the other global question that measured patients’ perceived health improvement, we detected higher scores on all dimensions as the perception of improvement was higher. We found statistically significant differences between those who said that his or her problem was totally solved compared with the others.
Finally, by age groups, the scores of all our areas were lower on the younger category (Table 6).
|
|
Discussion |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
The results obtained from the development and validation of the questionnaire provide evidence of its good psychometric properties.
The items were generated from information derived from different sources, including a literature review, focus groups with patients and professionals, and expert opinion. This lends support to the content validity of the questionnaire. In addition, once the questionnaire was developed and analysed in a pilot study, it was revised with patients and professionals to improve it and assess its face validity from both perspectives.
The results of the factor analysis showed six dimensions that generally had good internal consistency, even if those dimensions comprised only two items. This multidimensional structure matches the findings of many investigators in the analysis of patient satisfaction [10,20–24]. Furthermore, the specific dimensions obtained through this questionnaire are similar to those found in other instruments [3,4,12,25], which partially supports the construct validity of our tool.
The patients who took part in the study were generally highly satisfied with the quality of care. However, there was variability in the degree of satisfaction depending on the dimension. Although there was high satisfaction with privacy and cleanliness, it was lower in the areas of comfort and nursing care. These results contrast with those obtained in other studies [26–28], in which there was a clear bias toward very positive evaluations in all the dimensions that were analysed.
We also confirmed the hypothesis that on the one hand there is a positive relationship between patient satisfaction and their overall evaluation of the quality of health care received during their hospital stay and, on the other, the improvement in their well-being [15,29,30]. Such a positive relationship establishes a tendency toward showing higher degrees of satisfaction when patients’ perceptions of their health and the quality of care received are higher. This points toward greater criterion validity within this instrument.
Differences were not found in the sociodemographic variables of age and sex between respondents and non-respondents. This finding, together with the high response rate obtained, allowed us to generalize the conclusions to the target population. In addition, the response rate justified the data collection method that we used. The method of mailing questionnaires, coupled with follow-up letters to non-responders, facilitated high response rates, which had been confirmed previously [31–33]. However, it is still possible that the representativeness of the sample could be biased, due to the loss of data from the non-respondents and also due to missing answers at the item level. In our case, the minimum item response rate was 81.1% and the maximum was 89.7%, which is very encouraging.
In relation to the comparison of the satisfaction scores by age, we have found that younger patients show lower satisfaction scores. In any case, whether the sociodemographic variables are ‘crucial’ is highly debatable, because the only consistent evidence is that there is an age effect [7], as we have also shown in our study.
The results of these analyses showed preliminary support for the content validity, construct validity, and criterion validity of the questionnaire. Either our factor structure as well as the internal reliability results were, at least, as good as other patient satisfaction questionnaires [13,15,34].
Regarding reliability, only the internal consistency of the questionnaire dimensions was analysed using Cronbach’s alpha, which revealed results above the recommended minimum of 0.7 [35]. They can be considered good results, especially if we take into account that two of the dimensions had just two items. Furthermore, these results are consistent with those from other studies [15,29,30].
Evaluation of the psychometric properties of satisfaction questionnaires has been the main goal of several studies published during the last 15 years [13–15,34]. The analyses carried out all coincide in their aim to evaluate the internal consistency and the construct validity of the questionnaires, two properties that were also evaluated for the questionnaire discussed in this report.
Other psychometric properties that were evaluated in the previous studies included time-based measures, such as test–retest reliability [14] and predictive validity [34]. In our study, we also analysed the relationship between the questionnaire dimensions and both the patients’ perception of the global quality of health care and their perceived health improvement during hospitalization. However, we did not evaluate test–retest reliability or time-based criterion validity. Further studies should concentrate on evaluating the reliability of our instrument with these other time-based techniques.
To create this questionnaire, we followed the recommendations for the development of questionnaires: (1) generate the items by conducting a search of the bibliography, evaluating qualitative techniques, and consulting with experts; (2) conducting a pilot study to correct those questions that did not work; and (3) studying some of its psychometric properties. Nevertheless, our results still show that some questions and domains deserve improvement and some other psychometric properties have to be checked. In fact, two domains, privacy and cleanliness, had high ceiling effects and low Cronbach’s alpha so both need a deep review to improve these results.
Another possible limitation concerns the time of the year when data collection took place. Seasonal factors or vacation periods can determine the type of pathologies in hospitals and the resultant workload for hospital workers, which in turn can affect the assessments of hospital stay. For that reason, it might be advisable to carry out satisfaction surveys at different times of the year whenever possible.
We would recommend in future that information is obtained about patient health status, because several authors have shown that it affects patients’ perceptions of the health care received during their hospital stay and, thus, satisfaction [3,13,14].
These limitations should be taken into account when generalizing the results to the population and also when trying to use this new questionnaire in other settings or countries.
This questionnaire has undergone preliminary validation among a wide sample of patients. We showed that it is possible to achieve good response rates and reasonably discriminating scales. In addition, it is a short questionnaire, from which it is easy to calculate scores for each of its six dimensions, thus facilitating the presentation of results.
The in-patient satisfaction questionnaire described in this study could thus become a useful instrument in quality-of-care assessment, either being used by clinicians or managers. They should focus on the dimensions that excel as well as those that are open to improvement. In this way, patients are treated as one of the main sources of valuable insights into health care organizations, while at the same time recognizing that they can be one of the key beneficiaries of the improvements in the hospital environment derived from this type of study.
This study was supported by a grant from the Fondo de Investigación Sanitaria (FIS 00/0129). The authors of this article thank all personnel at the data gathering units of the four hospitals and the Research Committee of Galdakao Hospital for help in translating and editing this article.
The authors have no proprietary interest in any aspect of this report.
|
References |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
- FitzpatrickR. Surveys of patient satisfaction: I–important general considerations. BMJ 1991; 302: 887–889.
[Free Full Text] - SitziaJ, Wood N. Response rate in patient satisfaction research: an analysis of 201 published studies. Int J Qual Health Care 1998; 10: 311–317.
[Abstract/Free Full Text] - ClearyPD, Edgman-Levitan S, Roberts M et al. Patients evaluate their hospital care: a national survey. Health Aff 1991; 10: 254–267.[Medline]
- HendriksAA, Vrielink MR, Smets EM, van Es SQ, De Haes JC. Improving the assessment of (in) patients’ satisfaction with hospital care. Med Care 2001; 39: 270–283.[CrossRef][Web of Science][Medline]
- Van CampenC, Sixma H, Friele RD, Kerssens JJ, Peters L. Quality of care and patient satisfaction: a review of measuring instruments. Med Care Res Rev 1995; 52: 109–133.
[Abstract/Free Full Text] - DonabedianA. Evaluating the quality of medical care. Milbank Mem Fund Q 1996; 44: 166–203.[CrossRef]
- CrowR, Gage H, Hampson S et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess 2002; 6: 1–245.[Medline]
- DoeringER. Factors influencing inpatient satisfaction with care. QRB Qual Rev Bull 1983; 9: 291–299.[Medline]
- ThompsonAGH. The Measurement of Patients’ Perceptions of the Quality of Hospital Care. Unpublished doctoral thesis, University of Manchester, 1983.
- HardyGE, West MA, Hill F. Components and predictors of patient satisfaction. Br J Health Psychol 1996; 1: 65–85.[CrossRef][Web of Science]
- La MonicaEL, Oberst MT, Madea AR, Wolf RM. Development of a patient satisfaction scale. Res Nurs Health 1986; 9: 43–50.[Web of Science][Medline]
- NelsonEC, Hays RD, Larson C, Batalden PB. The patient judgment system: reliability and validity. QRB Qual Rev Bull 1989; 15: 185–191.[Medline]
- RubinHR, Meterko M, Nelson EC, Ware JE Jr, Hays RD. Patient judgments of hospital quality. Report of a pilot study. Med Care 1990; 28 (suppl.): S1–S56.
- SalomonL, Gasquet I, Mesbah M, Ravaud P. Construction of a scale measuring inpatients’ opinion on quality of care. Int J Qual Health Care 1999; 11: 507–516.
[Abstract/Free Full Text] - LabarereJ, Francois P, Auquier P, Robert C, Fourny M. Development of a French inpatient satisfaction questionnaire. Int J Qual Health Care 2001; 13: 99–108.
[Abstract/Free Full Text] - Agency for Healthcare Policy and Research. CAHPS 2.0 Survey and Reporting Kit. Rockville, MD: Agency for Healthcare Policy and Research, 1999.
- Mira SolvesJJ, Buil Aina JA, Rodriguez-Marin J, Aranaz Andres J. Calidad percibida del cuidado hospitalario. Gac Sanit 1997; 11: 176–189.[Medline]
- CronbachLJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297–334.[CrossRef][Web of Science]
- SAS Institute Inc. SAS/STAT User’s Guide. Version 8. Cary, NC: SAS Institute, 2000.
- Linder-PelzSU. Toward a theory of patient satisfaction. Soc Sci Med 1982; 16: 577–582.[CrossRef][Web of Science][Medline]
- HallJA, Dornan M. What patients like about their medical care and how often they are asked: a meta-analysis of overall satisfaction levels. Soc Sci Med 1988; 27: 935–939.[CrossRef][Web of Science][Medline]
- SinghJ. The patient satisfaction concept: a review and reconceptualization. Adv Consum Res 1989; 16: 176–179.
- MiraJJ, Vitaller J, Aranaz J, Herrero JF, Buil JA. La satisfacción del paciente: concepto y aspectos metodológicos. Rev Psicol Salud 1992; 4: 89–116.
- BakerR. Pragmatic model of patient satisfaction in general practice: progress towards a theory. Qual Health Care 1997; 6: 201–204.
[Abstract/Free Full Text] - RubinHR. Can patients evaluate the quality of hospital care? Med Care Rev 1990; 47: 267–326.[Medline]
- MiraJJ, Buil JA, Aranaz J et al. ¿Qué opinan los pacientes de los hospitales públicos? Análisis de los niveles de calidad percibida en cinco hospitales. Gac Sanit 2000; 14: 291–293.[Medline]
- JenkinsonC, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care 2002; 11: 335–339.
[Abstract/Free Full Text] - SmithC. Validation of a patient satisfaction system in the United Kingdom. Qual Assur Health Care 1992; 4: 171–177.[Medline]
- HaysRD, Larson C, Nelson EC, Batalden PB. Hospital quality trends. A short-form patient-based measure. Med Care 1991; 29: 661–668.[CrossRef][Web of Science][Medline]
- FinkelsteinBS, Singh J, Silvers JB, Neuhauser D, Rosenthal GE. Patient and hospital characteristics associated with patient assessments of hospital obstetrical care. Med Care 1998; 36: AS68–AS78.
- HarveyL. Factors affecting response rates to mailed questionnaires: a comprehensive literature review. J Market Res Soc 1987; 29: 341–353.
- AschDA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997; 50: 1129–1136.[CrossRef][Web of Science][Medline]
- TaiSS, Nazareth I, Haines A, Jowett C. A randomized trial of the impact of telephone and recorded delivery reminders on the response rate to research questionnaires. J Public Health Med 1997; 19: 219–221.
[Abstract/Free Full Text] - CareyRG, Seibert JH. A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care 1993; 31: 834–845.[Web of Science][Medline]
- NunnallyJC, Bernstein IH. Psychometric Theory. New York: McGraw-Hill, 1994.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. K Matis, T. A Birbilis, and O. I Chrysou Patient satisfaction questionnaire and quality achievement in hospital care: the case of a Greek public university hospital Health Serv Manage Res, November 1, 2009; 22(4): 191 - 196. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Antoniotti, K. Baumstarck-Barrau, M.-C. Simeoni, C. Sapin, J. Labarere, L. Gerbaud, L. Boyer, C. Colin, P. Francois, and P. Auquier Validation of a French hospitalized patients' satisfaction questionnaire: the QSH-45 Int. J. Qual. Health Care, August 1, 2009; 21(4): 243 - 252. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Moret, J.-M. Nguyen, C. Volteau, B. Falissard, P. Lombrail, and I. Gasquet Evidence of a non-linear influence of patient age on satisfaction with hospital care Int. J. Qual. Health Care, December 1, 2007; 19(6): 382 - 389. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||