International Journal for Quality in Health Care 11:319-328 (1999)
© 1999 International Society for Quality in Health Care
How valid and reliable are patient satisfaction data? An analysis of 195 studies
Research and Development Department, Worthing Hospital, Lyndhurst Road, Worthing BN11 2DH, UK. Tel: + 44 1903 285224. Fax: + 44 1903 285217. E-mail: john.sitzia@wash-trsthames.nhs.uk
Objective.To assess the properties of validity and reliability of instruments used to assess satisfaction in a broad sample of health service user satisfaction studies, and to assess the level of awareness of these issues among study authors.
Design.Examination and analysis of 195 papers published in 1994 in 139 journals. The following databases were searched: British Nursing Index, CINAHL, EMBASE, Medline, Popline, and PsycLIT.
Main measures.Number and types of strategies used for content, criterion, and construct validity, and for stability and internal consistency. Associations between validity/reliability and other study characteristics.
Results.Eighty-nine (46%) of the 195 studies reported some validity or reliability data; 76 reported some element of content validity; 14 reported criterion validity, with patient's intent to return the most commonly used criterion; four reported construct validity. Thirty-four studies reported internal consistency reliability, 31 of which used Cronbach's coefficient 
; eight studies reported test-retest reliability. Only 11 studies (6% of the 181 quantitative studies) reported content validity acid criterion or construct validity acid reliability. 'New' instruments designed specifically for the reported study demonstrated significantly less evidence for reliability/validity than did 'old' instruments.
Conclusion.With few exceptions, the study instruments in this sample demonstrated little evidence of reliability or validity. Moreover, study authors exhibited a poor understanding of the importance of these properties in the assessment of satisfaction. Researchers must be aware that this is poor research practice, and that lack of a reliable and valid assessment instrument casts doubt on the credibility of satisfaction findings.
Keywords: data collection, meta-analysis, patient satisfaction, psychometrics, questionnaires, reproducibility of results
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  T. Heidegger, M. Nuebling, D. Saal, and G. Kreienbuhl Patient-centred outcomes in clinical research: does it really matter? Br. J. Anaesth., January 1, 2008; 100(1): 1 - 3. [Full Text] [PDF]
|
|
|
|
 |
|
 S. Oltedal, A. Garratt, O. Bjertnaes, M. Bjornsdottir, M. Freil, and M. Sachs The NORPEQ patient experiences questionnaire: Data quality, internal consistency and validity following a Norwegian inpatient survey Scand J Public Health, October 1, 2007; 35(5): 540 - 547. [Abstract] [PDF]
|
|
|
|
 |
|
 J. H. Bjorngaard, T. Ruud, A. Garratt, and T. Hatling Patients' Experiences and Clinicians' Ratings of the Quality of Outpatient Teams in Psychiatric Care Units in Norway Psychiatr Serv, August 1, 2007; 58(8): 1102 - 1107. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Finn Doing a Double Take: Accounting for Occasions in Service Performance Assessment Journal of Service Research, May 1, 2007; 9(4): 372 - 387. [Abstract] [PDF]
|
|
|
|
 |
|
 K. Fiscella, P. Franks, M. Srinivasan, R. L. Kravitz, and R. Epstein Ratings of Physician Communication by Real and Standardized Patients Ann. Fam. Med, March 1, 2007; 5(2): 151 - 158. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Danielsen, A. M. Garratt, O. A. Bjertnaes, and K. I. Pettersen Patient experiences in relation to respondent and health service delivery characteristics: A survey of 26,938 patients attending 62 hospitals throughout Norway Scand J Public Health, January 1, 2007; 35(1): 70 - 77. [Abstract] [PDF]
|
|
|
|
 |
|
 R. Baltussen and Y. Ye Quality of care of modern health services as perceived by users and non-users in Burkina Faso Int. J. Qual. Health Care, February 1, 2006; 18(1): 30 - 34. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A M Garratt, O A Bjaertnes, U Krogstad, and P Gulbrandsen The OutPatient Experiences Questionnaire (OPEQ): data quality, reliability, and validity in patients attending 52 Norwegian hospitals Qual. Saf. Health Care, December 1, 2005; 14(6): 433 - 437. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H Davies, N Khera, and J Stroobant Portfolios, appraisal, revalidation, and all that: a user's guide for consultants Arch. Dis. Child., February 1, 2005; 90(2): 165 - 170. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. I. Pettersen, M. Veenstra, B. Guldvog, and A. Kolstad The Patient Experiences Questionnaire: development, validity and reliability Int. J. Qual. Health Care, December 1, 2004; 16(6): 453 - 463. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H Davies and R Howells How to assess your specialist registrar Arch. Dis. Child., December 1, 2004; 89(12): 1089 - 1093. [Full Text] [PDF]
|
|
|
|
 |
|
 T. Heidegger, M. Nuebling, R. Germann, H. Borg, K. Fluckiger, T. Coi, and Y. Husemann Patient satisfaction with anesthesia care: information alone does not lead to improvement: [La satisfaction du patient face a l'anesthesie : l'information ne suffit pas a l'amelioration] Can J Anesth, October 1, 2004; 51(8): 801 - 805. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. S. WESTAWAY, P. RHEEDER, D. G. VAN ZYL, and J. R. SEAGER Interpersonal and organizational dimensions of patient satisfaction: the moderating effects of health status Int. J. Qual. Health Care, August 1, 2003; 15(4): 337 - 344. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. KELLEY, B. CLARK, V. BROWN, and J. SITZIA Good practice in the conduct and reporting of survey research Int. J. Qual. Health Care, May 1, 2003; 15(3): 261 - 266. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Wensing and G. Elwyn Improving the quality of health care: Methods for incorporating patients' views in health care BMJ, April 19, 2003; 326(7394): 877 - 879. [Full Text] [PDF]
|
|
|
|
|
|
V. BANKAUSKAITE and O. SAARELMA Why are people dissatisfied with medical care services in Lithuania? A qualitative study using responses to open-ended questions Int. J. Qual. Health Care, February 1, 2003; 15(1): 23 - 29. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Heidegger, Y. Husemann, M. Nuebling, D. Morf, T. Sieber, A. Huth, R. Germann, P. Innerhofer, A. Faserl, C. Schubert, et al. Patient satisfaction with anaesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria{dagger}{ddagger} Br. J. Anaesth., December 1, 2002; 89(6): 863 - 872. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. A. J. HENDRIKS, F. J. OORT, M. R. VRIELINK, and E. M. A. SMETS Reliability and validity of the Satisfaction with Hospital Care Questionnaire Int. J. Qual. Health Care, December 1, 2002; 14(6): 471 - 482. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M Wensing and G Elwyn Research on patients' views in the evaluation and improvement of quality of care Qual. Saf. Health Care, June 1, 2002; 11(2): 153 - 157. [Abstract] [Full Text] [PDF]
|
|