International Journal for Quality in Health Care Advance Access originally published online on September 17, 2008
International Journal for Quality in Health Care 2008 20(6):412-420; doi:10.1093/intqhc/mzn041
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Content analysis of patient complaints
1 Department of Cardiology and Comprehensive Care, New York University College of Dentistry, New York, NY, USA
2 American Society of Law, Medicine and Ethics, Boston, MA, USA
3 Department of Critical Care Medicine, Foothills Medical Centre, University of Calgary, Calgary, AB, Canada
Address reprint requests to: Henry Thomas Stelfox, Department of Critical Care Medicine, Foothills Medical Centre, University of Calgary, 1403-29th St. NW, EG-23A, Calgary, AB, Canada T2N 2T9; E-mail: tom.stelfox{at}calgaryhealthregion.ca
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
Objective. To develop a standard taxonomy for inpatient complaints that could be adopted in a wide array of health service institutions.
Design. A taxonomy was developed by merging the coding schemes from eight prior studies of patient complaints, and then by revising the received coding scheme in light of the codes and clarifications that emerged from a content analysis of patient complaints.
Setting. Two Boston area hospitals.
Participants. Stratified random sample of 1216 complaints from patients in 2004.
Intervention(s). None.
Main outcome measure(s). Patient complaints codes, provider codes and inter-rater reliability.
Results. A taxonomy comprising 22 patient complaint codes and five provider codes was developed. Inter-rater agreement for complaint codes was good (median Kappa statistic 0.66, interquartile range 0.55–0.80). Four codes were each used in more than 10% of the patient complaints filed: unprofessional conduct (19%); poor provider–patient communication (17%); treatment and care of patient (16%); and, having to wait for care (11%). Of the coding for the profession of the person complained about, 47% of the patient complaints were about staff in general or did not specify a particular profession; 22% identified a physician or dentist; 12% nursing staff; 11% administrative or support staff and 8% allied clinical health professionals.
Conclusions. Standardized coding of patient complaint data may provide an opportunity for quality improvement, patient satisfaction and changes in patient care.
Keywords: patient complaint, patient satisfaction, quality of healthcare, malpractice, medical errors, quality assurance, healthcare
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
Patient complaints are increasingly recognized as a potentially valuable source of information. The Joint Commission and the federal Centers for Medicare and Medicaid Services mandate the collection and retention of these patient complaints, offering a treasury of valuable information [1, 2]. Researchers concerned about malpractice suits, most notably the Hickson and Pichert research group at Vanderbilt University, have studied patient complaints and found that a small number of physicians generate a disproportionate share of complaints [3–9]. They used patient complaints data to identify and intervene with outlier physicians to mediate risk management. However, a broader opportunity exists to utilize patient complaints as a source of information for understanding and improving systems of care and patient safety.
Patient complaints are unstructured and spontaneous information that the patient was motivated to give to the hospital. By filing complaints, patients give information to the hospital in their own language and on their own terms—the patients choose to report on what was important to them. That patients initiate the complaint process during their stay distinguishes the complaint process from patient satisfaction surveys distributed after discharge. Emergent in studies of patient complaints is a growing awareness of their potential value in understanding and improving systems of care. Each patient who complains offers information on patient satisfaction, a key quality measure [10]. When editorializing about the work of Hickson et al.' [8], Sage [11] suggested that health care organizations ‘elicit patients’ stories, capture information relevant to safety and feed that information back to the professionals who organize and deliver care'.
However, to create this feedback mechanism to ‘elicit patients’ stories' and ‘capture information relevant to safety’, a standardized system to collect, aggregate and analyze these patient complaints is needed. Previously, eight teams of researchers have analyzed patient complaint data: five studies were done in the USA [3, 4, 12–14], two in the UK [15, 16] and one in Australia [17]. The data ranged from depositions [12], questionnaires and interviews [3, 13] to patient-generated or staff-recorded written complaints [4, 14–17]. A review of the eight instruments identified significant overlap between the coding schemes, but it was unclear which instrument was superior. We therefore sought to develop a standard taxonomy that would consist of coding categories inherited from the prior eight studies of patient complaints, empirically test the synthetic taxonomy to determine which codes were used most frequently and which codes could be dismissed due to infrequent utilization and then to use the taxonomy to analyze patient complaints from two Boston area hospitals.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
Selection and review of patient complaint codes
A review of the peer-reviewed published literature identified eight coding schemes of patient complaints that utilized content analysis [3, 4, 12–17]. We built an initial taxonomy using the ‘received codes’ from these eight prior studies of patient complaints [3, 4, 12–17]. The original full taxonomy is summarized in Table 1 and includes the 84 codes that were received from the prior eight studies, plus the two codes that our research team added: ‘lack of accommodation for disability’ and ‘lack of safety—medical, clinical and public health’. The taxonomy included all codes found in the eight prior studies except those that: (i) described a problem that would not be found in, or under the purview of, either of our study hospitals (for example, ‘advised to go private’ or ‘charging for a letter’ [16]); or (ii) described social psychological stances or positioning arguments (for example, ‘anticipating hospital's justifications’ [15]).
|
In addition, we developed a coding scheme for the profession of the practitioner about whom the patient complained. We added this profession code because we wanted our results to have the potential to inform systems change (not just identification of errant physicians), giving us the ability to summarize complaints about any provider group that treated patients during their hospital experience. We generated a list of 34 profession codes (Table 1).
Patient complaints data
We randomly sampled 27% of the 4521 patient complaints submitted during 2004 to two Boston hospitals: Massachusetts General Hospital's (MGH) Office of Patient Advocacy, and Brigham and Women's Hospital's (BWH) Office of Patient and Family Relations. The sample was stratified by the number of complaints at each hospital and consisted of 1216 patient complaints: 449 from MGH (37%) and 764 from BWH (63%). The Massachusetts General Hospital (n = 942 licensed beds) and Brigham and Women's Hospital (n = 735 licensed beds) are teaching hospitals that serve as tertiary care referral centers for the city of Boston and each have over 40,000 inpatient admissions and over half a million outpatient visits per year. Patients or their family members initiated these complaints, either by phoning the Office (about 60%), meeting with an advocate in person (about 20%), or sending a letter, e-mail or fax (about 20%). Each complaint was received by a patient advocate, a hospital employee who entered the patient's narrative of events into the database. The hospital patient complaints database is a daily work log for the Office and a record of any activity that happens in the office. We excluded entries requesting Office services (e.g. a request for a notary public), entries that were beyond the purview of the Office (e.g. complaints about the cost of parking in a non-hospital lot) and compliments, and replaced these items with a newly sampled patient complaint. We excluded 45% of the complaints sampled from MGH, and 33% of the complaints sampled from BWH.
Coding patient complaints
Similar to the eight prior studies, we used content analysis, a technique in which researchers objectively and systematically record and count episodes described in written text to produce a quantitative description of the content of given text [18]. The first two authors coded the first 600 of the sampled complaints independently using the amalgamated taxonomy (Table 1). Coding disagreements were resolved through discussion and a consensus was established about final codes [19]. New codes were developed for complaints not captured by any of the existing codes inherited from the eight prior research studies, for example, a public health safety violation such as used needles not disposed of properly [20, 21]. As these new codes emerged, we wrote memos to record their justification, and introduced the new code to the other research team members to assure consensus regarding novelty before adding it to our taxonomy [10]. Box I is an example of how the content of a patient complaint was assigned three different complaint codes.
After we coded the first half of our sample and revised and condensed the taxonomy, we hired a new research assistant who was not involved in the development of the initial taxonomy to use the revised taxonomy to code the second half of the sample.
Analysis
Patient complaints were de-identified and imported into MAXQDA2 qualitative data analysis software (Verbi Software, Marburg, Germany) to facilitate data management and both qualitative and quantitative summaries. Face validity was assessed by a blinded reviewer (H.T.S.) who was presented with all of the passages from all of the patient complaints that had been coded with one particular code (n = 86 codes from the amalgamated coding scheme) by two coders (T.M. and A.N.) and was asked to determine what all of the passages had in common. If he offered a code identical to the code used by the first two authors, we considered the use of the code to describe these passages as valid. MAXQDA2 allowed us to retrieve all the segments of all patient complaints that were labeled with the same code. For example, we were able to create a file that contained any and all segments from the 600 patient complaints that were coded ‘unprofessional conduct’. We gave the senior author (H.T.S.) a word document that had the 115 passages that we coded ‘unprofessional conduct’ from the first 600 patient complaints we coded, however, we did not tell him what this particular set of passages had been coded as. We asked him to tell us what he thought the common code was, and he could chose one of the codes from the original taxonomy of 86 codes, or he could develop his own terminology. If his determination matched the code that the coders had actually used, we considered the code as having face validity. During this process, if he found text from a complaint that was not similar to the other passages in the document, we deemed it an error and removed or transferred it to a correct category.
| Box I A sample patient complaint with coding
|
Test-retest reliability was assessed for each of the four coders (three authors plus one research assistant) who coded a random sample of 45 patient complaints and then recoded those same patient complaints three months later. Inter-rater reliability was evaluated for the three coders with the best test-retest reliability scores by having each of these three coders code the same random sample of 45 patient complaints drawn from the 1216 patient complaints. Both test-retest and inter-judge reliabilities were assessed using Cohen Kappa reliability statistics [22].
The distribution of patient complaints was summarized using simple descriptive statistics. Statistical analyses were performed using Stata Version 9.2 (Stata Corp, College Station, Texas, USA). The Institutional Review Boards of the Massachusetts General Hospital and Brigham and Women's Hospital approved the study.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
Resultant taxonomy
After the face validity of the coding categories was checked and data were cleaned, we revised and reduced the original taxonomy. Codes with more than 5% of the complaint totals were allowed to stand alone, for example, ‘had to wait for care’ and ‘calls not returned’. Codes that received fewer than 5% of the complaint totals were folded into broader conceptual categories to ensure a parsimonious taxonomy. Consequently, an umbrella code absorbed all adopted components. For example, after reduction, the definition of ‘poor patient–provider communication’ contained all the elements of poor quality of information, incomplete information, and inadequate listening and response. Because of the legal and patient safety consequences of the codes in the ‘red flags’ suite (e.g. discrimination, assault), all codes in this category were kept regardless of the degree of utilization. The coding scheme was reduced from 86 complaint codes to 15 complaint codes plus seven ‘red flag’ codes for a total of 22 complaint codes, and from 34 profession codes to five groups of profession categories. Table 2 is the resultant revised taxonomy that we used to code the second 616 patient complaints.
|
The intra-rater reliabilities for coding patient complaints for the four coders were Kappa statistics of 0.82, 0.68, 0.54 and 0.32. Inter-rater reliability was evaluated for the three coders with the best test-retest reliability scores and was a global unweighted Kappa statistic of 0.49 for overall agreement on patient complaint coding. As would be expected, agreement was higher for the patient complaints that were assigned fewer codes. Patient complaints assigned a single code by each reviewer had high levels of agreement (Kappa statistic 0.88) while complaints assigned two codes (Kappa statistic 0.29) or three or more codes (Kappa statistic 0.09) had lower levels of agreement. Inter-rater reliabilities were higher for individual complaint codes (median Kappa statistic 0.66, interquartile range 0.55–0.80).
Patient complaints
Of the 1216 patient complaints analyzed, each averaged 1.5 complaint codes, that is, 1.5 issues that the patient identified to complain about, and that we coded. Of the 1216 patient complaints analyzed, the number of complaints coded ranged from 1 to 9 per patient complaint, that is, in some of the patient complaints filed the patient had only one issue to complain about, while in other patient complaints the patient complained about up to nine issues. In Box I the patient complained about three different issues.
The results are presented in Table 3. The 22 complaint codes in Table 3 are identical to the 22 presented in Table 2 but the order is different—in Table 3 the codes are presented in descending order determined by how often they were used (column 1). The MAXQDA2 software allowed us to merge codes into broader categories as well as recode, so we recoded the original 600 patient complaints and added them to the 616 patient complaints coded using the revised taxonomy in the second phase. Therefore, Table 3 represents the results of using the taxonomy presented in Table 2 on a total of 1216 patient complaints.
|
As you can see in Table 3, of the 22 codes in the final taxonomy, only four codes were assigned to more than 10% of the 1216 patient complaints analyzed: ‘unprofessional conduct’ (19%), ‘poor patient–provider communication’ (17%), ‘treatment and care of patient’ (16%) and ‘had to wait for care’ (11%).
The 1216 patient complaints identified 1806 providers. Of the 1806 professionals complained about, 47% of the patient complaints were about staff in general or did not specify a particular profession, 22% identified a physician or dentist, 12% nursing staff, 11% administrative or support staff and 8% allied clinical health professionals (Table 3).
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
A taxonomy comprising 22 patient complaint codes and five provider codes was developed. Inter-rater agreement for complaints and complaint codes was good. Four codes were each used in more than 10% of the patient complaints filed: unprofessional conduct (19%); poor provider–patient communication (17%); treatment and care of patient (16%) and having to wait for care (11%). Of the coding for the profession of the person complained about, 47% of the patient complaints were about staff in general or did not specify a particular profession; 22% identified a physician or dentist; 12% nursing staff; 11% administrative or support staff and 8% allied clinical health professionals.
In this study we examined the content of complaints from hospital patients, and we were able to determine about what and whom patients complained. When comparing the results from this study with the eight prior studies we used to build our taxonomy, there were continuities in that a majority of patients in all studies complained about poor patient-provider communication, poor patient treatment and care and having to wait for care. However, our study was unique in that the most common complaint was unprofessional conduct. This could be attributable to the eight prior studies labeling both unprofessional conduct and communication failures as communication problems, while we distinguished between a provider behaving unprofessionally vs. a communication failure between a provider and a patient (Box II).
Box II Coding samples
|
Another unique feature of this study is that we noted about whom the complaint was made. Prior studies that used patient complaints as data [3, 4, 12–17, 23] were interested in complaints about physicians, and offered implications of their findings that targeted physicians. In our study physicians and dentists received only 20% of the ‘unprofessional conduct’ complaints, 26% of the ‘poor communication’ complaints, 19% of the ‘treatment and care’ complaints and 11% of the ‘had to wait for care’ complaints. The finding that 78% of the complaints were not about physicians has implications for interventions, that is, programs that aim to improve patient care and reduce patient dissatisfaction should be directed at all staff, including administrative staff.
While examining this primary data, we realized that patients do not necessarily compartmentalize their hospital experience into separate clinical and administrative service components. Qualitative analysis suggested that the incidents that trigger a complaint are of a magnitude that surpassed the patient's threshold of endurance. Complaints would frequently read as cascades of problematic incidents that were tolerated by the patient until the ‘straw that broke the camel's back’. Therefore, if administrative service staff or allied health professionals are treating a patient poorly during their hospital stay, should the physician make a clinical error, that unfortunate instance could be interpreted more harshly in a context of cumulative hurt. Thus, if one considers the whole of a patient's hospital experience as a potential contributor to a possible lawsuit should an iatrogenic injury occur, understanding patient sources of frustration becomes even more desirable. Hence our recommendation that remedial interventions be directed at all staff rather than targeting physicians.
There is also a pragmatic component to expanding the focus of interventions beyond physicians. Taking the case of ‘unprofessional conduct’, the most common complaint that we found, administrative and support staff were the group most often cited as having ‘unprofessional conduct’ (26%). One could argue that the return on investment of in-service training for administrative and support staff to encourage them to conduct themselves in a more professional manner would have a greater impact on reducing the frequency of this complaint, because they are the group most cited for this offense. In addition, the cost/benefit ratio favors targeting administration and support staff because pulling them away from their duties for courtesy training costs the hospital much less than pulling physicians and nurses away from their clinical care duties.
Nettleton and Harding [16] noted that recording a patient complaint might be of limited value if it is not matched by adequate mechanisms for dealing with what has been complained about. A standard instrument for collection that allows rapid analysis of patient complaints opens the possibility of transforming the collection of patient complaints from a compliance activity into an opportunity for quality improvement, patient satisfaction and changes in patient care. Greater insight into the typology of patient complaints will allow for the prevention of those acts or omissions that are likely to lead to patient complaints in the first instance. Aggregated complaint typologies can be compared to determine if certain medical center units are experiencing a disproportionate share of a particular type of medical complaint. This taxonomy optimizes the use of complaints as a quality improvement tool [6] at the level of systems change. Standardized coding of patient complaint data will allow for reliable documentation of patient complaints so that comparisons can be made between units and hospitals, pointing to areas of potential rapid response to improve safety.
Our results need to be interpreted within the context of the study's limitations. First, the database is the hospital employee's rendering of what the worker comprehended and documented when they met with the patient. As such, this is secondary data and we cannot determine the degree to which the worker accurately captured the concerns of patients and their family. Second, our results are based on complaints recorded from patients at two tertiary academic hospitals and their associated outpatient clinics. Patients who are treated in other types of hospitals or settings may have a different set of experiences; therefore, the taxonomy needs to be evaluated in other settings. Third, although our taxonomy demonstrated good intra-rater and inter-rater reliability for three of the four reviewers participating in the study, its reliability needs to be evaluated further. Fourth, our taxonomy does not classify patient complaints by service unit, as these were not systematically recorded in the study databases. However, identifying the service unit with patient complaints as suggested by Chang and colleagues for reporting near misses and adverse events to The Joint Commission would enable the data to point to problems located in a particular unit at a particular time [24].
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
The patient complaints coding taxonomy developed represents a parsimonious tool that has been evaluated for face validity and reliability. The taxonomy combines the best elements of the eight previously published coding schemes while discarding those codes rarely employed. A standard taxonomy for collection that allows rapid analysis of patient complaints opens the possibility of transforming the collection of patient complaints into an opportunity for quality improvement, patient satisfaction and changes in patient care. Greater insight into the typology of patient complaints may allow for the prevention of those acts or omissions that are likely to lead to patient complaints in the first instance as well as opportunities for rapid response to improve safety.
|
Funding |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
This research project was funded by CRICO/Risk Management Foundation Research Grant Program, Cambridge, MA, USA.
|
Acknowledgments |
|---|
The authors wish to thank Sally Millar, RN (MGH) and Kathleen Gordon, RN (BWH) for allowing and facilitating data acquisition; Lisa de Saxe Zerden, MSW, and Alice Sodroski for research assistance; Farah Khandwala, MSc, for statistical consultation; and Benjamin Moulton, JD, MPH, of the American Society of Law, Medicine and Ethics for providing institutional support during the project. We presented an earlier version of this paper at the annual meeting of the American Public Health Association in November 2006.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionFundingReferences |
|---|
- Office of Quality Monitoring. The Joint Commission. Facts about The Joint Commission's complaint process. http://www.jointcommission.org/GeneralPublic/Complaint/oqm.htm (21 July 2008, date last accessed).
- Centers for Medicare and Medicaid Services. U.S. Department of Health and Human Services. Beneficiary complaint response program. http://www.cms.hhs.gov/BeneComplaintRespProg/ (21 July 2008, date last accessed).
- Hickson GB, Clayton EW, Entman SS, et al. Obstetricians' prior malpractice experience and patients' satisfaction with care. JAMA (1994) 272:1583–7.
[Abstract/Free Full Text] - Hickson GB, Pichert JW, Federspiel CF, et al. Development of an early identification and response model of malpractice prevention. Law Contemp Probl (1997) 60:7–29.[CrossRef][Web of Science]
- Hickson GB, Clayton EW, Githens PB, et al. Factors that prompted families to file medical malpractice claims following perinatal injuries. JAMA (1992) 267:1359–63.
[Abstract/Free Full Text] - Pichert JW, Miller CS, Hollo AH, et al. What health professionals can do to identify and resolve patient dissatisfaction. Jt Comm J Qual Improv (1998) 24:303–12.[Medline]
- Pichert JW, Federspiel CF, Hickson GB, et al. Identifying medical center units with disproportionate shares of patient complaints. Jt Comm J Qual Improv (1999) 25:288–99.[Medline]
- Hickson GB, Federspiel CF, Pichert JW, et al. Patient complaints and malpractice risk. JAMA (2002) 287:2951–7.
[Abstract/Free Full Text] - Hickson GB, Federspiel CF, Blackford J, et al. Patient complaints and malpractice risk in a regional healthcare center. South Med J (2007) 100:791–6.[Web of Science][Medline]
- Kravitz RL, Callahan EJ, Paterniti D, et al. Prevalence and sources of patients' unmet expectations for care. Ann Intern Med (1996) 125:730–7.
[Abstract/Free Full Text] - Sage WM. Putting the patient in patient safety: linking patient complaints and malpractice risk. JAMA (2002) 287:3003–5.
[Free Full Text] - Beckman HB, Markakis KM, Suchman AL, et al. The doctor-patient relationship and malpractice. Lessons from plaintiff depositions. Arch Intern Med (1994) 154:1365–70.
[Abstract/Free Full Text] - Garbutt J, Bose D, McCawley BA, et al. Soliciting patient complaints to improve performance. Jt Comm J Qual Saf (2003) 29:103–12.[Medline]
- Wofford MM, Wofford JL, Bothra J, et al. Patient complaints about physician behaviors: a qualitative study. Acad Med (2004) 79:134–8.[CrossRef][Web of Science][Medline]
- Lloyd-Bostock S, Mulcahy L. The social psychology of making and responding to hospital complaints: an account model of complaint processes. Law Policy (1994) 16:185–208.[CrossRef]
- Nettleton S, Harding G. Protesting patients: a study of complaints submitted to a Family Health Service Authority. Sociol Health Illn (1994) 16:38–61.[CrossRef][Web of Science]
- Anderson K, Allan D, Finucane P. A 30-month study of patient complaints at a major Australian hospital. J Qual Clin Pract (2001) 21:109–11.[CrossRef][Medline]
- Neuman WL. Social Research Methods: Qualitative and Quantitative Approaches (1997) Boston: Allyn and Bacon.
- Clair JM, Clark C, Hinote BP, et al. Developing, integrating, and perpetuating new ways of applying sociology to health, medicine, policy, and everyday life. Soc Sci Med (2007) 64:248–58.[CrossRef][Web of Science][Medline]
- Glaser BG, Strauss AL. The Discovery of Grounded Theory; Strategies for Qualitative Research (1967) Chicago: Aldine Pub Co.
- Strauss AL. Qualitative analysis for social scientists (1987) New York: Cambridge University Press.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics (1977) 33:159–74.[CrossRef][Web of Science][Medline]
- Stelfox HT, Gandhi TK, Orav EJ, et al. The relation of patient satisfaction with complaints against physicians and malpractice lawsuits. Am J Med (2005) 118:1126–33.[CrossRef][Web of Science][Medline]
- Chang A, Schyve PM, Croteau RJ, et al. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. Int J Qual Health Care (2005) 17:95–105.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M Wessel, G Helgesson, and N Lynoe Experiencing bad treatment: qualitative study of patient complaints concerning their treatment by public health-care practitioners in the County of Stockholm Clin Ethics, December 1, 2009; 4(4): 195 - 201. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||