OUP user menu

Using the patient safety indicators to detect potential safety events among US veterans with psychotic disorders: clinical and research implications

Eric G. Smith, Shibei Zhao, Amy K. Rosen
DOI: http://dx.doi.org/10.1093/intqhc/mzs026 321-329 First published online: 20 June 2012


Objective Patients with psychotic disorders often experience poorer health outcomes, but whether they experience increased risks of medical errors/patient safety events is less clear. A single state-level US study found acute care inpatients with schizophrenia were at higher risk of incurring some of the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSIs). We examined a nationwide sample of US Veteran's Health Administration (VHA) acute care inpatients to determine the rates observed among patients with psychotic disorders for a larger set of PSIs.

Design Retrospective cohort analysis using administrative data.

Participants and Setting Veterans with (n = 50 328) and without (n = 1 812 897) psychotic disorders (schizophrenia/schizoaffective disorder and other psychoses) admitted for acute care to US VHA hospitals during fiscal years 2003–06.

Methods and Main Outcome Measures PSI rates were calculated using AHRQ's PSI software.

Results Patients with psychotic disorders had significantly higher rates of postoperative respiratory failure, postoperative wound dehiscence and decubitus ulcer than those without psychotic disorders, although postoperative respiratory failures rates were not significantly higher among patients specifically diagnosed with schizophrenia. Patients with psychotic disorders had significantly lower rates of accidental puncture/laceration, foreign body left in during procedure and failure to rescue. However, the odds of failure to rescue were not significantly lower among surgical patients (the current focus of this PSI).

Conclusions Acute care inpatient veterans with psychotic disorders experienced higher rates of several PSIs, but lower rates of others. Whether lower rates of certain PSIs reflect better or worse care for this population is uncertain.

  • hospital care
  • quality indicators
  • patient outcomes (health status, quality of life, mortality)
  • safety indicators
  • pressure ulcers
  • appropriateness, under-use and over-use
  • mental health disorders
  • psychosis
  • respiratory failure


Psychotic disorders are common among hospitalized patients of the US Veteran's Health Administration (VHA), being present during almost 3% of VHA hospitalizations [1]. Thus, the VHA provides a unique opportunity to investigate risks incurred by patients with psychotic disorders during non-psychiatric (acute care) hospitalizations. Substantial literature supports the association between psychotic disorders or serious mental illness and poorer health outcomes and quality of care [117]. However, very little is known regarding the specific question of whether patients with psychotic disorders experience increased risks of the important health outcomes [18] of inpatient safety events. The existing literature is restricted to one small, localized study (limited to one US state) [19] that analyzed only a fraction of the safety endpoints targeted by the US Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSIs). We used the AHRQ PSIs to conduct our own investigation of inpatient safety among a large, nationwide sample of VHA patients. Our goal was to gain a fuller appreciation of the types of inpatient safety events for which patients with psychotic disorders may be at risk, both in general and specifically within the VHA.

PSIs use administrative data to screen for the occurrence of potential inpatient safety events based on discharge codes; they apply a highly specified methodology to define the population at risk (the denominator) and the potential safety event (the numerator). PSIs screen for a variety of adverse and potentially preventable outcomes such as decubitus ulcers, postoperative respiratory failures, particular infections and deaths, accidental injuries and foreign objects left inside patients after procedures. Given the importance of patient safety, the use of the PSIs as performance measures (8 of the 16 PSIs have been endorsed as quality measures by the US National Quality Forum [20]), quality improvement targets [21, 22] or more general indices of care quality [23] has been discussed and continues to be debated [2428].

We sought to apply the PSIs to assess rates of potential patient safety events among VHA medical/surgical inpatients with psychotic disorders. Given the large, nationwide scale of our study, we expected to be able to assess rates for additional hospital-level PSIs not examined in the only previous study of patients with psychotic disorders [19]. Because of the wide variety of PSIs and the many factors that may influence their occurrence and detection, we expected the risks of PSIs for patients with psychotic disorders to vary by PSI. However, given the existing literature which suggests generally poorer quality of care or worse health outcomes for patients with psychosis [117, 19], our primary hypothesis was that patients with psychotic disorders would be at increased risk for PSIs.


Sample and source of data

We examined all patients admitted to US VHA acute care bedsections for medical or surgical reasons [based on diagnosis-related groups (DRGs)] from 1 October 2002 to 30 September 2006. Data were obtained from the VA Patient Treatment File, an administrative database that contains records on all patients discharged from VHA acute and non-acute care facilities [29]. This data source provided us with information on demographics, diagnoses (one principal and up to 39 secondary ICD-9-CM diagnosis codes), all surgical and some medical procedures performed, length of stay and other information (e.g. dates of admission/discharge and discharge status) for each hospitalization. Data were aggregated across fiscal years, using patients' scrambled social security numbers (full details provided elsewhere [30, 31]).

Patients with ‘psychotic disorders’ were defined as patients having a primary or secondary discharge diagnosis that included ICD-9-CM codes for schizophrenia (295.1–295.6, 295.8, 295.9), schizoaffective disorder (295.7), psychosis NOS (unspecified psychosis) (298.9) or other psychotic disorders (297.x and 298.x, excluding 298.9).

Measures and analyses

We used the AHRQ PSI software, version 3.1a [3234], to facilitate comparison with a previous report examining patients with schizophrenia [19] as well as the prior study of patients with any mental illness [35]. The software calculates both observed and risk-adjusted rates of 16 non-obstetric hospital-level PSIs, from which we excluded: postoperative hip fracture (psychosis is an exclusion) [32]; transfusion reaction (no events among patients with psychotic disorders) and death in low-mortality DRGs (‘Psychoses’ is included in the ‘low-mortality DRGs’) [32]. In addition, the PSI ‘failure to rescue’, an indicator originally intended to examine a fraction of in-hospital deaths deemed particularly preventable, has been subsequently redefined to apply only to surgical patients and renamed ‘Death rate among surgical inpatients with serious treatable complications’. We examined both the original and newer definitions in this analysis.

The AHRQ software performs risk adjustment by applying coefficients previously derived by AHRQ for age, sex, an age×sex interaction, modified DRG and a series of comorbid illness categories that are drawn from the Elixhauser comorbidity measure [36]. Information concerning these demographic and diagnostic factors was obtained from the VHA patient treatment file. The comorbid illness categories are defined by discharge secondary diagnosis codes; for our primary analyses, we used 26 categories (listed in Table 1) excluding the category ‘Psychoses’, since this comorbid illness was present by definition in the sample with psychotic disorders and not present among individuals without psychotic disorders.

View this table:
Table 1

Patient characteristics of hospitalizations (medical and surgical) for patients with and without a diagnosis of psychotic disorder, 2003–06

In the few instances in which VHA databases do not currently contain all the data elements required by the PSI software, we calculated these variables using algorithms developed in previous work [30, 31]. For example, VHA records do not distinguish ‘elective’ and ‘non-elective admissions’, so non-elective admissions were considered as all admissions for urgent or emergent DRGs plus any admissions occurring between 5 p.m. and 5 a.m. or on weekends [31].

Secondary analyses

Since the AHRQ risk adjustment coefficients were derived from a data set exclusively consisting of non-VHA hospitals, we also conducted a secondary analysis using logistic regression to generate and apply VHA-specific coefficients for age, gender and comorbid illnesses. This analysis sought to determine whether or not a pattern of PSI findings (i.e. higher or lower odds of PSIs) similar to the main analysis would be observed. Because of the low incidence of some comorbid illnesses and/or PSIs, our logistic regression models did not converge for every PSI for all 26 comorbid illnesses; we therefore included only those illnesses for which the models converged for all PSIs (the first 16 illnesses listed in Table 1).

To facilitate more precise comparison between our findings and those from a non-VHA study of patients with schizophrenia [19], we also conducted a secondary analysis restricting our sample to only individuals with schizophrenia (ICD-9 CM: 295.0–295.6, 295.8, 295.9). This restriction also highlights a chronic psychotic disorder particularly likely to precede hospitalization that has been observed to be more reliably diagnosed than some other psychotic disorders [37].


Compared with patients without psychotic disorders, VHA patients with psychotic disorders were younger, more likely to be female and had lower rates of 13 out of 23 medical comorbid illnesses, including complicated hypertension, diabetes, peripheral vascular disease, congestive heart failure, valvular heart diseases, tumors and metastases (Table 1). However, patients with psychotic disorders had higher rates of 8 of 23 medical comorbidities, including lung disease, iron-deficiency anemia, neurological disease, hypothyroidism, liver disease and both weight loss and obesity. Patients with psychotic disorders also had higher rates of the three psychiatric comorbidities: depression, alcohol abuse and substance abuse. Patients with psychotic disorders were less likely to be admitted for surgery or electively, and had a longer average length of stay but a lower in-hospital mortality rate. Patients with psychotic disorders were most frequently diagnosed with schizophrenia (63%), followed by schizoaffective disorder (19%), psychosis NOS (16%) and other psychotic disorders (2%).

Observed and risk-adjusted PSI rates are shown in Table 2. Risk adjustment resulted in a sizable decrease in the rate of several postoperative complications (postoperative physiologic/metabolic derangement, respiratory failure and wound dehiscence), and a substantial increase in the rate of accidental puncture/laceration, but did not substantively change the direction of any association. After risk adjustment, patients with psychotic disorders were still observed to have significantly higher rates than those without psychotic disorders of the PSIs postoperative respiratory failure, postoperative wound dehiscence and decubitus ulcer (as indicated by non-overlapping 95% confidence intervals). However, significantly lower rates were detected for the PSIs accidental puncture/laceration, foreign body left in during procedure and failure to rescue.

View this table:
Table 2

Observed and AHRQ risk-adjusted rates of PSIs for patients with and without psychotic disorders

Table 3 reports our secondary analysis using VHA-specific coefficients for age, sex and comorbid illness. With the exception of select infections due to medical care, the odds ratios (ORs) were generally comparable with the rate ratios reported in Table 2 (i.e. both analyses indicated the same direction of risk—higher or lower—for inpatients with psychosis in the VHA). This analysis also allowed us to assess the effect of redefining the population of interest for the ‘failure to rescue’ to surgical patients only (matching the change AHRQ has made to this indicator). When we restricted our analysis to only surgical patients, the OR for patients with psychotic disorders became less pronounced and non-significant [OR 0.83 (0.45–1.54), P < 0.54].

View this table:
Table 3

OR of PSIs, obtained using VHA-derived covariate coefficientsa

We also examined PSI rates among patients with schizophrenia specifically (Table 4). Of the six PSIs observed to be significantly different among patients with and without psychotic disorders, only one of these one PSIs (postoperative respiratory failure) had a rate that became non-significantly different when the analysis was restricted to patients with schizophrenia. While the rate of this PSI decreased in patients with schizophrenia compared with all types of psychosis, for both these patient groups, the rates of postoperative respiratory failure were either statistically higher (all psychosis) or numerically higher (schizophrenia) than for patients without psychosis.

View this table:
Table 4

Risk-adjusted rates for patients with schizophrenia vs all psychotic disorders or without psychotic disorders


We examined whether veterans with psychotic disorders were at increased risk of potential inpatient safety events as measured by the AHRQ PSIs during non-psychiatric, acute care VHA hospitalizations. We observed that veterans with psychotic disorders were significantly more likely to experience three PSIs (postoperative respiratory failure, postoperative wound dehiscence and decubitus ulcer) than those without psychotic disorders. Similarly, postoperative respiratory failure and decubitus ulcer PSI findings have been reported previously in the limited literature involving other related but distinct patient populations. Higher odds of the postoperative respiratory failure PSI were observed in one previous study of non-VHA patients with schizophrenia from Maryland [19], although this study did not observe higher odds of the decubitus ulcer PSI. While no other study exists that focuses on patients with psychosis, a small study of non-VHA patients with any diagnosed mental illness who had undergone coronary artery bypass surgery did observe higher odds of the decubitus ulcer PSI [35].

Our finding that VHA patients with psychotic disorders have increased risks for the postoperative respiratory failure PSI may generalize to other health systems. The rates of postoperative respiratory failure have been observed to be statistically higher ([19] and this study) or numerically higher (this study, for patients specifically with schizophrenia) for patients with psychosis compared with those without psychosis in every instance in which this comparison has been made. The value of considering postoperative respiratory failure as a potential important safety risk for patients with psychosis is underscored by the fact that it is one of the most common of all PSIs, with a rate more than 10 times higher than some other PSIs.

Potential contributing causes to respiratory failure among patients with psychosis after surgery might include: (i) restrictions on ambulation; (ii) poor ability to comply with incentive spirometry or other pulmonary toileting; (iii) drug interactions between anesthetics or other medications and the patient's psychotropic medications [19]; (iv) use of benzodiazepines or antipsychotics on an ‘as-needed’ basis to address agitation, aggression or insomnia [19]; and (v) higher levels of cigarette smoking [1]. If justified through further research, possible interventions might include protocols governing adjustments to psychiatric medications during/after surgery and training to reinforce the need for frequent ambulation or pulmonary toileting among patients with psychosis.

We also observed higher rates of another common PSI, decubitus ulcer, in both patients with psychotic disorders and those specifically with schizophrenia. Some of the same potential factors contributing to respiratory failure might also be involved in increasing risk for decubitus ulcers (e.g. restrictions on, or reduced encouragement of, ambulation, use of sedating medications leading to increased time in bed and cigarette smoking). We note that higher decubitus ulcer PSI rates have also been observed in another VHA subpopulation that is at risk for lower quality care (i.e. African-Americans [38]), and that our observation of higher decubitus ulcer PSI rates among the subpopulation of patients with schizophrenia is in contrast to non-VHA findings [19]. Thus, it is possible that the decubitus ulcer PSI may be functioning to identify a meaningful patient safety difference (at least for vulnerable patient subpopulations) across hospital systems. The possibility that the PSIs might function to facilitate comparisons of health system performance internationally has been discussed by others, although the PSIs were found to be problematic for some aspects of this use [28]. More work would be needed to establish whether this PSI is detecting differences in safety between VHA and non-VHA hospitals, given that factors as simple as differences in decubitus ulcer surveillance, rather than incidence, across hospital systems could also explain this finding.

Regarding postoperative wound dehisence, this PSI has never been evaluated in patients with psychosis or mental illness previously, thus it is difficult to estimate its generalizability. However, mechanisms can be readily envisioned whereby patients with psychosis might be less able to provide quality wound self-care compared with other patients.

Contrary to our expectations, we also observed statistically significant lower rates among patients with psychotic disorders for two PSIs (accidental puncture/laceration and foreign body left in during procedure). Significantly lower rates were also observed for the failure to rescue PSI, but we are not emphasizing or discussing this finding further because this finding did not persist in our secondary analysis that applied the current denominator (surgical patients only) for this PSI (Table 3). Interestingly, lower accidental puncture/laceration rates have also been reported for another VHA patient group at risk for poorer health care (African-Americans) [38]. Given the low frequency of these PSIs, it is possible that data limitations with respect to administrative data in general, or VHA data specifically, impair the ability of these PSIs to accurately characterize these events. However, it is also possible that these PSIs may be particularly sensitive to differences in patient severity or provider behavior not captured in the AHRQ risk adjustment. Regarding patient severity, accidental puncture/laceration had the biggest proportional change in rate ratio after risk adjustment of any PSI, so residual differences in patient severity are certainly plausible. In addition, foreign body left in during procedure is the only PSI examined in this analysis whose rates receive no risk adjustment using the AHRQ methodology. Regarding provider behavior, these ‘procedure-based’ PSIs are not standardized by the number of procedures performed. Thus, if a patient group receives relatively fewer procedures from providers, lower ‘procedure-based’ PSI rates are possible (even if safety risks per procedure are greater). A potentially important question exists concerning whether, for these select PSIs at least in certain subpopulations, lower, rather than higher, PSI rates could indicate worse quality of care (depending on the medical value of any foregone procedures).

A primary limitation of our study is the uncertainty concerning how accurately the PSIs capture genuine patient safety events. Because PSIs are designed to use administrative data to highlight potential safety events that may require further evaluation, rates may be biased by misclassification if some of these administrative-data defined events do not reflect actual safety events [39]. The degree of this misclassification can be assessed only using a chart review. Validation of the PSIs is currently ongoing [24, 25, 27, 3942]. Even if PSIs typically succeed in capturing genuine patient safety events (i.e. exhibit specificity), variations in hospital coding practices for different patient groups could influence our results [30]. Other limitations include the fact that (as discussed in the Methods section) VHA administrative data provide most, but not all, of the data required for PSI calculations. As we discuss above, residual confounding may also remain due to patient-level differences in disease severity or provider-level differences in the intensity of care. However, if residual confounding is affecting our results, this implies that the existing AHRQ PSI methodology may not be sufficient to remove such confounding.

An important strength of our study is its large size, consisting of multiple years of data from the entire nationwide VHA hospital network, which provided us with a sample 25 times larger than the only previous study focusing on patients with a psychotic condition [19]. This size allowed us to analyze a greater number of PSIs than previously examined. For instance, we were able to uncover potential issues in the interpretation of procedure-based PSIs not identified previously. In addition, this size aided our efforts to derive our own VHA-based coefficients for our secondary analysis. This secondary analysis demonstrates that our findings changed little if separate coefficients from within the VHA system were generated for the risk adjustment factors employed in the PSIs (age, sex and comorbid illnesses). This suggests the applicability of AHRQ PSI coefficients across at least two health systems (US non-VHA and VHA hospitals).

In conclusion, our findings and those previously published [19, 35] suggest that acute care inpatients with psychotic disorders and/or mental illness may be at increased risk for several AHRQ PSIs, including postoperative respiratory failure and decubitus ulcer. A number of plausible mechanisms can be envisioned whereby patients with psychotic disorders might be at higher risk for these potential safety events. Given, however, that the relationship between PSI rates and the true occurrence of patient safety events has not been fully determined, and that interpretation of PSIs can be complex, further investigation of these findings within the VHA or in other health systems would be beneficial.


The authors want to thank Ann M. Borzecki, MD, MPH, and Kathleen Carey, PhD for their helpful comments on this manuscript.


This work was supported by a grant (IIR-02-44-1, from the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service), to A.M.R. E.G.S. is supported by a Veterans Health Administration Health Services Research and Development Service Career Development Award.


View Abstract