Objective. Efforts to improve pain assessment and treatment in critically ill patients are poorly studied and represent an opportunity to improve quality of care. We sought to improve pain assessment and treatment in patients in a surgical intensive care unit at an academic medical center.
Design. We performed a prospective study of pain assessment and treatment in two surgical intensive care units in 2001. We measured pain assessment as the percentage of 4-h intervals where the patient’s pain was measured using a visual analog scale. We measured pain treatment as the percentage of 4-h intervals where the patient’s pain score on the scale was ≤3. We then implemented four separate ‘plan–do–study–act’ cycles to improve pain assessment and treatment.
Main outcome measures. We evaluated the percentage of 4-h patient-nursing intervals that were scored numerically pre- and post-intervention. We evaluated the percentage of 4-h patient-nursing intervals where the patients had a pain score of ≤3 pre- and post-intervention. In addition, we monitored naloxone use as a measure of adverse events related to pain treatment.
Results. Our baseline assessment of pain was 42% and the baseline treatment was 59%. After 5 weeks, pain assessment improved to 71% and pain management improved to 97%.
Conclusion. Our interventions were associated with significant improvements in pain assessment and treatment without an increase in adverse events related to pain therapy. Our interventions were relatively simple and may be implemented broadly. Our interventions provide insights into the application of complexity theory in improvement efforts.
Pain control in hospitalized patients is important to patients, providers, and regulators. Indeed the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) has developed standards for the assessment and management of pain in accredited hospitals and other health care settings . Providers need to improve their performance with respect to pain management . Patients in intensive care units (ICUs) may be at particular risk for poor pain management, yet relatively little is known about pain assessment and control in these patients .
Pain in hospitalized patients can be viewed as a preventable adverse event. There are multiple methods to classify adverse events in pain management . Based on the available literature and a review of pain management in our ICUs, we classified these adverse events as failures in documentation and failures in management. We found that providers did not use a standard tool to measure pain and varied widely in their responses to patient-reported pain.
As part of an Institute for Health Care Improvement project ‘Quantum Leaps in Patient Safety’, one of the authors (P.P.) led a team to develop a comprehensive patient safety program. As part of this program, we sought to reduce patients’ pain in the ICU. To accomplish this, we used the change model developed by Nolan and Langley . This model proposes three questions, which we have paraphrased as follows: (i) ‘What is your goal?’; (ii) ‘How will you know when you reach your goal?’; and (iii) ‘What will you do to reach your goal?’. These are followed by tests to change using ‘plan–do–study–act’ (PDSA) cycles.
The goals of this project were to: (i) improve pain documentation such that for 85% of patient-nursing intervals (defined as a 4-h interval of patient care), pain would be measured using a modified visual analog scale (VAS); and (ii) improve pain management such that for 85% of patient-nursing intervals the pain score was ≤3 on a modified VAS scale.
The study design was a pre- and post-intervention study.
We studied patients in two surgical ICUs at the Johns Hopkins Hospital (Baltimore, MD). One ICU was a 15-bed, general surgery/trauma ICU that cares for vascular, transplant, and trauma patients, while the second was an 11-bed oncologic surgery ICU that cares for abdominal surgery patients.
We sought to improve pain documentation and then pain management. In our ICUs, pain is measured at least every 4 h. Therefore each 4-h interval (called a patient-nursing interval) was the denominator for our measures, and there were six data points for each 24-h period.
For improving pain documentation, we measured the percentage of patient-nursing intervals per week where pain was measured using the modified VAS scale (Figure 1). For patients who were unresponsive, a 10-point scale based on behavioral responses was used (Figure 2). This is the tool recommended in the hospital pain assessment protocol for adult patients unable to cooperate with nursing assessment. For improving pain management, the measure was the percentage of patient-nursing intervals per week, where pain score on the modified VAS was ≤3.
The behavioral pain assessment scale for patients unable to provide self-report of pain.
To be included in a patient-nursing interval (denominator), the patient has to be physically present in the ICU for at least 2h of the interval. To reduce the burden of data collection, we randomly sampled 10–15 patients per ICU per week to obtain these measures. These 10–15 patients were obtained by randomly selecting five patients per ICU on each of 2–3 days per week. We alternated each day of measurement by taking the first five patients who met the above criteria, starting with the lowest-numbered rooms in the ICU one day, then starting with the highest-numbered rooms the next. This selection has a random nature to it. We suspect there is no systematic pattern regarding which patients are assigned to which rooms. The rooms are all equally accessible to nursing staff. We expect that no bias is present regarding the quality of pain management according to room number. The pain scores were obtained from one of the investigators (M.E.) reviewing the medical record of these patients for the previous 24 h. If a patient had been admitted to the ICU for <24 h, we only used data from their ICU admission. The patients sampled represented ∼40% of our ICU patients during that time period.
We collected data for 2 weeks and then implemented our tests of change. We selected tests of change through structured interviews with ICU nursing and physician staff on barriers to documenting and managing pain. We attempted to develop multifaceted tests of change that were effective (evidence-based), efficient, and that could be broadly implemented .
Our project essentially created and implemented a feedback loop regarding pain. We designed interventions, carried them out, and then attempted to learn from and base new strategies on the results of our interventions.
Improving assessment of pain: PDSA cycles
We implemented several consecutive interventions to improve pain assessment (Table 1). These are the PDSA models put forth by Nolan and Langley . When employed, one establishes a plan for change, executes that plan, looks at the outcome of the intervention and determines its effectiveness, and then conceives a final plan through the synthesis of the information from this cycle.
Timeline showing dates of implementation of individual ‘plan–do–study–act’ (PDSA) cycles
Date of implementation
Beginning at week 1
Approximately 30 September 2001
Prior to week 3
Approximately 17 October 2001
Prior to week 3
Approximately 20 October 2001
Prior to week 4
Approximately 1 November 2001
Prior to week 5
Approximately 12 November 2001
PDSA 1: Educate staff regarding the importance of pain and how to measure pain using a modified VAS scale
Plan. After interviewing staff, we recognized that many staff failed to understand the importance of using a standard measure to document pain in patients. We hypothesized that by increasing staff awareness regarding the need for a standard measure for pain, we would improve compliance with pain assessment using a modified VAS scale . The scale was modified in that it possessed individual hashmarks at each point on the 10-point scale.
Do. One of the investigators (M.E.) provided in-service sessions for physician and nurse staff in the participating ICUs on the importance of standard measures to document pain and on how to measure pain using a modified VAS scale. These sessions were conducted in collaboration with Nursing Educators, who are the considered leaders in nursing education.
Study. In an effort to gauge the impact of our intervention, we measured the percentage of nurses leaving the in-service sessions who agreed that it is important to measure pain using the VAS scale. We found that >90% of the ∼80 nurses who attended our in-service sessions agreed that pain should be measured using a VAS or modified VAS scale.
Act. The in-service session regarding the importance of pain management is now a standard in-service in these ICUs.
PDSA 2: Provide modified VAS scales at patients’ bedsides
Plan. We identified that a barrier to improved pain documentation was the lack of consistent availability of tools, such as cards, to measure pain scores. We hypothesized that by mechanically attaching cards at the bedside we would improve pain documentation.
Do. We went to both ICUs and identified that it was not always possible to find a VAS card. We attached VAS cards to each patient’s bed in the surgical ICU.
Study. We audited the percentage of beds with VAS cards for 2 weeks after this intervention and found that 99% of beds in both ICUs had a VAS card. These cards also detail the behavioral scoring system for patients who are unable to cooperate voluntarily with the modified VAS scale.
Act. We created a standard policy to have VAS cards attached to the bedside.
PDSA 3: On rounds, have residents document and report patients’ pain scores for the last 24 h
Plan. We found that the physicians and nurses in the ICU did not clearly communicate regarding patients’ pain. While presenting patients during rounds, the residents would simply report what type of analgesia the patient was receiving [intravenous patient-controlled analgesia (PCA), epidural PCA, intermittent narcotics] without reporting the effectiveness of that therapy. Pain scores were rarely reported. We hypothesized that by requiring the residents to report the patients’ pain scores explicitly we would improve documentation and management of pain.
Do. We modified the form the residents use to present patient information during rounds to include a section for the patients’ pain scores over the last 24 h. We instructed the ICU attendings, fellows, residents, and nurses that this information was to be reported daily.
Study. When we audited this, we found that for 96% of patients over 3 weeks, the residents recorded the patient’s pain score in their notes.
Act. We have incorporated the patients pain scores into the standard form used to present patient information during rounds.
PDSA 4: Create expectation that a pain score >3 is a defect
Plan. We found that physicians and nurses did not have a clear idea of what pain score should warrant therapy. There was also a misconception that some patients report pain scores that may not correlate with physiological pain. We hypothesized that we would improve the management of pain by creating the expectation that a pain score >3/10 was a preventable adverse event that warrants therapy.
Do. We created this expectation with the combined effort of physicians and nurses involved in this study. The authors of this paper are intensive care physicians and formulated an educational plan in conjunction with the ICU Nurse Educator. We instructed our ICU physicians and nurses that they had to create an explicit and immediate plan to manage the pain of any patient with a pain score >3. We did not create explicit protocols or guidelines on how to manage pain. Most ICU patients (∼55%) are on epidural or intravenous patient-controlled analgesia (IV PCA) and standard orders exist for their pain management. In addition, most of the ICU fellows and the ICU residents were anesthesiologists who are knowledgeable regarding pain management. The problem seemed to be complacency with high pain scores rather than knowledge regarding how to manage pain.
Study. We evaluated physician and nurse acceptance that a pain score >3 represented a preventable adverse event.
Act. We now educate all fellows, residents, and nurses that a pain score >3 is a preventable adverse event that requires an immediate, explicit intervention to reduce the pain.
We also wanted to ensure that our improvement efforts did not negatively impact patients. We were concerned that our intervention to improve pain management might increase a patient’s risk of having a narcotic overdose. A narcotic overdose in our ICU is routinely treated with naloxone, a narcotic antagonist. To assess narcotic overdose, we monitored the use of naloxone in our ICU. We found that over a 3-month period before and during the study period, only one patient per month required treatment with naloxone for narcotic overdose. Therefore, no additional adverse effects in terms of naloxone usage were considered a function of our interventions.
During the first week of the study period, 71 of 171 (42%) nursing intervals were not measured on a standard 10-point modified VAS or behavioral scale, but included a verbal description of pain such as ‘mild’ or ‘horrible’. By the end of the 5-week study period, the number of patients whose pain was scored using a modified VAS had increased to >70% (Figure 3; Table 2).
Percentage of nursing intervals per week where patients’ pain scores were measured using a visual analog scale (VAS). Implementation of ‘plan–do–study–act’ (PDSA) cycles is shown temporally using arrows.
Five-week data showing nursing intervals, visual analog scale (VAS) scoring, and patients with VAS scores ≤3
% VAS ≤3
nNI, number of nursing intervals sampled; nVAS, number of nursing intervals where patients’ pain was scored with a modified VAS; VAS≤3, number of nursing intervals where patients’ pain was scored with a modified VAS score of ≤3; % VAS, percentage of nursing intervals where patients’ pain was scored with a modified VAS; % VAS ≤3, percentage of nursing intervals where patients’ pain was scored with a modified VAS score of ≤3.
In addition, it was found that only 59% of patients were recorded to have VAS scores for pain ≤3 in the initial stages. By the end of the study period, this number had increased to well over 90% (Figure 4).
Percentage of nursing intervals per week where patient’s pain score was ≤3 on the visual analog scale (VAS). Implementation of ‘plan–do–study–act’ (PDSA) cycles is shown temporally using arrows.
There is a general upward trend for both of the above outcomes with the implementation of each successive PDSA cycle.
Through this intervention, we were able to significantly improve pain assessment and treatment for patients in several intensive care units ICUs. The intervention was successful, probably because we targeted multiple points in the decision-making process . We are in the process of broadly applying this intervention throughout our hospital. We initially had two main goals: to improve pain assessment and to improve pain treatment. Nonetheless, improvements in pain documentation led to improvements in pain management. This may indicate that the improved pain assessment highlighted for providers the opportunities to improve pain treatment. The ‘feedback loop’ nature of our interventions is a concept that may be considered and implemented in the generalization of these results to other clinical venues.
It may seem surprising that pain treatment improved without introducing specific protocols to manage pain. Nonetheless, our results are consistent with the theory of complex adaptive systems described by Plsek and Greenhalgh , where improvement efforts that involve complex decisions may be more successful by applying some simple rules rather than creating complex algorithms. The model of Stacey  suggests a certainty-agreement diagram, with degree of agreement on one axis and degree of certainty on the other. ‘Simple’ issues have high degrees of both certainty and agreement. ‘Chaotic’ issues have low degrees of certainty and agreement. The remaining issues fall into the so-called ‘zone of complexity’.
In this intervention, our PDSA cycles can be viewed as simple rules: measure pain scores with a VAS scale, report patients’ pain scores during rounds, and do not accept pain scores >3. With these, the ICU staff were able to devise their own interventions to manage pain. Our hospital has standard order sets for epidural and IV PCA. We did not modify these for our intervention.
One concern with efforts to improve pain management is the risk of complications from overuse of pain medications. We evaluated this by monitoring the use of naloxone, a narcotic antagonist commonly used to treat narcotic overdose. We found that our improvements in pain management were not associated with increased naloxone use.
We learned several lessons from participating in this improvement effort. Firstly, we learned that we assessed patients’ pain poorly. Without a standard metric for pain, it is difficult to evaluate and improve performance. Secondly, we learned that the staff in the ICU often wanted to improve quality, in this case pain assessment, but were hindered by the system. Our simple, low-cost intervention of placing modified VAS scales at each bed significantly improved pain documentation. Thirdly, we learned that we do not always need explicit detailed guidelines to improve processes. Rather, simple guidelines that allow staff to modify therapy to the patient’s needs may be more effective than a complex, detailed guideline. A significant component of our intervention was a cultural change to define a pain score >3 as a defect.
In addition to the lessons learned, we also recognize several limitations to our study. Firstly, we only sampled 10–15 patients per unit per week to obtain information regarding pain assessment and pain treatment. Nonetheless, each of these patients provided up to six data points per day. This sample size was selected for both methodological and practical reasons. Sixty to 90 data points per week should provide a stable estimate of pain documentation and management in our population. We also had limited resources for data collection. One of the authors (M.E.) reviewed the patients’ medical records and recorded their pain scores. A second limitation is the potential lack of generalizablity of our findings. One of our PDSA cycles had the residents reporting pain scores of patients during rounds, but many ICUs do not have intensive care physicians making rounds on patients. It is unclear whether our intervention would have been successful in an ICU that lacked ICU physicians making rounds.
A third limitation is that we did not obtain quantitative data on the opinions of nurses prior to attending in-service sessions.
Patients experiencing pain can be viewed as a preventable adverse event. We implemented an improvement effort to improve both pain assessment and treatment. In the absence of detailed rules for pain management, we significantly improved both pain assessment and treatment without increasing adverse events related to pain management. The approach to improvement used in this intervention was informed by complexity theory.