International Journal for Quality in Health Care Advance Access originally published online on June 1, 2007
International Journal for Quality in Health Care 2007 19(4):187-194; doi:10.1093/intqhc/mzm015
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Monitoring patients using control charts: a systematic review
1 Department of Public Health and Epidemiology, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
2 Queen Elizabeth Hospital, Edgbaston, Birmingham B15 2TH, UK
3 Division of Medical Sciences, University of Birmingham, Edgbaston, Birmingham, UK
Address reprint requests to: Mohammed A. Mohammed, Tel: 0121 414 7187; Fax: 0121 414 7878; E-mail: m.a.mohammed{at}bham.ac.uk
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionReferences |
|---|
Objectives. To systematically review the uses control charts to monitor clinical variables in individual patients.
Data sources. Systematic searches of MEDLINE, CINAHL, Embase and five other databases yielded 74 studies, of which seven met our inclusion criteria of using control charts to monitor clinical variables for disease at an individual patient level.
Review methods. Included articles were reviewed independently by two reviewers. Data were extracted on study design, clinical condition or disease being monitored, clinical variable or marker, measurement method, outcome measure and any changes in clinical indicator identified in the articles.
Results. Control charts were applied to four conditions—hypertension, asthma, renal function post-transplant and diabetes. Studies fell into two categories. Three studies sought to determine the ‘performance’ of control charts in comparison with existing ‘gold standard methods’ in terms of sensitivity and specificity based on moderate sample sizes (n = 35–45). This category of studies found control charts to be simple, low-cost, effective tools with good sensitivity and specificity characteristics and concluded in favour of control charts. The other four studies were individual patient case-studies in which the use of control charts to monitor clinical variables was associated with a positive impact on patient and carer experience albeit anecdotally and with varying degrees of attention.
Conclusions. Control charts appear to have a promising but largely under-researched role in monitoring clinical variables in individual patients. Furthermore, rigorous evaluation of control charts is required.
Keywords: chronic diseases, quality of care, quality monitoring, random variation, statistical process control, variation
Understanding, monitoring and controlling variation in clinical variables is an integral part of clinical practice [1]. Changes in clinical variables, such as blood glucose levels or blood pressure measurements may be due to changes in the patients' underlying condition or biological processes, measurement error, or random variation. Monitoring systems need to be able to detect material changes in the clinical variable (i.e. detect a signal) from background noise to support appropriate clinical decision-making [1]. Monitoring systems must also minimize false positives/negatives that may arise from background noise that could lead to inappropriate clinical decision-making [1]. Recently, statistical process control charts (also known as control charts) have been advocated for use in chronic disease monitoring [1, 2].
Control charts were originally developed by Walter Shewhart as a tool for monitoring and controlling manufacturing processes [3]. Control charts distinguish between two sources of variation: ‘common cause’ variation, which is intrinsic to any process and ‘special cause’ variation, caused by a factor extrinsic to the process [4]. A key feature of this classification (common versus special cause variation) is that the actions required to address them are different. To reduce common cause variation, we need to change the underlying process in some fundamental way and for special cause variation, we need to find the extrinsic factor and then act on it.
Shewhart developed a set of simple, graphical tools—control charts—for distinguishing between the two types of variation. Fig. 1 shows a typical control chart of daily systolic blood pressures for a hypertensive patient. Typically, control charts have a central line (the mean) and upper and lower lines representing control limits, which are usually set at three-sigma (standard deviations) from the mean [5]. Any data points that fall outside these limits, or unusual patterns (determined by various run tests) on the control chart, suggest a special cause [5].
|
Other commonly used control charts include run charts and cumulative-sum (CUSUM) charts [6]. The key features of the control charts that have been used commonly used for monitoring variables at patient level such as blood pressure or blood sugar are described elsewhere [5, 7, 8]. Control charts for count or attribute data, which fall outside the scope of this review (because no included study applied this type of chart), are described elsewhere [5, 7, 8] (Table 1).
|
The use of industrial quality control techniques in healthcare settings was advocated in the late 1980s [9, 10]. Since then, control charts have been widely used in the monitoring and improvement of hospital performance [4, 6] and are increasingly being adopted for public health surveillance [6, 11, 12]. These uses have been described elsewhere [6]. Control charts have also been advocated for use in monitoring clinical variables at an individual patient level [1, 13–15] but despite their obvious potential and ease-of use, dissemination and uptake of these applications appears to have been less wide-spread [1].
We carried out a systematic review of uses of control charts to monitor variation in individual patient variables. The review describes applications of control charts and identifies any evidence that indicates whether control charts are associated with improvements in processes or outcomes of care including patient experience.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Search strategy
We searched the following databases: MEDLINE (OVID) 1966 to March week 1 2006 and In-process and other non-indexed citations, CINAHL 1982 to March week 2 2006, Embase, the Cochrane CENTRAL database of controlled trials and DARE (Database of Abstracts of Reviews of Effectiveness) (Issue 1, 2006), and the Social Science Citation Index. 1970-11 March 2006. To capture any publications reported in business and manufacturing literature, we searched Business Source Premier (EBSCO) and Compendex. The search strategy included the following keywords—control charts, statistical process control (including the acronym SPC) and relevant MeSH terms. Full details of the search strategy are available from the authors. The biographies of all retrieved studies were scanned for other relevant studies. Follow-up citations of retrieved studies were identified using the Web of Science Cited Reference search.
Selection of studies
Papers were eligible for inclusion if they described any application of control charts to monitor variation in clinically relevant marker(s) of diseases or risk factors for disease at an individual patient level. All study types were eligible for inclusion, except review and modelling articles. Cohort applications were also excluded if control charts were not used to identify changes at an individual patient level. There were no restrictions on language or year of publication.
Two reviewers (RT and MM) independently reviewed the titles and abstracts of all identified studies against predetermined inclusion and exclusion criteria. Differences were resolved through discussion and consensus. A kappa score of 0.72 indicated substantial agreement between reviewers [16].
Data analysis
Data were extracted from included studies by one reviewer using a modified data extraction sheet [17] and checked by a second reviewer. Because of the heterogeneous nature of study designs, participants and clinical outcome measures used in the included studies, we present a qualitative synthesis of the available evidence.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
We identified 74 relevant abstracts of which 14 considered the application of control charts to individual patient variables. Of these, seven met the inclusion criteria and were included in this review. Summary details of excluded studies are shown in Table 2.
|
The seven included studies are shown in Table 3. Four studies used control charts to monitor changes in peak expiratory flow rate in asthmatic patients [18–21], two studies monitored changes in blood pressure in patients with hypertension ([22, 23]—also monitored blood glucose levels in diabetic patients) and one study [24] monitored serum creatinine in patients who underwent a kidney transplant. The number of patients in the seven studies formed essentially two clusters, with three studies [18, 21, 24] being based on 35–45 patients and four case-studies [19, 20, 22, 23] involved up to three patients. The latter studies were reported as case-studies. The three largest studies compared the accuracy of control charts as a tool or action point for identifying clinically significant changes with existing gold-standard methods [18, 21, 24].
|
Studies used different variables for monitoring with different monitoring intervals and durations, even for the same condition (Table 4). Of the four asthma studies, one used the number of attack-free days [19] and three used peak expiratory flow rate [18, 20, 21] with the number of daily readings ranging from a single daily measure [20] to the best of three daily measures [18, 21]. Two studies used pragmatic monitoring intervals determined by the patient [21, 22]. Data were collected by patients using self-monitoring equipment in all but one study [24]. In this study, the variable was serum creatinine level which is not suitable for self-monitoring. In the majority of studies, charts were constructed by a clinician or researcher although in one study [22], a patient plotted his own readings on a pre-constructed chart.
|
The most commonly used control chart was the XmR chart [18, 20, 22, 23] (Table 4). One study monitored serum creatinine using a CUSUM chart [24], one asthma study used a time between control chart [19] and the remaining asthma study plotted sub-grouped peak flow rates measures using an X-bar chart [21]. With the exception of the serum creatinine study [24], all studies used three-sigma limits to identify special cause variation [18–23] (Table 5).
|
Interventions
In two studies, control charts showing special cause variation were brought into statistical control by making changes to the patients' treatment regime [19, 20]. In one study, an asthmatic patient's control chart indicated an improvement in the patient's status (special cause variation above upper control limit.). An investigation into the underlying factors contributing to this change showed that this was due to a reduction in exposure to irritants in the patient's home [20]. In two studies [22, 23], separate control charts for peak flow rates calculated before and after changes in treatment regime showed a significant drop in mean readings between the two measurement periods. In one study, the asthmatic patient's control charts showed no evidence of special cause variation and lay within safe clinical limits, suggesting that the patient's condition was stable [19] (Table 5).
Comparing control charts with other methods of identifying clinical changes
Three studies compared the use of control charts with existing methods of identifying significant changes in clinical variables (either clinical assessment or published standards) [18, 21, 24]. Two studies found that control charts of peak flow rates were a more accurate method of identifying asthma exacerbations than action points or decision-rules taken from published guidelines [21, 24]. However, Hayati et al. [18] found lower levels of sensitivity and specificity when using control charts as compared with usual detection methods for detecting occupational asthma. They noted that the usual detection method used in their study appeared to be an inaccurate detector of occupational asthma and argued that the lower diagnostic accuracy of control charts was a reflection of diagnostic inaccuracy with the ‘gold standard’ measure (Table 5).
Time to detection
Control charts (using three different decision rules to signal special cause variation) would have identified exacerbations in asthma in some patients before they were clinically identified, with the greatest number of exacerbations identified using a decision-rule of any point below the three-sigma limit [21]. In contrast, Piccoli et al. [24], found no difference in the time taken to identify statistically significant differences in detection time using CUSUM charts or full clinical assessment (Table 5).
Rules for detecting special causes of variation
Three of the studies in this review only used a single exceedence of three-sigma as indicative of special cause variation [18, 19, 22] with two studies using more than one rule [20, 21] (Table 4). Of these, only one study [21] looked at the diagnostic accuracy of each rule. In this case, using a detection rule of two out of three consecutive points between two- and three-sigma had a more favourable sensitivity and specificity than the other the most commonly used rule (a single exceedence of three-sigma limits was shown to have a high false-positive rate).
Patient and carers experience
Three of the included studies suggested that control charting was associated positively with patients' or carers' experiences [20, 22, 23], and that patients did not find the data collection problematic or burdensome although drop-out rates and data completion rates were not explicitly provided. All three studies argued with varying amounts of largely anecdotal qualitative evidence, that patient and carer experience is enhanced by the use of control charts. Herbert and Neuhauser [22], in their case-study report, provided the most information about patient/carer experiences based on interviews with patients and comments from carers including administrative staff. Solodky et al. [23] also argued that use of control charts enabled patients to become partners in their process of care but little evidence is provided to support this. Boggs et al. [20] also claimed that the quality of the patient–doctor consultation was also enhanced because the charts helped to provide greater clarity and focus for the consultation.
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
This review identified seven studies that used control charts to monitor clinical variables in individual patients, which fell into two categories. Three of these studies [18, 21, 24] sought to determine the accuracy of control charts in comparison with existing ‘gold standard methods’ in terms of sensitivity and specificity. This category of studies found control charts to be simple, low-cost, effective tools with good sensitivity and specificity characteristics and concluded in favour of control charts. The other four studies [19, 20, 22, 23] were essentially case-study reports in which control charts were used to monitor clinical variables in individual patients. Appropriately, notions of sensitivity and specificity do not appear in this category of studies, but the positive impact on quality of care, patient experience and carer experience is highlighted, albeit anecdotally and with varying degrees of attention.
Given the importance of monitoring clinical variables, the apparently increasing burden of chronic disease management and the range of potential applications for control-chart-based monitoring (e.g. hypertension, diabetes, asthma etc.) is perhaps surprising that use of control charts for monitoring has received so little attention [1]. There are several possible reasons for this.
Monitoring per se has a number of inherent challenges which include the issues of (a) choice of variable to monitor, (b) frequency of monitoring, (c) who will monitor (care giver or patient). These are generic issues, which have been discussed by Glasziou et al. [1]. However, control based monitoring also comes with its own specific issues. These are discussed below.
An important step in control chart based monitoring is to decide which type of chart to use. There are many types of charts [5–9], with the Shewhart individual charts being the most simplest to construct and works by plotting one data point at a time although CUSUM charts can also be used. We found one example of CUSUM charts being used to monitor individual patients [24]. CUSUM charts have been proposed as a more effective performance monitoring tool than Shewhart charts as they are able to detect change more quickly [8]. However, the relative complexity of constructing CUSUM charts and the need for prior specification of parameters such as a target value and the expected within person variation of the clinical variable of interest, may act as a barrier to their uptake by individual clinicians/patients [5, 24]. Research to explore patient and carer preferences regarding different control charts would be useful especially given the increasing emphasis on patient self-management [1, 25].
Control charts have traditionally used three-sigma control limits [3, 5, 7, 8]. Assuming that the underlying process is stable, the probability of a data point falling outside the control limits is about 1 in 370 for data which is normally distributed [8]. In a clinical situation, this choice of sigma is likely to rest on the severity of the consequences in which the costs of looking for special cause variation, when it does not exist, need to be balanced against the costs of overlooking such a signal, when it does exist [26]. However, none of the studies included in this review compared the performance of control charts with different sigma levels and this appears to be an important issue in the clinical domain. Similarly, questions about choice and combinations of additional rules to signal special causes of variation may also merit study especially as one study [21] found that a detection rule of two out of three consecutive points between two- and three-sigma had a more favourable sensitivity and specificity than a single exceedence of the three-sigma limits.
The ultimate purpose of data is to provide a basis for action to support improvement and so ultimately monitoring schemes need to be evaluated, ideally using randomized controlled trials (RCTs), in terms of their effectiveness at improving patient outcomes. Unfortunately linking monitoring to outcomes is not straightforward because the casual pathway between the upstream monitoring method and health outcome is complex [1]. For example, even with an optimum monitoring scheme, a care giver is required to factor other information about the patient, devise a treatment/management plan and implement it subject to patient consent and compliance. Furthermore, it has been argued that monitoring using control charts alone is not likely to be effective unless it is combined with education and support based on the principles of systems thinking known as continuous self-improvement [25]. So, where control charts are being used like diagnostic tests [18, 21, 24], it appears that a rational progression for studies is to begin by determining accuracy—sensitivity and specificity, and only if this is shown to be adequate does it make sense to study the possible impact on therapy and outcomes. Where monitoring using control charts is being used to manage individual patients, a rigorous evaluation of control charts using outcomes is ultimately required to move beyond the largely anecdotal evidence reported here. Health outcomes are usually compared in randomized controlled trials (RCTs) but this too presents some difficulties—unlike RCTs of drugs, blinding of carer to the intervention is not feasible and so there is a serious risk of contamination, which could bias the process of care in the control group. One way around this is to make the carer the unit of experimentation, but an adequately powered clustered study is likely to be very costly because of the inherently large sample size involved.
In the meanwhile, more comparative studies using proxy outcome measures including patient and carer preferences/perspectives, or even underpowered trials [27] (which could inform the sample size calculations for more definitive cluster RCTs) appear to be urgently required to rigorously evaluate the potentially useful role of control charts in clinical monitoring [1].
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
- Glasziou P, Irwig L, Mant D. Monitoring in chronic disease: a rational approach. Br Med J (2005) 330:644–8.
[Free Full Text] - Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care (2003) 12:458–64.
[Abstract/Free Full Text] - Shewhart WA. Economic Control of Quality of Manufactured Product (1931) New York: D. Van Nostrand Company. (reprint in 1980 by ASQC Quality Press).
- Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman and clinical governance: Shewhart's forgotten lessons. Lancet (2001) 357:463–7.[CrossRef][ISI][Medline]
- Wheeler D. Advanced Topics in Statistical Process Control. The Power of Shewhart's Charts. (1995) United States: SPC press, Inc.
- Woodall W. The use of control charts in health-care and public-health surveillance. J Qual Technol (2006) 38:89–104.
- Wheeler D. Making sense of data (2003) Knoxville, Tennessee: SPC Press.
- Ryan TP. Statistical methods for quality improvement (2000) 2nd ed. New York: John Wiley and Sons.
- Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med (1989) 320:53–6.[ISI][Medline]
- Berwick DM. Controlling variation in health care: a consultation from Walter Shewhart. Med Care (1991) 29:1212–25.[ISI][Medline]
- Morton AP, Whitby M, McLaws M-L, Dobson A, McElwain S, Looke D, et al. The application of statistical process control charts to the detection and monitoring of hospital-acquired infections. J Qual Clin Pract (2001) 21:112–7.[CrossRef][Medline]
- Benneyan JC. Statistical quality control methods in infection control and hospital epidemiology, part I: Introduction and basic theory. Infect Control Hosp Epidemiol (1998) 19:194–214.[ISI][Medline]
- Boggs PB. Rethinking asthma from business to bedside. Ann Allergy Asthma Immunol (2005) 95(Suppl 2):3–9.
- Alemi F, Sullivan T. Tutorial on risk adjusted X-bar charts: applications to measurement of diabetes control. Qual Manag Health Care (2001) 9:57–65.[Medline]
- Bucuvalas JC, Ryckman FC, Arya G, Andrew B, Lesko A, Cole CR, et al. A novel approach to managing variation: Outpatient therapeutic monitoring of calcineurin inhibitor blood levels in liver transplant recipients. J Pediatr (2005) 146:744–50.[CrossRef][ISI][Medline]
- Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas (1960) 20:37–46.[CrossRef][ISI]
- Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising the evidence: reviewing disparate data systematically. Qual Health Res (2002) 12:1284–99.
[Abstract/Free Full Text] - Hayati F, Maghsoodloo S, Devivo MJ, Carnahan BJ. Control chart for monitoring occupational asthma. J Saf Res (2006) 37:17–26.[CrossRef]
- Alemi F, Neuhauser D. Tools, methods, and strategies: time-between control charts for monitoring asthma attacks. Jt Comm J Qual Saf (2004) 30:95–102.[Medline]
- Boggs PB, Wheeler D, Washburne WF, Hayati F. Peak expiratory flow rate control chart in asthma care: chart construction and use in asthma care. Ann Allergy Asthma Immunol (1998) 81:552–62.[ISI][Medline]
- Gibson PG, Wlodarczyk J, Hensley MJ, Murree-Allen K, Olson LG, Saltos N. Using quality-control analysis of peak expiratory flow recordings to guide therapy for asthma. Ann Intern Med (1995) 123:488–92.
[Abstract/Free Full Text] - Hebert C, Neuhauser D. Improving hypertension care with patient-generated run charts: physician, patient, and management perspectives. Qual Manag Health Care (2004) 13:174–7.[Medline]
- Solodky C, Chen H, Jones PK, Katcher W, Neuhauser D. Patients as partners in clinical research: a proposal for applying quality improvement methods to patient care. Med Care (1998) 36(Suppl):AS13–20.[CrossRef][ISI][Medline]
- Piccoli A, Rizzoni G, Tessarin C, Calconi G, Filippini M, Dugo M, et al. Long-term monitoring of renal transplant patients by a CUSUM test on serum creatinine. Nephron (1987) 47:87–94.[ISI][Medline]
- Alemi F, Neuhauser D, Ardito S, Headrick L, Moore S, Hekelman F, Norman L. Continuous self-improvement: systems thinking in a personal context. Jt Comm J Qual Improv (2000) 26:74–86.[Medline]
- Nelson LS. Technical Aids: Notes on the Shewhart Control Chart. J Qual Technol (1999) 31:124–6.
- Lilford R, Stevens AJ. Underpowered studies. Br J Sug (2002) 89:129–31.[ISI][Medline]
- Chu J, Neuhauser DV, Schwartz I, Aye HH. The efficacy of automated/electrical twitch obtaining intramuscular stimulation (atoims/etoims) for chronic pain control: evaluation with statistical process control methods. Electromyogr Clin Neurophysiol (2002) 42:393–40.[Medline]
- Freidank-Mueschenborn E, Fox AW. Cusums to measure chronic daily headache. Headache (2006) 46:110–4.[CrossRef][ISI][Medline]
- Kent DL, Vermes D, McDonell M, Henikoff J, Fihn SD. A model for planning optimal follow-up for outpatients on warfarin anticoagulation. Med Decis Making (1992) 12:132–41.
[Abstract/Free Full Text] - Milligan PE, Banet GA, Waterman AD, Gatchel SK, Gage BF. Substitution of generic warfarin for coumadin in an HMO setting. Ann Pharmacother (2002) 36:764–8.[Abstract]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||