OUP user menu

Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review

Mirella Minkman, Kees Ahaus, Robbert Huijsman
DOI: http://dx.doi.org/10.1093/intqhc/mzl071 90-104 First published online: 2 February 2007

Abstract

Purpose Health care organizations have to improve their performance for multiple stakeholders and organize integrated care. To facilitate this, various integrated quality management models can be used. This article reviews the literature on the Malcolm Baldrige Quality Award (MBQA) criteria, the European Foundation Quality Management (EFQM) Excellence model (Excellence award models) and the Chronic Care Model. The focus is on the empirical evidence for improved performance by the implementation of interventions based on these models.

Data sources A systematic literature review from 1995 to May 2006 in the Pubmed, Cochrane, and ABI- databases was conducted.

Study selection After selection, 37 studies were included, 16 in the Excellence award model search and 21 in the Chronic Care Model search.

Data extraction and results of analysis Data were retrieved about the main intervention elements, study design, evidence level, setting and context factors, data collection and analysis, principal results and performance dimensions. No Excellence Award model studies with controlled designs were found. For the Chronic Care Model, one systematic review, one meta analysis and six controlled studies were included. Seventeen studies (2 in Excellence award model, 15 in Chronic Care Model) reported one or more significant results.

Conclusion There is some evidence that implementing interventions based on the ‘evidence-based developed’ Chronic Care Model may improve process or outcome performances. The evidence for performance improvement by interventions based on the ‘expert-based developed’ MBQA criteria and the EFQM Excellence model is more limited. Only a few studies include balanced measures on multiple performance dimensions. Considering the need for integrated care and chronic care improvement, the further development of these models for guiding improvements in integrated care settings and their specific context factors is suggested.

Keywords
  • performance improvement
  • total quality management
  • chronic disease
  • organizational models
  • evidence-based

Purpose

In order to prosper in today's dynamic health care systems, organizations such as hospitals must work effectively, be innovative and organize efficiently. A focus on multiple performance measures is needed to assess the quality level reached [1]. Not only patient outcome measures, but also worker's satisfaction and organizational and financial performance have to be managed and improved. This multidimensional approach by health care management corresponds with current definitions of the quality of care itself. The Institute of Medicine defines good care as safe, effective, timely, patient-centered and efficient. This definition also reflects multiple dimensions of Quality, including organizational aspects like a streamlined care process, good access and a financially healthy organization [2].

Another development is observed in the literature. The characteristics and boundaries of health care organizations are changing. ‘Patient-centered care’ focusses on the total needs of patients, not only on the services provided by one professional or organization. It is important to sustain seamless integrated care during the whole care process. For health care organizations, this requires ‘horizontal’ coordination, collaboration with other organizations and community partners or service integration. Partners in the care chain and the functioning of the care chain or care network as a whole contribute to the quality of care. The international literature on integrated care, disease management and development of care chains and networks addresses this issue [3].

To facilitate the improvement of health care quality and performance, a large range of quality management and organizational models has been developed [4, 5]. In this article, we focus on frequently used quality management models in health care: the European Foundation for Quality Management (EFQM) Excellence model and the Malcolm Baldrige Quality Award (MBQA) criteria on the one hand and the Chronic Care Model on the other. We selected these integrated quality management models on the basis of multiple criteria. First, these models all consist of multiple ‘enablers’ of good quality care (for instance, leadership or delivery system design). Enablers cover the processes, structure and mean values of an organization [6]. Second, these models focus on multiple performance dimensions for multiple stakeholders (for instance, organizational performance, worker satisfaction). Lastly, they assume dynamic relationships between improved performance and implementation of interventions based on the models enablers [7, 8]. The EFQM/MBQA and the Chronic Care Model are frequently used as frameworks for local improvement or national collaborative improvement programs. In this article, we focus on the available empirical evidence for these models in respect of improving health care performance. The research question is: what empirical evidence is available for improved performance in health care settings by implementing interventions based on the enablers of the EFQM Excellence model/MBQA criteria or the Chronic Care Model?

The EFQM Excellence model and MBQA criteria

The EFQM Excellence model conceptualizes organizations by discerning enabler and performance elements as ingredients for striving toward excellence [6, 7]. The EFQM Excellence model shows many parallels with the assessment model of the MBQA and international quality award criteria [9]. Originally, these widespread quality management models were developed in the private sector and may be viewed as an operationalization of Total Quality Management philosophies. Whereas the MBQA criteria consist of seven elements (leadership, strategic planning, customer and market focus, measurement, analysis and knowledge management, human resource focus and, process management and results), the EFQM Excellence model consists of nine elements (leadership, policy and strategy, management of people, partnership and resources and processes, key performance results, and people, customer and society results). Both models have healthcare-specific versions and are used in all types of health care organizations, regardless of sector and size or maturity [6, 9]. They are integrated models that cover quality management as an integral part of all professional and management functions at all levels of an organization. A basic premise of the models is that enablers direct and drive performance; organizations with well-developed enablers will have excellent results [6, 9]. Because of their comparability, we will focus on the EFQM and MBQA models as one category.

The Chronic Care Model

The Chronic Care Model identifies the essential elements of a (local) health care system, which encourage high-quality chronic disease care. The model is based on evidence-based change concepts and responded to the need for a quality improvement model that fits the characteristics of chronic care. The model can be used for various chronic illnesses, health care settings and target populations. The Chronic Care Model has also been used as an improvement tool in multiple chronic care improvement collaboratives [10].

The Chronic Care Model describes six elements–the community, the health system within it and four elements within the health system: self-management support, delivery system design, decision support and clinical information systems. Similar to the EFQM Excellence model and the MBQA criteria, the Chronic Care Model focusses on multiple dimensions of performance and on multiple stakeholders. Successful implementation of interventions based on the six elements may result in productive interactions between informed and activated patients and prepared and proactive care teams and in better functional and clinical outcomes. An expanded model based on the Chronic Care Model has a number of extras relating to patient safety, staff development, cultural aspects, coordination and the six performance dimensions of Institute of Medicine (IOM) definition of quality [11].

Summarizing, the EFQM/MBQA model and the Chronic Care Model are both integrated quality models that are adopted by many health care organizations in order to direct effective interventions and improved performance. Each model consists of enabler elements and performance dimensions and assumes positive relationships between them. Although these models are commonly used in practice, less is known about empirical evidence concerning the effects of interventions, based on the elements of these models, on improved performance. This conclusion, regarding the EFQM/MBQA models was supported by Shortell et al. [12], Nabitz et al. [6] and others [5, 1315], who stated that although the EFQM/MBQA models have high face validity, there are only a few publications in the academic literature. The Chronic Care Model is based on evidence-based directions for each element, but extensive research on the effects of the model as a whole on improved performance remains limited and comes mainly from self-reported, uncontrolled studies [1618].

Øvretveit [19] argues that, when assessing the (evidence for) achieved results of quality improvements, the influence of context factors and the degree of context dependence of the interventions have to be considered. Conditions that are likely to influence results are the type of health care system, social values, health reform, the history of quality and the language and politics of quality. According to Øvretveit, conditional interaction is systematically obscured in randomized controlled trials.

Data sources and study selection

We searched the PubMed, Cochrane, and ABI/Inform databases from 1995 to April 2006 and the reference lists of relevant papers. Study selection was based on the following criteria. First, we focussed on studies with empirical data published in peer-reviewed journals. Further, we included only studies that used the model as a basis for implemented interventions and focussed on multiple or all elements. We conducted two searches: one on studies using the EFQM Excellence model and the MBQA criteria and national variants, and one on studies using the Chronic Care Model. Search terms were ‘Baldrige’, ‘EFQM’, ‘MBQA’, ‘excellence model’, ‘quality award’ and national variants like ‘Deming quality award’ in the first and ‘chronic care’ combined with ‘model’ and ‘chronic care model’ in the second. One author screened the initial search results (M.M.) and at least two authors screened the selected studies (M.M. and R.H. or K.A.). All of the authors independently assessed the evidence levels of selected studies; differences in interpretation were resolved by consensus. A specified evidence-level table based on Effective Practice and Organization of Care group (Review group of the cochrane collaboration [EPOC]) criteria was used (Table 1). Evidence levels range from systematic reviews (A1) and randomized trials (A2) to descriptive non-analytical studies of multiple projects (D1), single projects (D2) or literature reviews (D3). The criterion for significant change in all studies was set at p≤0.05.

View this table:
Table 1

Search results and evidence level classification

LevelDescriptionEFQM/MBQAChronicCare Model
A1Systematic review02
Review of data of multiple RCT studies
A2Randomized trial Comparative study with (random) intervention and control group design01
BControlled trial0
Trial with intervention and control group and comparisons on outcomes04
B1 more measurement points01
B2 one measurement point
CNon-controlled study
C1 multiple case, more measurement points37
C2 multiple case, one measurement point50
C3 single case, more measurement point53
C4 single case, one measurement point10
DDescriptive, non-analytical
D1 multiple projects02
D2 single project20
D3 literature review01
Total number of studies1621

Data extraction and results

Data were retrieved regarding the study design and evidence level, setting (organization and country), domain model elements in intervention, data collection and analysis techniques, main results and context factors described. Of an initial total of 850 studies, 16 were included from the EFQM/MBQA search [6, 12, 13, 15, 2031] and of initial 686 studies, 21 were included from the Chronic Care Model search [10, 1618, 3248], (table 1).

EFQM and MBQA results

The characteristics of the EFQM/MBQA studies are reported in Table 2. Regarding the evidence levels, no A- or B-level studies were found. Eight of the 14 C-level studies reported data on more than one measurement point (Table 1). Ten studies used the EFQM Excellence model and five the MBQA criteria as a model for improvement. Twelve of the 16 studies were published in 1999–2002. Eleven studies were conducted in Europe, four in the USA and one in Korea. Study settings were mainly hospitals (eight studies) and/or primary or community care services (six studies). In three studies, the results were statistically tested; two of them reported one or more significant improvements [12, 15]. Six of the eight C1- and C3-level case studies reported improved outcomes, but none are confirmed by statistical analysis.

View this table:
Table 2

Reviewed EFQM or MBQA studies

AuthorsModel elements in interventionEvidence levelSetting and countryData collection and analysisMain results
Goonan and Stoltz [13]Baldrige improvement programC1One US health service (21 US hospitals and 3 nursing homes)Measurement of multiple indicators on Baldrige categories. No statistical testingSystematic quality improvement efforts resulted in first Baldrige Quality Award in health care. Number of improvements on multiple criteria, benchmarked when possible
Sanchez et al. [20]Eight-year EFQM improvement programC1Thirty-one organizations (hospitals, primary care, mental health, emergency services) in Spain.Mean percentage of maximum possible self-assessment scores for result and enabler criteria. Customer satisfaction surveys. Seven key performance indicators. No statistical testingIncrease in self-assessments, 10 organizations (32%) scored >400 points, two (6%) >500 points. Scores improved, especially ‘processes’. Customer satisfaction scores outperformed averages in national benchmark. Some improvements in key performance measures (4 year measures)
Shirks et al. [21]Three-year Baldrige improvement programC1Eleven US service network organizationsThree-year assessment scores on 6 Baldrige categories. No statistical testingNo significant improvement on six categories in the overall group scores or in two service networks with 4 year measurements. Average scores between 30 and 40% level
Arcelay et al. [22]Three-year EFQM improvement programC2Twenty-six public health services (hospitals, primary care, prov. management) in SpainMeasurement of EFQM self-assessment scores. No statistical testingAverage score of 259 (max. 500), 5 organizations scored between 100 and 200, 9 between 200 and 300, 9 between 300 and 400 3 no scores
Goldstein and Schweikhart [15]Baldrige category 1–6 and organizational performanceC2Two-hundred and twenty US hospitalsMailed questionnaire to head of quality departments. 14 weighted Baldrige categories and five result categories. Correlative statistics (regression analysis)All relationships between the Baldrige categories 1–6 and performances were statistically i.p.v. are significant. Health care results and financial and market results were less well predicted. Strongest relationship between Baldrige criteria and staff and work system results
Lee et al. [23]Seven Baldrige categories, CQI pyramidC2Sixty-seven Korean hospitalsMailed questionnaire to head of quality departments. Fourty three Baldrige items. Correlative statistics (t-tests, ANOVA, regression analysis)Average Baldrige implementation score of 3.34 (max.5). Customer satisfaction achieved highest score (3.88). Implementation score higher for larger hospitals (not significant)
Moeller [24]EFQM improvement programC2Seventeen German hospitalsEFQM weighted assessment scores on nine criteria No statistical testingTwo hospitals scored <200 points, 9 scored 201–300, 5 scored 301–400 and 1 scores 400. Largest intervals (min–max. score) measured at processes and people results
Shortell et al. [12]TQM programs based on five principlesC2Sixty-one US hospitalsCross-sectional examination of relationships between organizational culture, quality improvement processes and outcomes. Correlative statistics (path analysis based on least squares regression)Significant associations for improved performance for human resources management and patient outcomes, not financial outcomes. No significant association between TQM implementation and LoS and charges for six chronic conditions. A team-focussed, risk-taking culture, personal development and focussed implementation were positively associated with degree of TQM implementation
Freer and Jackson [25]Three-year Baldrige improvement programC3One UK trust (hospital and community services)Baldrige self-assessment scores on 28 items, 4-year measurements. No statistical testingUpward trend in self-assessment scores from 219 (t = 1) to 455 (t = 4). Baldrige framework was particularly useful to integrate services
Harr [26]Five-year EFQM improvement programC3One Swiss dental practiceMeasurement on multiple indicators on nine EFQM criteria. No statistical testingUpward trend on multiple indicators (customer, people, society and key performance results). Positive scores related to available benchmarks
Harten et al. [27]EFQM improvement program (including ISO)C3One Dutch rehabilitation clinicProcess analysis and two measurements on assessment scores on five EFQM criteria. Comparison with national benchmark. No statistical testingUpward trend in four of six EFQM assessment scores, one equal (people management) and one decreased (resources). Participation in quality program related to greater work satisfaction. Assessment scores on first measure outperformed four of five criteria in benchmark
Jackson and Bircher [28]EFQM improvement programC3One UK primary care clinicMeasurement of multiple indicators on EFQM performance criteria. Survey on worker satisfaction. No statistical testingImprovements on multiple indicators: clinical outcomes, organizational efficiency, staff satisfaction
Nabitz et al., 2000 [6]EFQM improvement programC3One Dutch addiction clinic. Overview of European activitiesEFQM assessment scores on nine criteria, pre- and post-improvement. No statistical testingImproved assessment score from 350 to 510 resulted in Dutch Quality Award
Gene-Badia et al. [29]EFQM improvement programC4One primary health care organition in SpainEFQM self-assessment scores on nine criteria. Scores are compared with external assessment scores. No statistical testingComparable assessment scores and areas for improvement between different assessments
Dunn and Mathews [30]EFQM improvement programD2One voluntary organization in IrelandAnalyses of steps taken and operationalization of the model (including ISO, IiP, BSC). No statistical testingKey performance indicators were established and measured on a yearly basis
Holland and Fennel [31]EFQM improvement programD2One UK Health TrustBaseline self-assessment by EFQM-based score tool. Rating on five level scale. No statistical testingBaseline assessment conducted with developed tool, resulting in action plans

The study by Goldstein and Schweikhart [15] in 220 US hospitals provided the strongest evidence: all the relationships between the MBQA categories and examined performance were statistically significant. They found the strongest relationship with staff and work system results. Health care, financial and market results were less well predicted by the MBQA criteria. Sanchez et al. [20] and Shirks et al. [21] measured the results of their EFQM and MBQA improvement programs over 4-year periods and found positive trends for process performance, but no significant improvements for any other performances. In three-quarter of the included studies, three or more apparent context factors such as characteristics of the health care system, social values or the history of quality assurance were discussed. The effects of these factors on performances are less well described. Shortell et al. [12] and Lee et al. [23] explicitly included statistical analyses on context factors. Lee concluded that scientific skills in decision making and the adoption of a quality information system were the most important contributing factors. Shortell found significant relations for a participative, flexible and risk-taking organizational culture. Larger hospitals experienced lower clinical efficiency due to more bureaucratic and hierarchical cultures that served as a barrier for quality improvement implementation.

Chronic Care Model results

The characteristics of the Chronic Care Model studies are reported in table 3. Regarding the evidence levels, one meta-analysis and one systematic review were found [16, 17], one randomized trial [18], five controlled studies [3236], and a variety of case studies and project reviews (see table 1). Eighteen studies reported interventions on four or more Chronic Care Model elements. Most studies were published in the period 2003–2006 (15 studies). Almost all studies were conducted in the USA. Study settings were often primary or community care settings (15 studies), hospitals (4 studies) and/or outpatient clinics (4 studies) or networks of combined services. Eighteen of 21 studies included diabetes patients, 5 included asthmatic patients, 3 cardiovascular patients and 2 depressed patients. Fifteen studies reported one or more statistically significant improvements. Six studies did not test their results statistically.

View this table:
Table 3

Reviewed Chronic Care Model studies

AuthorsModel elements in interventionEvidence levelSetting and countryData collection and analysisMain results
Bodenheimer et al. [17]SM, SD, DS, CISA1Thirty-nine international studies on diabetes careNumber of CCM elements implemented, number of significant improvements in process or outcomes of care. Overview of studies on costs. No statistical testingThirty-two of thirty-nine shows≥1 significant improvement. All studies with four elements improved process and outcome; 15/23 studies with one to three elements improved≥1 outcomes, 16/20 improved≥1 process measure. No review of effects on costs
Tsai et al. [16]SM, DSD, DS, CIS, HCO, CRA1One-hundred and twelve studies; 27 on asthma, 21 congestive heart failure, 33 depression, 31 diabetes, 107 outpatient, 5 inpatient settingsMeta-analysis of clinical outcomes, QoL, processes of care. Correlative statistics (effect sizes, Hedges g, risk ratio, SD, random effects meta-regression models)Significant improvements on outcomes and process measures with≥1 CCM element implementation. Effects somewhat stronger for DSD and SM. Mixed evidence for QoL results consistent for a variety of chronic illnesses
Piatt et al. [18]SM, DSD, DS, CIS, HCO, CRA2Eleven randomized US pcp: 3 practice CCM implementation (20 patients), 3 practice provider education (38 patients), 5 practice usual care (51 patients)Chart review. Five outcome measures and measures on well-being, knowledge, empowerment, self-monitoring. Descriptive and correlative statistics (Paired t-test, McNemar's, ANOVA, regression analysis, mixed modeling)Significant improvement in two outcome measures (A1C, non-HDL) and self-monitoring of blood glucose in the CCM group compared with the other groups. Within the CCM group also significant improvements on A1C, HDL, empowerment and self-monitoring
Benedetti et al. [32]SM, DSD, DS, CIS, HCO, CRB1One US multi-specialty practice: 698 diabetes patients. Control group: 1300 diabetes patientsParticipation levels of CCM implementation (1, 2 or 3 years), 12 patient process and outcome measures. Survey on provider satisfaction. Descriptive statistics (two-tailed t-tests, F-tests)Significant improvement in 7 of 12 outcomes measures in intervention group. Improved eye exams and blood pressure significantly associated with participation level. Provider satisfaction increased from 28 to 78% in intervention group
Chumbler et al. [33]SM, DSD, DS, CISB1Eight hundred diabetes patients (400 treatment, 400 comparison) in the USA and Puerto Rico in outpatient care coordination programInpatient service use (hospital admissions, LoS) and outpatient use. Emergency Department (ED) visits, primary care visits, ophthalmology, podiatry, diabetes clinic visits. Correlative statistics (regression analysis, difference–indifference approach)Significant decrease in primary care visits in control group, increase (not significant) in intervention group. No differences between groups in podiatry, opthalmology and diabetes clinic visits. Both groups significantly decreased hospital admissions, LoS, ED visits
Mangione et al. [34]SM, DSD, DS, CIS, HCO, CRB1Nine intervention sites (8 pcp, 1 pp); 4 control sites (3 pcp, 1 pp). 385 and 126 asthma patients in the USAFourteen asthma-related process indicators. Medical record data collection, phone interviews with parents/guardians. Correlative statistics (regression models, linear probability model)Significant improvement for six of eight process measures (peak flow monitoring, action plan), two outcome measures (general and asthma specific QoL) in intervention group, control group no improvement. Overall process measures significantly improved in intervention group, control group unchanged
Schonlau et al. [35]SM, DSD, DS, CIS, HCO, CRB1six intervention sites (four pcp, two pp); three control sites (two pcp, one pp). One-hundred and twenty three and 62 asthma patients in the USANine asthma-related indicators. Medical record data collection, patient telephone surveys. Correlative statistics (t-tests, regression models, probability model)Significant improvement of overall interventions sites scores. Significant higher SM scores, satisfaction with clinician and educator communication and educational sessions attendance in intervention sites. No significant improvements in asthma-specific QoL, LoS or acute service use
Feifer et al. [36]SM, DSD, DS, CIS, HCO, CRB2Nine US pcp, diabetes and cardiovascular patients, 21 control pcp sitesGroup interviews for CCM implementation by ACIC survey. twenty-two disease-specific indicators. Correlative statistics (Spearman rank correlations)DS, SM and DSD pos. correlated with clinical performance. Five of 22 elements pos. correlated: guidelines, team leadership, follow-up, self-care support and behavioral therapy. More fit with CCM related to better clinical performance
Bonomi et al. [37]SM, DSD, DS, CIS, HCO, CRC1One-hundred and eight US hc organizations (clinics/managed care/safety nets) on diabetes, asthma, depression or CHF patientsPre- and post-measurement of implementation of CCM elements on ACIC scale. Descriptive and correlative statistics (paired t-tests, correlation analysis)For diabetes and CHF teams, significant improvements in all six CCM elements ACIC subscale scores. The most substantial improvements in DSD, CIS and DS scores. Strong and positive correlations between ACIC scores and faculty ratings
Chin et al. [38]SM, DSD, DS, CIS, CRC1Nineteen US chc, 1620 diabetes patientsChart review on eight process measures and one outcome measure (HbA1c). Provider survey. Phone interviews to assess implementation, facilitators and spread. Correlative statistics (regression analyses)Significant improvements on seven process measures (HbA1c measurement, eye and foot exam, dental referral, lipid and urine microalbumin ass., dietary consult), outcome measure (HbA1c) not significant, 95% of respondents positive about improvement program, including CCM approach
Daniel et al. [39]SM, DSD, DS, CIS, HCO, CRC1Thirty-nine pcp teams of one US state in two diabetes collaborativesSix process measures and three outcome measures (HbA1c, LDL, blood pressure). No statistical testingMost teams showed improvements in absolute scores. Medians of all nine measures increased at collaborative level. Absolute improvements higher for process than for outcome measures
Landis et al. [40]SM, DSD, DS, CIS, HCO, CRC1Six US community-based family medicine programs on diabetesProvider Recognition Program scores (six diabetes process and four outcome measures). ACIC survey on CCM implementation. No statistical testingThree of six teams improved total PRP score, five of six improved average ACIC score. Team with most implemented interventions improved PRP scores most (eligible for Recognition Award)
Sperl-Hillen, et al. 2000 [41]SM, DSD, DS, CISC1Eighteen US pcp, 7000 diabetes patientsTwo process measures (HbA1c and LDL test rate) and outcome measures (HbA1c and LDL). Descriptive statistics (z-tests, t-tests)Significant improvements in all process and outcome measures (HbA1c and LDL outcomes and test rates)
Wagner et al. [10]SM, DSD, DS, CIS, HCO, CRC1Twenty-three US organizations (clinics, community clinics, managed care, safety net) diabetes patientsHbA1c assays and levels in patient. With SM goals, documented smoking status, self-reported data. ACIC survey on CCM implementation. Descriptive statistics (t-tests)Improved post-measures for HbA1c assays and levels, goal-setting and smoking status (no stat. testing). ACIC measurement showed significant improvement in overall scores, SM, DSD, DS and CIS. Improved ratings by faculty members
Sperl-Hillen et al. [42]SM, DSD, DS, CIS, HCO, CRC1Seventeen US pcp, diabetes patientsACIC survey on implementation of CCM. Three process measures (HbA1C and LDL testing, combined), three outcome measures (LDL, HbA1C, combined). Correlative statistics (Pearson correlations)ACIC scores ranged from 4.9 to 6.3. Significant correlation between implementation of DSD and process and outcome measures. SM and CIS were associated, but not significant
Mohler and Mohler [43]SM, DSD, DS, CISC3One US pcp, 387 diabetes patientsThree process measures (aspirin use, eye and urine examinations), three outcome measures (blood pressure, LDL, HbA1c). Survey on physician satisfaction. Financial outcomes. No statistical testingUpward trend of percentage of patients meeting goals on all six measures. Twelve of 14 physicians increased satisfaction with diabetes care. Costs per patient increased $114 per year, due to nurse case-manager and data clerk salary
Siminerio, et al. [44]SM, DSD, DSC3One US pcp, 104 diabetes patientsAdherence to standards (six measures), provider-perceived barriers to care. Five outcome measures (HbA1c, blood pressure, LDL, knowledge, empowerment levels). Descriptive statistics (McNemars test, paired t-test)Significant improvements of standard adherence on six measures and patients' diabetes knowledge. Educated patients significantly improve HbA1c and H/LDLc levels. No significant improvements in outcome measures and patient-empowerment levels
Stroebel et al. [45]SM, DSD, DS, CIS, HCO, CRC3One-hundred and nine patients with diabetes, hypertension and/or hyperlipidemia in a US free army clinicSix outcome measures (on blood pressure, HbA1c level, risk group LDL). Number of patients that improve at least one level. Descriptive statistics (paired t-tests)Seventy-three percent of patients significantly improved on ≥1 measure, 64% of hypertension patients improved ≥1 stage, 53% of diabetes patients improved at least 1% HbA1c, 58% of hyperlipidemia patients dropped ≥1 risk group. Significant improvements for group results on three outcome measures (arterial pressure, HbA1c and LDL)
Bodenheimer et al. [46]SM, DSD, DS, CISD1One pp (diabetes), two integrated delivery systems (diabetes, asthma), One chc (diabetes) in the USADescription of characteristics, implemented interventions and main resultsCases illustrate a broad variety of interventions and results. None achieved full implementation of the model, all booked results on clinical outcome or process measures
Wang et al. 2004 [47]Case 1: HCO, CIS, DSD Case 2: SM, DSD1Two chc, diabetes patients in the USADescription of characteristics, implemented interventions and some results. Interviews with team leadersCases illustrate interventions, some results and lessons learned. Case 1 improved average HbA1c from 9.4 to 7.8 and Case 2 from 6.9 to 6.8, increase in number of patients with SM goals
Bodenheimer [48]SM, DSD, DS, CIS, HCO, CRD3Expert selection of international studies (in search for meta-analyses and reviews)Report of evidence found on model and models componentsEvidence available for each CCM component. Few studies available with evidence on multiple components. Relation between number of components used and improved clinical outcomes suggested. No conclusions on most effective components for specific diseases
  • SM, self management; DSD, delivery system design; DS, decision support; CIS, clinical information system; HCO, health care organization; CR, community resources; Pcp, primary care practice; Chc, community health center; Pp, private practice; LoS, length of stay; ACIC, assessment of chronic illness care; CHF, congestive heart failure; QoL, Quality of Life.

The strongest evidence was found in the meta-analysis of 112 studies by Tsai et al. [16]. Tsai et al. found evidence for significant improvements on process or outcome measures by implementing at least one Chronic Care Model element. The review by Bodenheimer et al. [17] also showed high percentages of studies with positive effects, especially for studies which included four elements or self-management interventions. Compared with normal care or interventions supported by professional education, Piatt et al. [18] found that the Chronic-Care-Model-based group performed significantly better on two diabetes clinical outcome measures and self-management monitoring. Similar to Tsai et al., the B-level studies by Benedetti et al. [32], Mangione et al. [34] and Schonlau et al. [35] reported mainly significant improvements on outcome (HbA1C, LDL. and so on) or process measures (peak-flow monitoring, clinical testing and so on) at operational level. Chumbler et al. [33] found no changes in performance, except for increased service-use in primary and ED care. Sperl-Hillen et al. [42] analysed whether each Chronic Care Model element contributed equally and found positive correlations for delivery system design and positive associations for self-management and clinical information systems. Feifer et al. [36] found decision support, self-management and delivery system design to be positively correlated with clinical performance. Improved fit with the Chronic Care Model was related to clinical performance in this study. The performance dimensions included in the A- and B-level studies were further analysed (Table 4). Almost all the studies measure clinical or efficiency results such as test outcomes, length of stay or numbers of clinical exams, whereas less attention is paid to financial or professional results (such as worker's satisfaction).

View this table:
Table 4

Included performance dimensions (A and B level studies)

StudyClinical results/QoLEfficiency resultsaWorker/professional resultsCustomer resultsbFinancial results
Bodenheimer et al. [17]+++
Tsai et al. [16]++
Piatt et al. [18]++
Benedetti et al. [32]+++
Chumbler et al. [33]+
Mangione et al. [34]+++
Schonlau et al. [35]+++
Feifer et al. [36]++
  • aService use (clinical exams, protocols followed, length of stay and so on); bsatisfaction, knowledge, empowerment — QoL, Quality of Life.

Regarding context factors, one-third of the studies described three or more context factors, mostly characteristics of the health care system, setting and patient populations. Only a few studies discuss influences of context factors on performances measured. Landis et al. [40] concluded that the one site that clearly outperformed the other five in her study, had a strong organizational foundation of a quality improvement culture and strong physician leadership. Also, Bodenheimer et al. [17] concluded that visionary clinical leadership and financial conditions are needed for successful improvements in chronic care.

Discussion and conclusion

Our finding in this review was that there is weak evidence for improved performance by implementing interventions based on the EFQM or MBQA models elements in health care settings. No randomized or controlled studies were found. The small number of EFQM/MBQA studies is surprizing because these models are widespread and have been used for many years. For the Chronic Care Model, the studies used more solid designs and methods. Some evidence has been found that implementing interventions based on the Chronic Care Model improves performance, but the conclusions are all drawn in the US settings for specific patient groups. Considering the quality of the studies, the description of the implemented interventions was often limited. For the EFQM/MBQA studies, the data in the multiple case studies were not systematically measured over time, making statements on improved performance impossible. An explanation for these differences found may lie in the origin of the models and their use in practice. The EFQM/MBQA models are ‘experience-based’, whereas the Chronic Care Model is ‘evidence-based’. The data in the EFQM and MBQA studies were mainly gathered from improvement projects, instead of research projects designed for statistical testing. The Chronic Care Model studies focus merely on patient groups and clinical measures, which better ‘fit’ the more biomedically oriented scientific research paradigm of controlled (randomized) trials.

Although the models have different origins, some elements show similarities. Interventions on ‘clinical information systems’ in the Chronic Care Model correspond to interventions in the EFQM ‘resources’ element. Shifts in care processes or tasks of workers both fit in Chronic Care Model's ‘delivery system design’ and EFQM/MBQA's ‘processes’ and ‘people’. Although the Chronic Care Model pays attention to aspects such as leadership (within health care organization), these elements combined with health policy are more emphasized in the EFQM/MBQA models. On the other hand, the Chronic Care Model defines ‘self-management’ as a crucial element, whereas the EFQM/MBQA models do not. It would appear that the EFQM/MBQA models are mainly used as management tools, e.g. at strategic level, whereas the Chronic Care Model is mainly used as a tool to optimize care for a specific patient group at the more operational or process level. Comments made on the Chronic Care Model include the fact that aspects such as culture, leadership and a greater business focus are missing [37], whereas the EFQM/MBQA is sometimes said not to provide a sufficient ‘health care fit’ [6]. Regarding integrated care, the studies focussed merely on just one organization. In the study by Shortell et al. [49] regarding the impact of quality improvement on clinical practice, no studies focussing on the continuum of care were found. Some studies, however, addressed the need for integrated care and management of the total care process. Freer and Jackson [25] stressed the helpfulness of the MBQA program for integrating services, and Chumbler et al. [33] measured inpatient and outpatient clinic outcomes to stress the interrelatedness for diabetic patients. Although ‘the community’ enabler in the Chronic Care Model points out relationships with other (care and welfare) organizations, integrated care chains are not the domain subjects of study. With regard to the increasing numbers of chronically ill and the need for integrated care, further development of these models is required in terms of both their usefulness and their applicability to care chains.

No studies covered more than three performance dimensions. With the attention to costs and efficiency in current health, it is surprizing that only a few Chronic Care Model studies measured financial performances. Moreover, measures of worker satisfaction (the care team) and patient judgements are also often lacking. The assessment procedures used in the EFQM/MBQA studies include multiple performance dimensions, but information about the results on these dimensions is often not systematically reported.

The EFQM/MBQA studies paid more attention to the influence of context factors than the Chronic Care Model studies. As known from the literature, organizational characteristics such as culture and leadership and political developments affect the results [50,51]. The included studies conform this by naming these factors as influencing factors. For the EFQM/MBQA models, there are also studies in other sectors available. These studies show mixed but mostly positive results. Kaynak [52] found 18 studies on the relationship between total quality management implementation and improved performance, all of which showed one or more positive effects. A recent controlled study by Boulter et al. [53] found evidence that the 120 award-winning companies experienced a greater increase in shared values, capital expenditure, growth in assets and reduction in costs over both short and long periods of time. Summarized, the results indicate that effective implementation of the EFQM model makes good economic sense in non-health care settings. Another interesting issue is how organizations develop in increasing performance. Both the EFQM/MBQA models and the Chronic Care Model have five ‘development phases’ that suggest pathways for growth [6, 8, 9]. The assumption is that improved performance is related to growth in the developmental phase. In this perspective, insight into the relationship between interventions, organizational development and performance is interesting, but is yet hardly a subject of research.

Our research contains several limitations. There is a lack of insight as to which models' elements contribute the most to performance and to which confounding and context variables are present. Furthermore, the effects of collaborative improvement-program interventions are not separated out when assessing the results. Another limitation concerns the methodological quality of the studies. The interventions differ from one study to another, meaning that generalizations are hazardous and that the findings are not reproducible for larger populations or other organizations. The absence of publication bias cannot be guaranteed. Also, we conducted a search for studies in which reference was made to the use of the model, whereas other studies that might have implemented comparable interventions were not included. Finally, Grol [54] and Øvretveit and Gustafson [55] stress the complexity of solid research designs given the large number of possible interacting dimensions, making it difficult to prove firm relationships. The richness of interventions, confounding variables and effects of organizational development mean that the evidence for relationships between using the model as a whole and performance largely remains a grey area.

Despite these caveats, this review does support the conclusion that interventions based on the Chronic Care Model may improve process and outcome measures in some situations. For the EFQM/MBQA, the evidence found is less strong. Future research should pay special attention to the use and effects of the models in integrated care settings and to balanced measurement of multiple performance dimensions. Next to this, more knowledge on the relationship between organizational development, context factors and improved performance is needed. Both models have possibilities for the further development of practical and evidence-based tools for improving integrated care.

References

View Abstract