OUP user menu

Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review

Mirella Minkman , Kees Ahaus , Robbert Huijsman
DOI: http://dx.doi.org/10.1093/intqhc/mzl071 90-104 First published online: 2 February 2007


Purpose Health care organizations have to improve their performance for multiple stakeholders and organize integrated care. To facilitate this, various integrated quality management models can be used. This article reviews the literature on the Malcolm Baldrige Quality Award (MBQA) criteria, the European Foundation Quality Management (EFQM) Excellence model (Excellence award models) and the Chronic Care Model. The focus is on the empirical evidence for improved performance by the implementation of interventions based on these models.

Data sources A systematic literature review from 1995 to May 2006 in the Pubmed, Cochrane, and ABI- databases was conducted.

Study selection After selection, 37 studies were included, 16 in the Excellence award model search and 21 in the Chronic Care Model search.

Data extraction and results of analysis Data were retrieved about the main intervention elements, study design, evidence level, setting and context factors, data collection and analysis, principal results and performance dimensions. No Excellence Award model studies with controlled designs were found. For the Chronic Care Model, one systematic review, one meta analysis and six controlled studies were included. Seventeen studies (2 in Excellence award model, 15 in Chronic Care Model) reported one or more significant results.

Conclusion There is some evidence that implementing interventions based on the ‘evidence-based developed’ Chronic Care Model may improve process or outcome performances. The evidence for performance improvement by interventions based on the ‘expert-based developed’ MBQA criteria and the EFQM Excellence model is more limited. Only a few studies include balanced measures on multiple performance dimensions. Considering the need for integrated care and chronic care improvement, the further development of these models for guiding improvements in integrated care settings and their specific context factors is suggested.

  • performance improvement
  • total quality management
  • chronic disease
  • organizational models
  • evidence-based


In order to prosper in today's dynamic health care systems, organizations such as hospitals must work effectively, be innovative and organize efficiently. A focus on multiple performance measures is needed to assess the quality level reached [1]. Not only patient outcome measures, but also worker's satisfaction and organizational and financial performance have to be managed and improved. This multidimensional approach by health care management corresponds with current definitions of the quality of care itself. The Institute of Medicine defines good care as safe, effective, timely, patient-centered and efficient. This definition also reflects multiple dimensions of Quality, including organizational aspects like a streamlined care process, good access and a financially healthy organization [2].

Another development is observed in the literature. The characteristics and boundaries of health care organizations are changing. ‘Patient-centered care’ focusses on the total needs of patients, not only on the services provided by one professional or organization. It is important to sustain seamless integrated care during the whole care process. For health care organizations, this requires ‘horizontal’ coordination, collaboration with other organizations and community partners or service integration. Partners in the care chain and the functioning of the care chain or care network as a whole contribute to the quality of care. The international literature on integrated care, disease management and development of care chains and networks addresses this issue [3].

To facilitate the improvement of health care quality and performance, a large range of quality management and organizational models has been developed [4, 5]. In this article, we focus on frequently used quality management models in health care: the European Foundation for Quality Management (EFQM) Excellence model and the Malcolm Baldrige Quality Award (MBQA) criteria on the one hand and the Chronic Care Model on the other. We selected these integrated quality management models on the basis of multiple criteria. First, these models all consist of multiple ‘enablers’ of good quality care (for instance, leadership or delivery system design). Enablers cover the processes, structure and mean values of an organization [6]. Second, these models focus on multiple performance dimensions for multiple stakeholders (for instance, organizational performance, worker satisfaction). Lastly, they assume dynamic relationships between improved performance and implementation of interventions based on the models enablers [7, 8]. The EFQM/MBQA and the Chronic Care Model are frequently used as frameworks for local improvement or national collaborative improvement programs. In this article, we focus on the available empirical evidence for these models in respect of improving health care performance. The research question is: what empirical evidence is available for improved performance in health care settings by implementing interventions based on the enablers of the EFQM Excellence model/MBQA criteria or the Chronic Care Model?

The EFQM Excellence model and MBQA criteria

The EFQM Excellence model conceptualizes organizations by discerning enabler and performance elements as ingredients for striving toward excellence [6, 7]. The EFQM Excellence model shows many parallels with the assessment model of the MBQA and international quality award criteria [9]. Originally, these widespread quality management models were developed in the private sector and may be viewed as an operationalization of Total Quality Management philosophies. Whereas the MBQA criteria consist of seven elements (leadership, strategic planning, customer and market focus, measurement, analysis and knowledge management, human resource focus and, process management and results), the EFQM Excellence model consists of nine elements (leadership, policy and strategy, management of people, partnership and resources and processes, key performance results, and people, customer and society results). Both models have healthcare-specific versions and are used in all types of health care organizations, regardless of sector and size or maturity [6, 9]. They are integrated models that cover quality management as an integral part of all professional and management functions at all levels of an organization. A basic premise of the models is that enablers direct and drive performance; organizations with well-developed enablers will have excellent results [6, 9]. Because of their comparability, we will focus on the EFQM and MBQA models as one category.

The Chronic Care Model

The Chronic Care Model identifies the essential elements of a (local) health care system, which encourage high-quality chronic disease care. The model is based on evidence-based change concepts and responded to the need for a quality improvement model that fits the characteristics of chronic care. The model can be used for various chronic illnesses, health care settings and target populations. The Chronic Care Model has also been used as an improvement tool in multiple chronic care improvement collaboratives [10].

The Chronic Care Model describes six elements–the community, the health system within it and four elements within the health system: self-management support, delivery system design, decision support and clinical information systems. Similar to the EFQM Excellence model and the MBQA criteria, the Chronic Care Model focusses on multiple dimensions of performance and on multiple stakeholders. Successful implementation of interventions based on the six elements may result in productive interactions between informed and activated patients and prepared and proactive care teams and in better functional and clinical outcomes. An expanded model based on the Chronic Care Model has a number of extras relating to patient safety, staff development, cultural aspects, coordination and the six performance dimensions of Institute of Medicine (IOM) definition of quality [11].

Summarizing, the EFQM/MBQA model and the Chronic Care Model are both integrated quality models that are adopted by many health care organizations in order to direct effective interventions and improved performance. Each model consists of enabler elements and performance dimensions and assumes positive relationships between them. Although these models are commonly used in practice, less is known about empirical evidence concerning the effects of interventions, based on the elements of these models, on improved performance. This conclusion, regarding the EFQM/MBQA models was supported by Shortell et al. [12], Nabitz et al. [6] and others [5, 1315], who stated that although the EFQM/MBQA models have high face validity, there are only a few publications in the academic literature. The Chronic Care Model is based on evidence-based directions for each element, but extensive research on the effects of the model as a whole on improved performance remains limited and comes mainly from self-reported, uncontrolled studies [1618].

Øvretveit [19] argues that, when assessing the (evidence for) achieved results of quality improvements, the influence of context factors and the degree of context dependence of the interventions have to be considered. Conditions that are likely to influence results are the type of health care system, social values, health reform, the history of quality and the language and politics of quality. According to Øvretveit, conditional interaction is systematically obscured in randomized controlled trials.

Data sources and study selection

We searched the PubMed, Cochrane, and ABI/Inform databases from 1995 to April 2006 and the reference lists of relevant papers. Study selection was based on the following criteria. First, we focussed on studies with empirical data published in peer-reviewed journals. Further, we included only studies that used the model as a basis for implemented interventions and focussed on multiple or all elements. We conducted two searches: one on studies using the EFQM Excellence model and the MBQA criteria and national variants, and one on studies using the Chronic Care Model. Search terms were ‘Baldrige’, ‘EFQM’, ‘MBQA’, ‘excellence model’, ‘quality award’ and national variants like ‘Deming quality award’ in the first and ‘chronic care’ combined with ‘model’ and ‘chronic care model’ in the second. One author screened the initial search results (M.M.) and at least two authors screened the selected studies (M.M. and R.H. or K.A.). All of the authors independently assessed the evidence levels of selected studies; differences in interpretation were resolved by consensus. A specified evidence-level table based on Effective Practice and Organization of Care group (Review group of the cochrane collaboration [EPOC]) criteria was used (Table 1). Evidence levels range from systematic reviews (A1) and randomized trials (A2) to descriptive non-analytical studies of multiple projects (D1), single projects (D2) or literature reviews (D3). The criterion for significant change in all studies was set at p≤0.05.

View this table:
Table 1

Search results and evidence level classification

Data extraction and results

Data were retrieved regarding the study design and evidence level, setting (organization and country), domain model elements in intervention, data collection and analysis techniques, main results and context factors described. Of an initial total of 850 studies, 16 were included from the EFQM/MBQA search [6, 12, 13, 15, 2031] and of initial 686 studies, 21 were included from the Chronic Care Model search [10, 1618, 3248], (table 1).

EFQM and MBQA results

The characteristics of the EFQM/MBQA studies are reported in Table 2. Regarding the evidence levels, no A- or B-level studies were found. Eight of the 14 C-level studies reported data on more than one measurement point (Table 1). Ten studies used the EFQM Excellence model and five the MBQA criteria as a model for improvement. Twelve of the 16 studies were published in 1999–2002. Eleven studies were conducted in Europe, four in the USA and one in Korea. Study settings were mainly hospitals (eight studies) and/or primary or community care services (six studies). In three studies, the results were statistically tested; two of them reported one or more significant improvements [12, 15]. Six of the eight C1- and C3-level case studies reported improved outcomes, but none are confirmed by statistical analysis.

View this table:
Table 2

Reviewed EFQM or MBQA studies

The study by Goldstein and Schweikhart [15] in 220 US hospitals provided the strongest evidence: all the relationships between the MBQA categories and examined performance were statistically significant. They found the strongest relationship with staff and work system results. Health care, financial and market results were less well predicted by the MBQA criteria. Sanchez et al. [20] and Shirks et al. [21] measured the results of their EFQM and MBQA improvement programs over 4-year periods and found positive trends for process performance, but no significant improvements for any other performances. In three-quarter of the included studies, three or more apparent context factors such as characteristics of the health care system, social values or the history of quality assurance were discussed. The effects of these factors on performances are less well described. Shortell et al. [12] and Lee et al. [23] explicitly included statistical analyses on context factors. Lee concluded that scientific skills in decision making and the adoption of a quality information system were the most important contributing factors. Shortell found significant relations for a participative, flexible and risk-taking organizational culture. Larger hospitals experienced lower clinical efficiency due to more bureaucratic and hierarchical cultures that served as a barrier for quality improvement implementation.

Chronic Care Model results

The characteristics of the Chronic Care Model studies are reported in table 3. Regarding the evidence levels, one meta-analysis and one systematic review were found [16, 17], one randomized trial [18], five controlled studies [3236], and a variety of case studies and project reviews (see table 1). Eighteen studies reported interventions on four or more Chronic Care Model elements. Most studies were published in the period 2003–2006 (15 studies). Almost all studies were conducted in the USA. Study settings were often primary or community care settings (15 studies), hospitals (4 studies) and/or outpatient clinics (4 studies) or networks of combined services. Eighteen of 21 studies included diabetes patients, 5 included asthmatic patients, 3 cardiovascular patients and 2 depressed patients. Fifteen studies reported one or more statistically significant improvements. Six studies did not test their results statistically.

View this table:
Table 3

Reviewed Chronic Care Model studies

The strongest evidence was found in the meta-analysis of 112 studies by Tsai et al. [16]. Tsai et al. found evidence for significant improvements on process or outcome measures by implementing at least one Chronic Care Model element. The review by Bodenheimer et al. [17] also showed high percentages of studies with positive effects, especially for studies which included four elements or self-management interventions. Compared with normal care or interventions supported by professional education, Piatt et al. [18] found that the Chronic-Care-Model-based group performed significantly better on two diabetes clinical outcome measures and self-management monitoring. Similar to Tsai et al., the B-level studies by Benedetti et al. [32], Mangione et al. [34] and Schonlau et al. [35] reported mainly significant improvements on outcome (HbA1C, LDL. and so on) or process measures (peak-flow monitoring, clinical testing and so on) at operational level. Chumbler et al. [33] found no changes in performance, except for increased service-use in primary and ED care. Sperl-Hillen et al. [42] analysed whether each Chronic Care Model element contributed equally and found positive correlations for delivery system design and positive associations for self-management and clinical information systems. Feifer et al. [36] found decision support, self-management and delivery system design to be positively correlated with clinical performance. Improved fit with the Chronic Care Model was related to clinical performance in this study. The performance dimensions included in the A- and B-level studies were further analysed (Table 4). Almost all the studies measure clinical or efficiency results such as test outcomes, length of stay or numbers of clinical exams, whereas less attention is paid to financial or professional results (such as worker's satisfaction).

View this table:
Table 4

Included performance dimensions (A and B level studies)

Regarding context factors, one-third of the studies described three or more context factors, mostly characteristics of the health care system, setting and patient populations. Only a few studies discuss influences of context factors on performances measured. Landis et al. [40] concluded that the one site that clearly outperformed the other five in her study, had a strong organizational foundation of a quality improvement culture and strong physician leadership. Also, Bodenheimer et al. [17] concluded that visionary clinical leadership and financial conditions are needed for successful improvements in chronic care.

Discussion and conclusion

Our finding in this review was that there is weak evidence for improved performance by implementing interventions based on the EFQM or MBQA models elements in health care settings. No randomized or controlled studies were found. The small number of EFQM/MBQA studies is surprizing because these models are widespread and have been used for many years. For the Chronic Care Model, the studies used more solid designs and methods. Some evidence has been found that implementing interventions based on the Chronic Care Model improves performance, but the conclusions are all drawn in the US settings for specific patient groups. Considering the quality of the studies, the description of the implemented interventions was often limited. For the EFQM/MBQA studies, the data in the multiple case studies were not systematically measured over time, making statements on improved performance impossible. An explanation for these differences found may lie in the origin of the models and their use in practice. The EFQM/MBQA models are ‘experience-based’, whereas the Chronic Care Model is ‘evidence-based’. The data in the EFQM and MBQA studies were mainly gathered from improvement projects, instead of research projects designed for statistical testing. The Chronic Care Model studies focus merely on patient groups and clinical measures, which better ‘fit’ the more biomedically oriented scientific research paradigm of controlled (randomized) trials.

Although the models have different origins, some elements show similarities. Interventions on ‘clinical information systems’ in the Chronic Care Model correspond to interventions in the EFQM ‘resources’ element. Shifts in care processes or tasks of workers both fit in Chronic Care Model's ‘delivery system design’ and EFQM/MBQA's ‘processes’ and ‘people’. Although the Chronic Care Model pays attention to aspects such as leadership (within health care organization), these elements combined with health policy are more emphasized in the EFQM/MBQA models. On the other hand, the Chronic Care Model defines ‘self-management’ as a crucial element, whereas the EFQM/MBQA models do not. It would appear that the EFQM/MBQA models are mainly used as management tools, e.g. at strategic level, whereas the Chronic Care Model is mainly used as a tool to optimize care for a specific patient group at the more operational or process level. Comments made on the Chronic Care Model include the fact that aspects such as culture, leadership and a greater business focus are missing [37], whereas the EFQM/MBQA is sometimes said not to provide a sufficient ‘health care fit’ [6]. Regarding integrated care, the studies focussed merely on just one organization. In the study by Shortell et al. [49] regarding the impact of quality improvement on clinical practice, no studies focussing on the continuum of care were found. Some studies, however, addressed the need for integrated care and management of the total care process. Freer and Jackson [25] stressed the helpfulness of the MBQA program for integrating services, and Chumbler et al. [33] measured inpatient and outpatient clinic outcomes to stress the interrelatedness for diabetic patients. Although ‘the community’ enabler in the Chronic Care Model points out relationships with other (care and welfare) organizations, integrated care chains are not the domain subjects of study. With regard to the increasing numbers of chronically ill and the need for integrated care, further development of these models is required in terms of both their usefulness and their applicability to care chains.

No studies covered more than three performance dimensions. With the attention to costs and efficiency in current health, it is surprizing that only a few Chronic Care Model studies measured financial performances. Moreover, measures of worker satisfaction (the care team) and patient judgements are also often lacking. The assessment procedures used in the EFQM/MBQA studies include multiple performance dimensions, but information about the results on these dimensions is often not systematically reported.

The EFQM/MBQA studies paid more attention to the influence of context factors than the Chronic Care Model studies. As known from the literature, organizational characteristics such as culture and leadership and political developments affect the results [50,51]. The included studies conform this by naming these factors as influencing factors. For the EFQM/MBQA models, there are also studies in other sectors available. These studies show mixed but mostly positive results. Kaynak [52] found 18 studies on the relationship between total quality management implementation and improved performance, all of which showed one or more positive effects. A recent controlled study by Boulter et al. [53] found evidence that the 120 award-winning companies experienced a greater increase in shared values, capital expenditure, growth in assets and reduction in costs over both short and long periods of time. Summarized, the results indicate that effective implementation of the EFQM model makes good economic sense in non-health care settings. Another interesting issue is how organizations develop in increasing performance. Both the EFQM/MBQA models and the Chronic Care Model have five ‘development phases’ that suggest pathways for growth [6, 8, 9]. The assumption is that improved performance is related to growth in the developmental phase. In this perspective, insight into the relationship between interventions, organizational development and performance is interesting, but is yet hardly a subject of research.

Our research contains several limitations. There is a lack of insight as to which models' elements contribute the most to performance and to which confounding and context variables are present. Furthermore, the effects of collaborative improvement-program interventions are not separated out when assessing the results. Another limitation concerns the methodological quality of the studies. The interventions differ from one study to another, meaning that generalizations are hazardous and that the findings are not reproducible for larger populations or other organizations. The absence of publication bias cannot be guaranteed. Also, we conducted a search for studies in which reference was made to the use of the model, whereas other studies that might have implemented comparable interventions were not included. Finally, Grol [54] and Øvretveit and Gustafson [55] stress the complexity of solid research designs given the large number of possible interacting dimensions, making it difficult to prove firm relationships. The richness of interventions, confounding variables and effects of organizational development mean that the evidence for relationships between using the model as a whole and performance largely remains a grey area.

Despite these caveats, this review does support the conclusion that interventions based on the Chronic Care Model may improve process and outcome measures in some situations. For the EFQM/MBQA, the evidence found is less strong. Future research should pay special attention to the use and effects of the models in integrated care settings and to balanced measurement of multiple performance dimensions. Next to this, more knowledge on the relationship between organizational development, context factors and improved performance is needed. Both models have possibilities for the further development of practical and evidence-based tools for improving integrated care.


View Abstract