OUP user menu

Selecting indicators for the quality of diabetes care at the health systems level in OECD countries

Antonio Nicolucci, Sheldon Greenfield, Soeren Mattke
DOI: http://dx.doi.org/10.1093/intqhc/mzl023 26-30 First published online: 5 September 2006

Abstract

Purpose. In the context of the Organization for Economic Cooperation and Development (OECD) Quality Indicators Project, a set of quality indicators for diabetes care was developed, to be used for benchmarking the performance of health care systems.

Background. Diabetes complications markedly reduce quality and length of life and are also responsible for enormous health care costs. A large body of evidence has shown that several effective treatments and practices may substantially reduce this burden. However, a marked variability has been documented in preventive and therapeutic approaches, thus suggesting that the level of diabetes care currently delivered may not produce the possible health-related gains.

Methods. Existing quality indicators have been reviewed, with particular attention to the work done by the National Diabetes Quality Improvement Alliance ( NDQIA) in the US. All the measures identified were evaluated for their importance, scientific soundness, and feasibility. In addition, the panel members selected new distal outcome measures. These measures are currently not used in provider comparisons, but they could reveal valuable insight into the differential performance of health systems.

Results. Four process and two proximal outcome measures were selected among those endorsed by the NDQIA. In addition, three new long-term outcome measures have been proposed to gain insight into whether and to what degree differences in the processes and intermediate outcomes that are captured by the established measures translate into better outcomes for patients.

Conclusions. The measures selected can contribute to policymakers’ and researchers’ understanding of differences in the quality of diabetes care between health systems. Further work is required to assess the availability of reliable and comparable data across OECD countries.

  • diabetes mellitus
  • quality of care
  • quality indicators
  • process easures
  • outcome measures

Diabetes mellitus (DM) has become one of the most important public health challenges for the 21st century. Over 150 million adults are affected by DM worldwide, and this number is expected to double in the next 25 years [1,2]. The rate of increase in diabetes incidence will be dramatically higher in developing countries: between 1995 and 2025, the number of individuals with diabetes is expected to increase by 170% in the developing world, compared with 42% in developed nations [1]. Therefore, by the year 2025 over 75% of the people with diabetes will reside in developing countries. The expected epidemic of diabetes will be paralleled by a skyrocketing increase in the incidence of its chronic complications responsible for the huge premature morbidity and mortality associated with the disease. Diabetes is the leading cause of blindness in industrialized countries, and it represents the most frequent cause of visual handicap in people aged <60 years [3]. About one-fifth of diabetic patients develop end-stage renal disease (ESRD) during their lifetime. Diabetes is currently the major cause of renal failure requiring dialysis treatment or renal transplantation in the US, Europe, and Japan, and its incidence is rapidly increasing in all countries, leading to the definition of diabetic nephropathy as ‘a medical catastrophe of worldwide dimensions’ [4]. Among the more disabling diabetes complications are foot complications. Non-traumatic amputations are 15 times more frequent in diabetic patients than in the general population. Approximately half of the 110,000 lower-extremity amputations performed in the US each year are on individuals with diabetes; in addition, 30–50% of first-time amputees will require additional amputations within 1–3 years, and 50% will die within 5 years of an initial major amputation [5].

Macrovascular complications represent the leading cause of morbidity, mortality, and resource consumption in type 2 diabetes and their burden is expected to grow in the next years because of the increasing incidence of the disease worldwide.

Individuals with type 2 diabetes have a two- to four-fold increased risk of cardiovascular disease (CVD) compared with non-diabetic subjects, and CVD mortality rates are 1.5–4.5 times higher than in the general population [6]. The increase in the incidence of coronary events is greater for more severe clinical outcomes, such as myocardial infarction and sudden death, than for less serious outcomes, such as angina pectoris [6]. Furthermore, the case fatality rate after a myocardial infarction among subjects with diabetes is higher than that for patients without diabetes; after a first cardiac event, 50% of patients with diabetes die within 1 year, and half of those who die do so before they reach the hospital (sudden deaths) [7 ].

Diabetes is responsible for premature CVD deaths; the adjusted relative hazard of CVD mortality is in fact five times higher in the age range 30–49 and 40–59 years for diabetic individuals as opposed to non-diabetic ones [8,9]. Median life expectancy is estimated to be 8 years lower for diabetic adults aged 55–64 years [10]. In recent years, a decline in heart disease mortality in the general US population has been documented, mainly due to reduction in cardiovascular risk factors and improvement in treatment of heart disease. Nevertheless, such a positive trend was not documented for diabetic patients, thus suggesting that these changes may have been less effective for people with diabetes, particularly women [11].

Diabetes complications not only markedly reduce quality and length of life but are also responsible for enormous health care costs. Fortunately, a large body of evidence has clearly shown that several effective treatments and practices may substantially reduce this burden [12]. However, a marked variability has been documented in the application of preventive and therapeutic strategies, thus suggesting that the level of diabetes care currently delivered may not produce the possible health-related gains.

Given these premises, diabetes has been identified by the Organization for Economic Cooperation and Development (OECD) as having priority for the development of quality indicators.

Methods

Selection of quality indicators

Much progress has been made in the development, specification, and field-testing of measures for diabetes in the US and European countries [13–15]. Much of the work on standardized measures for the quality in this area originated in the US, where the development of measures began with the Diabetes Quality Improvement Program [13], a physician-driven quality improvement program, which has evolved into the National Diabetes Quality Improvement Alliance (NDQIA), comprising all of the organizations in the US that are concerned about the care of patients with diabetes [16]. Quality performance measures have been identified by reviewing the literature, consulting with experts, and surveying organizations already using diabetes performance measures. Required criteria included (i) credible evidence linking process measures to important clinical outcomes and modifiability of the clinical outcome measures (i.e. they could be improved) by the efforts and interventions of health care systems; (ii) feasibility (i.e. whether the measure could be collected accurately, reliably, and at a reasonable cost); and (iii) variability across health care settings, so that there would be opportunity for improvement. The goal was to develop a set of measures that was comprehensive yet parsimonious to minimize the burden of data collection. The recently revised list of nine consensus measures, which have also gained widespread acceptance internationally, contains six indicators for care processes and three for outcomes of care (Table 1).

View this table:
Table 1

Quality indicators for diabetes care identified by the National Diabetes Quality Improvement Alliance [12]

Process measures
    Percentage of patients with one or more HbA1c tests annually
    Percentage of patients with at least one LDL cholesterol test annually
    Percentage of patients with at least one test for microalbuminuria during the measurement year or who had evidence of medical attention for existing nephropathy
    Percentage of patients who received a dilated eye examination or evaluation of retinal photography by an ophthalmologist or optometrist during the current year or during the prior year if the patient is at low risk of retinopathy
    Percentage of patients receiving at least one foot examination annually
    Percentage of patients whose smoking status was ascertained and documented annually
Outcome measures
    Percentage of patients with most recent HbA1c level >9.0% ( poor control)
    Percentage of patients with most recent LDL cholesterol <130 mg/dl
    Percentage of patients with most recent blood pressure <140/90 mmHg
  • HbA1c, hemoglobin A1c; LDL, low-density lipoprotein.

All measures use the number of clinically diagnosed diabetic patients as their denominator. For the outcome measures, it is commonly stipulated that the patients who were not tested have failed the standard, to avoid creating incentives for undertesting challenging cases. Furthermore, recent data clearly suggest that missing information on intermediate outcome measures, such as hemoglobin A1c (HbA1c) and low-density lipoprotein (LDL) cholesterol levels, represent a strong, independent predictor of long-term cardiovascular outcomes [17 ].

Given the amount of work already done, the OECD diabetes panel, composed of the authors of this article, simply reviewed the existing measures and evaluated to what extent each of them could be used for benchmarking of health systems by aggregating them to the national level. In addition, the panel members selected new distal outcome measures. Because of attribution problems, these measures are currently not used in provider comparisons, but they could reveal valuable insight into the differential performance of health systems.

Results

Although the importance and scientific soundness of the nine measures identified by the NDQIA go uncontested and recent studies indicate that these measures could be used for health system comparisons [18,19], there are some challenges to their feasibility for international comparisons. The first four process measures (HbA1c tests, LDL tests, microalbuminuria tests, and eye examinations) stand the best chance of being implementable, as they refer to distinct clinical services and the delivery of those services is usually well documented in administrative records, such as billing data. The four measures can thus be recommended for use without reservations. Reliably measuring the remaining two process measures on the international level may be challenging, however. A foot examination is not a well-defined clinical test and may not be billed as an individual service in many countries, hampering the ability to extract the necessary information from administrative data. Documentation of smoking status is also not done in administrative data. Thus, both measures are likely to require dedicated data collection, such as medical record review, and substantial efforts would be needed to ascertain that data could be collected in a comparable fashion across countries. A more widespread use of electronic medical records, which can be expected in many OECD countries, may improve the situation in the long run.

As to the outcome measures, whereas administrative records document reliably whether a test has been conducted, they do not record the test result. It is therefore not possible to construct the measures for HbA1c and LDL control from such data sources. However, electronic laboratory data, which are becoming more common, and national reporting requirements could provide the necessary information. Measuring and reporting blood pressure control in a comparable fashion would be more challenging. The protocols for measuring and reporting of blood pressure would have to be standardized across countries, and data collection would require dedicated reporting or electronic medical records, whose implementation lags substantially the implementation of electronic laboratory data systems. In addition, some data suggest that blood pressure control may be subject to ceiling effects [15] and therefore not be useful to discriminate sufficiently the performance of health systems. Further research would thus be necessary before implementation of this indicator.

After these considerations, the expert panel identified the set of indicators reported in Table 2.

View this table:
Table 2

Measures proposed for use in health system comparisons

AreaIndicator name
Processes of diabetes careAnnual HbA1c testing
Annual LDL cholesterol testing
Annual screening for nephropathy
Annual eye examination
Proximal outcomesHbA1c control
LDL cholesterol control
Distal outcomesLower-extremity amputation rates
Kidney disease in persons with diabetes
Cardiovascular mortality in patients with diabetes
  • HbA1c, hemoglobin A1c; LDL, low-density lipoprotein.

New measures proposed for use in health system comparisons

It should be emphasized that the recommended set of measures covers mainly proximal concepts, i.e. quality characteristics under the immediate control of providers of medical care. For policy purposes, those measures should be combined with indicators that reflect more distal concepts, i.e. the long-term outcomes for the chronically ill patients. Although such indicators could never be used to compare individual providers, because they represent the end product of numerous little steps that cannot be attributed to a single provider, they are very useful for overall health system comparisons. Several such measures are currently discussed in the literature for possible use at the health system level, such as rates of lower-extremity amputations and the incidence of kidney disease. In addition, a novel outcome measure, CVD mortality in patients of certain ages with diabetes, could be considered. A detailed description of the importance and scientific soundness of these indicators can be found in the OECD Health Technical Paper No.15 [20].

As for major amputations, this indicator is typically derived from hospital discharge information and the ability to construct it reliably for international comparisons depends on the comparability of coding and reporting practices across countries. Amputation rates, or the numerator, should be ascertainable in a reliable fashion in administrative data, as is done currently in the US by the Centers for Disease Control and Prevention, because such major procedures usually influence hospital payments and are thus reliably reported. But it may be difficult to reliably identify the diabetic population or the denominator, because diabetes may only be recorded as comorbid condition rather than the primary reason for admission and coding of such secondary diagnoses may vary across countries. Because of the importance of this complications and plausibility of the concept behind the indicators, this measure has great potential. But it needs to be further studied before adopting it for international comparisons. If implemented, the measure should be confined to major amputations, because minor amputations of the toes and feet may be done to prevent major amputations and could conceivably be the result of improved system and patient surveillance that leads to earlier detection and treatment of foot lesions.

The evaluation of the incidence of ESRD also poses some methodological problems. Although rates of dialysis initiation and of renal transplantation are readily available in Europe and the US, this information only approximates the true incidence and prevalence of ESRD, because the definition of ESRD and the criteria for dialysis initiation might differ across countries. Ascertainment of the cases could be achieved more precisely by standardized laboratory tests, such as blood creatinine levels or glomerular filtration rate (GFR). However, consensus would have to be reached on the threshold level, and creatinine and other variables would have to be universally required and recorded to estimate GFR.

Finally, monitoring CVD mortality, particularly in the age band 35–50 years, could represent an important indicator of the overall care delivered to diabetic patients, to be used for international comparisons. Although conceptually appealing, the implementation of this indicator would be challenging. Mortality data, as the numerator for the indicator, are readily available in almost all countries, and causes of death are usually reported using internationally standardized coding systems, such as the ICD-9 or ICD-10 systems. However, identifying the diabetic population as the denominator for the indicator is difficult. For example, it will depend on national reporting requirements and conventions whether and to what degree underlying diseases that led to the actual cause of death are recorded on death certificates. Alternatively, diabetic patients could be identified from drug prescription databases, and mortality data (with specific cause of death) then linked using unique patient identifiers. This method has already been successfully applied [21], but privacy concerns and operational complexity may stand in the way of widespread adoption. Another data source could be national disease registries, which are now commonly available for cancer diagnoses, that prospectively track all patients or representative samples of patients with chronic diseases.

To summarize, all three discussed new outcome measures have great potential to contribute to policymakers’ and researchers’ understanding of differences in the quality of care between health systems, in particular because they represent the long-term outcomes of diabetes care.

Discussion

Diabetes care can be considered as an area in which there is widespread consensus on good practice patterns and international convergence on measures for processes and outcomes of care.

From the revision of the existing measures, a robust and comprehensive set of indicators for the quality of diabetes care has emerged on the international level that could be used to benchmark health systems. These indicators strictly reflect the monitoring of diabetes and its complications, as well as the short-term results in metabolic and lipid control. Some operational issues need to be resolved before implementation, such as the availability of comparable data for some indicators, and international consensus will have to be found on the periodicity of the process indicators and the threshold levels of the outcome indicators.

As for the new distal outcome measures proposed, they would help to gain insight into whether and to what degree differences in the processes and intermediate outcomes that are captured by the established measures translate into better outcomes for patients. However, these measures are not yet universally accepted so that further research remains necessary before they could be recommended for benchmarking the performance of health care systems. A particular challenge will be to assure that the necessary data can be collected in a reliable and comparable fashion across OECD countries. The next phase of the project will thus be focused on the operational feasibility of the proposed indicators through a survey of data availability in OECD countries. Further work is also needed to ensure that the set of indicators selected fully reflects the quality of diabetes care provided at the national level and to evaluate whether additional indicators need to be incorporated.

Footnotes

  • This article describes the key recommendations made by the authors to the OECD Health Care Quality Indicators Project. It reflects the opinion of the authors and not an official position of the OECD, its member countries or institutions participating in the project. The full report can be found at http://www.oecd.org/dataoecd/28/34/33865546.pdf.

References

View Abstract