OUP user menu

★ Editor's choice ★

Measuring chronic care delivery: patient experiences and clinical performance

Thomas D. Sequist, Ted Von Glahn, Angela Li, William H. Rogers, Dana Gelb Safran
DOI: http://dx.doi.org/10.1093/intqhc/mzs018 206-213 First published online: 6 April 2012

Abstract

Objective To assess the relationship between clinical care metrics and patient experiences of care among patients with chronic disease.

Design Cross-sectional survey and clinical performance data.

Setting Eighty-nine medical groups across California caring for patients with chronic disease.

Participants Using patient surveys, we identified 51 129 patients with a chronic disease.

Main Outcome Measures Using patient surveys, we produced five composite measures of patient experiences of care and self-management support (scale 0–100). Using Health Plan Employer Data and Information Set data, we analyzed care for asthma, diabetes and cardiovascular disease, producing one composite summarizing clinical processes of care and one composite summarizing outcomes of care. We calculated adjusted Spearman's correlation coefficients to assess the relationship between patient experiences of care, clinical processes and clinical outcomes.

Results Clinical performance was higher for process measures compared with outcomes measures, ranging from 91% for appropriate asthma medication use to 59% for controlling low-density lipoprotein cholesterol in the presence of diabetes. Performance on patient experiences of care measures was the highest for the quality of clinical interactions (88.5) and the lowest for delivery of self-management support (68.8). Three of the 10 patient experience–clinical performance composite correlations were statistically significant. These three correlations involved composites summarizing integration of care and quality of clinical interactions, and ranged from a low of 0.30 to a high of 0.39.

Conclusions Chronic care delivery is variable across diseases and domains of care. Improving care integration processes and communication between health-care providers and their patients may lead to improved clinical outcomes.

  • chronic disease
  • quality measurement
  • patient-centered care
  • quality of care
  • quality improvement

Introduction

The epidemic of chronic disease has important implications for improving the quality of care in the US health-care system. Nearly 50% of the US population, over 125 million Americans, is affected by a chronic condition [1], and 20% suffer from more than one chronic disease [2]. Chronic diseases are responsible for 70% of deaths in the USA and over 75% of health-care costs [3], and it is projected that the burdens associated with chronic conditions will continue to grow [4].

Unfortunately, the majority of physicians and patients report that it is difficult to obtain high quality care for chronic illnesses [5]. While some gains in clinical quality for chronic disease care have been achieved, there is still substantial room for improvement [6], and progress in patient experiences of care may be lagging behind [7]. Improving delivery of chronic disease care will involve increasing the effectiveness of care delivery, as well as engaging patients in the management of their condition [8].

As physician groups become increasingly engaged in clinical programs to improve chronic disease care [9], comprehensive performance measurement represents an important component of an overall improvement strategy. While prior work has explored the interaction between patient experiences of care and clinical outcomes [1012], there are little data to help physician groups understand these interactions in the setting of more complex chronic care management.

Using California statewide data, our study had three goals: (i) to report on clinical quality and patient experiences of care among patients with chronic conditions; (ii) to evaluate the association between measures of clinical processes and clinical outcomes for individual chronic conditions; and (iii) to evaluate the association between measures of clinical performance and measures of patient-centered care.

Methods

Study setting

We used data collected from medical groups participating in a statewide performance measurement initiative in California during 2007. These medical groups, a mix of free-standing physician practices and independent practice associations, provide healthcare for ∼90% of the state's insured population, and range in size from 20 physicians to several thousand physicians.

Patient survey eligibility

We combined data collected from two sampling frames in 2007, a medical group level survey and an individual physician level survey (Fig. 1). The medical group level survey sample identified all patients aged 18 years and older who had a visit with either a primary care or a specialist physician during the 10-month period from January to October 2007. A random sample of 900 patients per medical group was selected and linked to a single physician (primary care or specialist) based on their visit history. The survey was administered in a three-stage process, with an initial mailing, a follow-up mailing and a final attempted telephone interview. The overall response rate was 37%.

Figure 1

Enrollment of patients being treated for chronic conditions. Patients with chronic disease were identified using surveys and were included in the study based on receiving care from a primary care physician or specialist within one of the medical groups self-reporting clinical performance data.

The individual physician level survey sample was derived from a subset of the medical groups who volunteered. Patients were identified based on the presence of an office visit with an individual physician during the 10-month period from January to October 2007. A random sample of 100 patients per physician was then selected. These patients were exclusive of the 900 patients sampled from that site for the medical group level survey. The survey was administered in a two-stage process, with an initial mailing followed by a second mailing to non-responders. The overall response rate was 34%.

Survey sample sizes were chosen with the goal of achieving a minimal threshold reliability for provider-level comparisons of 0.70 [13]. The response rates achieved are consistent with those achieved in other statewide surveys, where non-response analyses have not demonstrated any biases or challenges to the integrity of provider-level analyses of patient experience measures [13].

Survey instrument

Patient experiences of care were measured based on the CAHPS® Clinician and Group Survey endorsed by the National Quality Forum [14]. Self-management support was assessed using a validated composite measure based on five individual survey items (Table 1) [15].

View this table:
Table 1

Measures of patients' experience

Composite measuresIndividual item content
Quality of clinical interactions (α = 0.94)In the last 12 months, how often did this doctor explain things in a way that was easy to understand?
In the last 12 months, how often did this doctor listen carefully to you?
In the last 12 months, how often did your personal doctor give you easy to understand instructions about what to do to take care of the health problems or concerns that were bothering you?
In the last 12 months, how often did this doctor show respect for what you had to say?
In the last 12 months, how often did this doctor spend enough time with you?
In the last 12 months, how often did this doctor seem to know the important information about your medical history?
Integration of care (α = 0.63)In the last 12 months, how often did this doctor seem informed and up-to-date about the care you got from specialist doctors?
In the last 12 months, when this doctor sent you for a blood test, X-ray or other test, how often did someone from the doctor's office follow up to give you the test results?
Office staff (α = 0.88)In the last 12 months, how often were clerks and receptionists at this doctor's office as helpful as you thought they should be?
In the last 12 months, how often did clerks and receptionists at this doctor's office treat you with courtesy and respect?
Organizational access (α = 0.81)In the last 12 months, when you called this doctor's office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed it?
In the last 12 months, when you made an appointment for a check-up or routine care with this doctor, how often did you get an appointment as soon as you needed it?
In the last 12 months, when you called this doctor's office with a medical question during regular office hours, how often did you get an answer to your question that same day?
In the last 12 months, when you called this doctor's office after regular office hours, how often did you get the help or advice you needed?
Wait time includes time spent in the waiting room and exam room. In the last 12 months, how often did your visits at this doctor's office start within 15 min of your appointment?
Self-management support (α = 0.73)In the last 12 months, did this doctor or any of these other providers give you clear instructions about how to manage this health condition?
In the last 12 months, did this doctor or any of these other providers work with you to set personal goals for managing this health condition?
In the last 12 months, did this doctor or any of these other providers help you figure out ways to overcome the things in your daily life that got in the way of managing this health condition?
In the last 12 months, how often did this doctor seem informed and up-to-date about care you received for this condition from other providers?
In the last 12 months, did this doctor or any of these other providers ask you whether this health condition makes it hard to do the things you need to do each day (for example, at home or at work)?
  • α, Cronbach's α for composite measure.

We identified patients receiving care for a chronic disease by asking the survey question ‘In the last 12 months, did you have any health problems or conditions for which you took medicine or got care for 3 months or longer?’. Patients also identified specific chronic conditions from a pre-specified list of diseases, and were allowed to select the presence of more than one chronic condition.

Clinical performance measures

As part of the statewide performance measurement program, clinical data were voluntarily submitted by 89 medical groups using a combination of administrative and medical record data. These 89 medical groups represent 51% of all groups statewide (Fig. 1). The remaining groups were not able to submit data due to the lack of information systems infrastructure to facilitate large-scale data extraction on clinical quality measures. The submitted data are processed by the National Committee for Quality Assurance and performance measures are calculated using standard specifications from the Health Plan Employer Data and Information Set (HEDIS). We evaluated measures for the management of asthma, cardiovascular disease and diabetes (Table 2).

View this table:
Table 2

Measures of clinical quality

Composite measureIndividual item content
Processes of care (α= 0.86)Appropriate asthma medications
 DenominatorAdults 18–75 years old diagnosed with persistent asthma
 NumeratorProvided at least prescription for inhaled steroid, nedocromil, cromolyn sodium, leukotriene modifier or methylxanthine during the measurement year
Cardiovascular disease: LDL cholesterol testing
 DenominatorAdults 18–75 years old discharged alive for acute myocardial infarction, coronary artery bypass graft or percutaneous transluminal coronary angioplasty, or who had a diagnosis of ischemic vascular disease
 NumeratorLDL cholesterol test within 1 year
Diabetes care: HbA1c testing
Diabetes care: LDL cholesterol testing
Diabetes care: Nephropathy screening
 DenominatorAdults 18–75 years with diabetes diagnosis
 NumeratorsHbA1c exam within the prior year
LDL cholesterol exam within the prior year
Urine microalbumin exam within the prior year
Outcomes of care (α= 0.91)Cardiovascular disease: LDL cholesterol testing
 DenominatorAdults 18–75 years old meeting criteria for cardiovascular disease and having an LDL cholesterol test within 1 year
 NumeratorLDL cholesterol result <130 mg/dl
Diabetes care: LDL cholesterol control
Diabetes care: HbA1c control
 DenominatorAdults 18–75 years old with diabetes and lab test within 1 year
 NumeratorsLDL cholesterol result <130 mg/dl
HbA1c < 9%
  • α, Cronbach's α for composite measure.

Composite measure creation

We created four composite measures of patient experience, one composite measure of disease self-management support and two composite measures of clinical quality. The patient experience composites have been validated in prior studies and included one measure of the quality of clinical interactions, three measures of organizational features of care (integration of care, office staff and organizational access) [13, 16] and one measure of self-management support (Table 1) [15]. Composite measures of clinical quality were developed for diabetes, cardiovascular disease and asthma management by separating process of care measures from clinical outcomes (Table 2) [17]. All composite measures demonstrated good internal validity, with Cronbach's α scores exceeding a minimum threshold of 0.70 with the exception of the integration of care, which demonstrated a score of 0.63 (Tables 1 and 2).

Numeric composite scores for clinical quality measures and patient experience measures were calculated using the adjusted half-scale rule to produce ratings on a scale from 0 to 100, with higher scores representing either better patient experiences or superior clinical quality. This involves first transforming the individual item to a 0–100 scale [18]. HEDIS scores are already ranked as such, while patient survey responses require conversion (e.g. a five-point Likert response item would generate a score of 0, 25, 50, 75 or 100).

Correlation analyses

We calculated non-parametric Spearman's correlation coefficients between individual process of care measures and their relevant outcome measure at the medical group level, including those between: (i) annual HbA1c monitoring and achieving HbA1c control (<9%) for patients with diabetes, (ii) annual low-density lipoprotein (LDL) cholesterol monitoring and achieving LDL cholesterol control (<130 mg/dl) for patient with diabetes, and (iii) annual LDL cholesterol monitoring and achieving LDL cholesterol control (<130 mg/dl) for patients with cardiovascular disease.

We next calculated correlation coefficients at the medical group level between the two composite measures of clinical quality (process composite and outcome composite) and the five composite measures of patient experiences, for a total of 10 correlations. While the clinical composite scores were calculated based on patients with specific chronic diseases, the correlations with patient experiences were performed among all patients screening in for chronic disease, regardless of the specific condition.

For all correlation coefficients, we calculated the convergence estimates, or adjusted correlation coefficients, to represent the true correlation that would be observed between the two composite scores in the absence of measurement error associated with creating the composites. The convergence estimates were calculated using the Spearman–Brown reliability coefficients for the patient experience composite (α1) and the clinical quality composite (α2), along with the Spearman correlation coefficient (ρ) in the following manner: convergence = Embedded Image.

This study protocol was approved by the Human Studies Committee at Tufts Medical School. All analyses were carried out using the STATA statistical package, version 9.0.

Results

We identified 51 129 patients with a chronic condition being treated across 89 medical groups (Fig. 1). These patients reported the presence of hypertension (34%), diabetes (16%), arthritis or joint disease (15%), back pain (11%), depression (11%), asthma (6%), cancer (5%), coronary artery disease (4%), congestive heart failure (2%), other heart disease (4%) and pregnancy/pre-natal condition (3%). Among patient-reported experiences of care, the performance was the highest for the quality of clinical interactions (mean score 88.5) and the lowest for the delivery of self-management support (mean score 68.8, Table 3). The variation in these patient-reported measures across medical groups, as measured by the standard deviation of the group effect, was generally smaller than that observed for the clinical process and outcomes measures.

View this table:
Table 3

Chronic care performance across participating medical groups

Patients, naMedical groups, nAverage rateStandard deviationb
Patient experiences of care
 Quality of clinical interactions51 1298988.53.2
 Integration of care51 1298976.84.7
 Office staff51 1298985.13.5
 Organizational access51 1298975.14.6
 Self-management support51 1298968.83.5
Clinical performance
 Appropriate asthma medications800.913.7
 LDL cholesterol testing (cardiac)770.847.9
 LDL cholesterol testing (diabetes)890.786.8
 HbA1c testing (diabetes)890.837.0
 Nephropathy testing (diabetes)890.768.9
 Process of care composite890.836.6
 LDL cholesterol control (cardiac)760.6812.0
 LDL cholesterol control (diabetes)890.5912.7
 HBA1c control (diabetes)890.6715.0
 Outcomes of care composite890.6413.4
  • aIndividual patient sample sizes not available for clinical performance metrics as these measures represented summaries at the level of individual medical groups.

  • bStandard deviation across medical groups.

Clinical performance was higher for the process measures of care compared with outcomes measures of care, ranging from a high of 91% for the process measure focused on the use of appropriate asthma medications to a low of 59% for the outcome measure of controlling LDL cholesterol in the presence of diabetes. Variation in clinical performance among medical groups was higher for outcomes measures compared with the process measures of care, with standard deviations for outcomes measures nearly twice the magnitude of those for process measures (Table 3).

Spearman's correlations between processes of care and outcomes of care for diabetes and cardiovascular disease were relatively high, including annual HbA1c monitoring–HbA1c control (0.59, P= 0.03); and annual LDL cholesterol monitoring–LDL cholesterol control for diabetes (0.60, P= 0.02) and cardiovascular disease (0.64, P= 0.01).

Among the 10 adjusted correlation coefficients between patient experience composites and clinical performance composites, 3 (30%) were statistically significant (Table 4). The integration of care composite demonstrated a significant positive correlation with both the clinical process of care and the clinical outcomes of care composites, while the quality of clinical interactions composite demonstrated a significant positive correlation with the clinical processes of care composite.

View this table:
Table 4

Adjusted correlations between patient experiences of care and clinical performance

Patient-reported experiencesa
Quality of clinical interactionsIntegration of careOffice staffOrganizational accessSelf-management support
Clinical processes of care
 Process of care composite0.38*0.30*0.330.270.29
Clinical outcomes of care
 Outcomes of care composite0.340.39*0.300.300.16
  • aValues represent convergence estimates (adjusted Spearman's correlation coefficients) between clinical quality composite measure and patient experience composite measure.

  • *P < 0.05 for adjusted correlation coefficient. Statistically significant correlations are shown in bold.

Discussion

Improving the quality of chronic disease care will involve a commitment to providing clinically effective, patient-centered care. In this statewide evaluation of chronic disease care, we identified substantial variation among medical groups on measures of clinical performance and patient experiences of care. Performance on the process measures of clinical care was uniformly higher than that on outcomes measures of care, and delivery of self-management support lagged behind other standard measures of patient experiences of care. The correlations between clinical processes of care and outcomes of care were significant and of larger magnitude than those observed between clinical measures and patient experiences of care. These latter correlations were all positive, with the strongest relationships noted for the quality of clinical interactions and integration of care composites.

Our finding that clinical performance on outcomes measures of care is lower than performance on process measures of care is consistent with prior analyses of quality of care [6]. The relatively strong correlations we identified between processes of care and outcomes of care at the medical group level highlight the importance of at least annual monitoring of important disease markers. Increasing patient engagement and the provision of self-management support is often cited as a potential solution to improving clinical outcomes for patients with chronic diseases [12]. However, we found no correlations between self-management support and clinical performance. Rather, the quality of clinical interactions and the integration of care were significantly correlated with clinical processes and outcomes.

While our analyses of correlations between clinical performance and patient experiences of care do not definitively prove causality, our findings suggest several important avenues for medical groups to focus on as part of a strategy to improve the quality of chronic disease management. First, establishing a systematic approach to increasing performance on process measures, such as maintaining disease registries and conducting population outreach, is critical. Secondly, providers should focus on ensuring that care plans for patients with chronic disease are effectively coordinated across multiple providers. This latter point is particularly relevant, given the fact that many patients with chronic disease are cared for simultaneously by a combination of physicians [19]. Medical homes and accountable care organizations represent attractive models of how to foster better integration of care within the health-care delivery system [20, 21]. Finally, better approaches to engaging patients in development and completion of a chronic disease care plan are needed, as self-management support scores were the lowest of the patient experiences of care measures we measured.

Electronic health records and other forms of information technology can play an important role in chronic care management. More widespread use of shared electronic health records can enable better access to relevant health information across multiple providers, fostering more efficient communication between providers, and ultimately better integration of care [22]. Other forms of information technology can be used to help providers communicate more efficiently with their patients between routine office visits, including personal health records or e-mail [23]. However, in order to truly improve the value of communication between patients and providers, the use of these electronic tools will need to be accompanied by additional training programs demonstrated to help physicians communicate in the context of the office visit [24].

Despite a few significant correlations between patient experiences of care and clinical performance, the majority of the correlations we analyzed were not significant, though all were positive. This is consistent with prior research that has found only modest correlations between clinical measures and patient experiences, and no evidence of conflict between these two goals [11, 18]. The absence of strong and consistent correlations does not undermine the value of measures of patient experiences. Rather, our findings indicate that clinical care delivery and patient experiences, including self-management support, may often represent distinct aspects of care delivery and potentially parallel quality improvement strategies.

Some research suggests that better patient experiences might enhance adherence and ultimately improve patient outcomes [10]. Our null findings may be a limitation of the cross-sectional study design, as we are not able to comment on whether longitudinal changes in measures of patient-centered care are associated with subsequent changes in clinical outcomes, or vice versa. Our findings are also challenged by the difficulty of correlating patient reports of care among patients with any chronic disease to clinical outcomes that were disease-specific.

Our analyses are strengthened by the inclusion of a large number of medical groups, physicians and patients across the state, as well as the evaluation of multiple chronic conditions. However, the results may not be generalized to other settings with health-care environments different from California, where provider groups may be more exposed to external incentives to improve quality [25]. In addition, while the medical groups in our study were located throughout the state, inclusion in our study was reliant on voluntary participation in the performance measurement project. Finally, we also analyzed multiple correlations, increasing the role of chance in identifying the few significant correlations we highlighted.

In conclusion, despite strong correlations between clinical process and outcomes measures, significant gaps in performance on these outcomes measures were common. We identified potentially important correlations between clinical performance and patient reports of clinical interactions and integration of care, reinforcing the need to maintain a focus on patient experiences as part of an overall program to improve chronic disease care.

Funding

This work was supported by the California Healthcare Foundation.

References

View Abstract