International Journal for Quality in Health Care 16:219-227 (2004)
International Journal for Quality in Health Care vol. 16 no. 3 © International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved
The applicability of the Consumer Assessments of Health Plans Survey (CAHPS®) to Preferred Provider Organizations in the United States: a discussion of industry concerns
1 RAND Corporation, Santa Monica. CA, 2 University of California at Los Angeles, 3 RAND Survey Research Group, Santa Monica, CA, 4 University of California at Irvine, CA, USA
Objective. This paper examines the applicability of a leading patient survey, the Consumer Assessments of Health Plans Study (CAHPS®), to Preferred Provider Organizations (PPOs) in the United States.
Design. Elite interviews were conducted with users of the CAHPS® survey in PPO settings.
Study participants. Study participants attended either the California Healthcare Foundation Quality Performance Measurement in Preferred Provider Organizations Forum or the National Conference to Examine PPO Quality. Eleven representatives of state and federal government health care purchasers, commercial PPO plans, and survey vendors were included.
Main outcome measures. The interview included 21 questions addressing experiences with and concerns about using the CAHPS® survey in PPO settings.
Results. Respondents raised concerns about the influence of out-of-network care on CAHPS® reports and ratings of PPO health plans. Suggestions were made for additional PPO-relevant items such as after-hours care, numbers and types of specialists in the PPO network, and disease management.
Conclusions. Modifications to some of the CAHPS® survey items are needed to address concerns of users about their applicability in PPO settings.
Keywords: CAHPS®, patient evaluations of care, patient surveys, Preferred Provider Organizations
Address reprint requests to Leo S. Morales, 1700 Main Street, M-28, Santa Monica, CA 90407-2138, USA. E-mail: morales{at}rand.org
Accepted for publication December 11, 2003.
The Utilization Review Accreditation Commission, Inc. (URAC), the Consumer Coalition for Quality Health Care, and the National Committee for Quality Assurance (NCQA) co-sponsored a conference on quality measurement in Preferred Provider Organizations (PPOs) in March 2000. One of the key issues discussed at this meeting was the use of standardized consumer surveys to assess experiences of consumers using a PPO. Although URAC currently uses consumer surveys in the accreditation of PPOs, PPO administrators have expressed concerns about the applicability of these surveys in PPO settings. To address these concerns, the conference organizers commissioned RAND investigators to prepare an issue paper on the use of the Consumer Assessments of Health Plans Study (CAHPS®) surveys, a leading standardized consumer survey, in PPO settings in the United States.
This paper examines the applicability of the CAHPS® survey to consumers subscribed to PPOs. We review the history of CAHPS®, the definition of PPOs, and previous research on the use of CAHPS® in PPO/FFS settings. We also report the findings of 11 elite interviews with individuals experienced in administering the CAHPS® survey in PPO settings and discuss strategies for facilitating the use of CAHPS® in PPO settings in the future.
PPO definition
PPOs are widely acknowledged, along with Health Maintenance Organizations (HMOs), as the predominant form of managed care in the United States. The PPO care management model is based on open access for patients and a discounted fee for service (FFS) reimbursement system for providers. Typically, PPOs allow members to access any network provider for small co-pay, and out-of-network providers for higher co-pay. The principal distinction between PPOs and HMOs is that HMOs are insurance risk-bearing entities, while the majority of PPOs are not. PPOs contract with risk-bearing entities such as insurers or self-insured employers to provide access to a network of providers and cost containment through discounted fees for network providers. PPOs often perform only a subset of the typical health plan administrative and fiscal functions.
PPOs can be either stand-alone, non-risk network organizations, or can be part of a risk-bearing insurance organization. The majority of PPOs are non-risk bearing organizations that contract to provide a discounted provider network to a payer or benefits administrator. Non-risk PPOs are frequently used by self-insured employers, which cover one-third of all employees who receive employer-sponsored health benefits [1].
Although some PPOs offer a wider array of services, such as actuarial and benefits design, claims administration, and utilization management, the majority of PPOs offer no more than a discount network and related customer service. According to a recent study, most PPO executives report that they are not usually involved in the design of health benefit packages and have very limited involvement in direct medical management [2].
The Employee Benefit Research Institute estimates that PPOs serve >98 million people in the United States, compared with 81 million served by HMOs, and 18 million by indemnity products. HMO and PPO plans together serve >85% of the United States’ commercially insured population [1]. The popularity of PPO plans has increased over time. According to a recent examination of quality evaluations of PPOs, only 18% of covered workers had a PPO plan option in 1988. By 1993, this had increased to 49% of covered workers, and by 1999 to 62%. The number of PPOs in existence has increased to reflect their growing popularity. Over 1000 PPO plans are now available, representing a 60% increase over the last decade [3].
The history of the CAHPS® survey
In the face of growing demand for standardized consumer-oriented information about the quality of health care, the Agency for Healthcare Research and Quality (AHRQ) and the Healthcare Financing Administration (HCFA) funded the CAHPS® project in 1995. (HCFA is now known as the Centers for Medicare and Medicaid Services [CMS].) The main goal of the project was to produce a survey instrument that could be used in a variety of settings to collect reliable and valid information from health plan enrollees about the ambulatory care they received [4]. The CAHPS® survey results have been used to help guide consumer choices among different health plans and to aid in quality improvement efforts.
The CAHPS® core survey was designed for use in FFS and capitated care settings for both the commercially and publicly (Medicare and Medicaid) insured. The CAHPS® survey is also designed to assess adult and pediatric care and can be administered via mail or telephone. To make the CAHPS® survey usable in multicultural settings, it has been translated into Spanish [5,6], Korean, Vietnamese, Chinese, and Khmi.
The first generation CAHPS® (1.0) survey became available in 1997. At the time it was introduced, the NCQA maintained a distinct Member Satisfaction Survey, which differed from the CAHPS® 1.0 instrument with respect to content, approach, and protocol requirements. In order to establish a single national standard, the CAHPS® development team and NCQA worked together to merge the two surveys into a common survey that would incorporate the perspectives and needs of consumers, purchasers, and health plans. The resulting instrument, CAHPS® 2.0H, was released in 1999 (the CAHPS® 2.0H survey includes a set of supplemental Health Plan Employer Data and Information Set (HEDIS) items not included in the CAHPS® 2.0 core survey). In 1999, CAHPS® survey results were available to 90 million consumers in the United States, including 39 million Medicare beneficiaries, 9 million federal employees, 40 million people covered by plans reporting to NCQA, and people in plans surveyed by other sponsors including state Medicaid programs [7–9]. In 2003, the 3.0 version of the CAHPS® survey was released. The 3.0 survey along with supporting materials are available online (www.cahps-sun.org/products/kit.asp).
Previous research on the use of CAHPS® in PPO/FFS settings
There is limited information and no published research about the use of CAHPS® in PPO settings. For example, the National CAHPS® Benchmarking Database (NCBD) Phase 1.0 (1998) data include eight PPO plans out of a total of 54 plans, NCBD Phase 2.0 (1999) includes eight PPO plans out of 307, and NCBD Phase 3 (2000) includes 16 PPO plans out of 282. Only 14 PPOs submitted CAHPS® 2.0H survey results in 2000 to the NCQA.
CMS has conducted an annual survey of managed care Medicare beneficiaries using the CAHPS® survey since 1998 [10]. Currently, CMS fields a FFS survey as well as the managed care survey. In addition, CMS conducts a survey of disenrollees from Medicare health plans. Results of the Medicare survey administrations are described in several published articles [11–18].
Laboratory experiments suggested that CAHPS® information could effect health plan choices by commercially insured [19] and Medicaid beneficiaries [20]. However, Farley et al. found in two separate studies that CAHPS® information did not affect health plan choices made by Iowa or New Jersey Medicaid beneficiaries [21,22]. No research has been conducted on the effect of CAHPS® information on the selection of PPOs.
Elite interviews with purchasers, providers, and survey vendors
To gain further understanding regarding the issues that relate to the use of CAHPS® in PPO settings, we interviewed selected representatives of United States state and federal government health care purchasers, commercial PPO plans, and survey vendors, about their experiences in administering the CAHPS® survey. The purpose of the interviews was to obtain preliminary feedback from some of the users of the CAHPS® surveys in the PPO setting.
 |
Methods |
|---|
 Top Methods ResultsDiscussionSummaryReferences |
|---|
A convenience sample of interview participants was identified from among persons attending the National Conference to Examine PPO Quality sponsored by URAC, the Joint Commission on Accreditation of Healthcare Organizations, and the National Committee for Quality Assurance in Washington, DC (15–16 March 2001), and persons attending the Quality Performance Measurement in Preferred Provider Organizations, a quality of care forum sponsored by the California HealthCare Foundation in Oakland, CA (26 January 2001). To identify individuals for this study, we asked members of our research group and URAC to recommend persons who were knowledgeable about and experienced in administering CAHPS® in PPO settings. Our only inclusion criteria were that the persons be experienced in administering CAHPS® and available for an interview during the field period of the study.
The potential respondents were initially contacted by mail. The initial contact materials included a brief cover letter describing the purpose of the survey, a one-page overview of the survey contents, and a copy of the CAHPS® 2.0 survey. Subsequently, potential respondents were contacted by telephone to schedule the interviews. Of the 17 potential survey respondents, three declined to be interviewed, one was not reachable, one was not familiar with the CAHPS® survey and therefore excluded, one was not available within the survey field period, and 11 completed the survey.
We constructed an interview guide consisting of 21 questions addressing three broad domains of interest. The complete list of questions in the interview guide is included in Appendix A (see Supplementary data available at IJQHC Online). The first set of five questions was designed to collect information about the PPO with which the respondent was affiliated. The second set of six questions was designed to collect information about specific concerns regarding use of the CAHPS® survey in PPO settings. The final set of 10 questions was designed to collect information about the methods used by each respondent to administer the CAHPS® surveys to PPO members. Each interview took an average of 41 minutes to complete (range 25–63 minutes). All interviews were conducted by a single interviewer between 26 March and 18 April 2001.
Participants
Nine of the 11 participants represented the experiences of 11 PPOs with CAHPS®; one participant represented the experience of a survey vendor administering CAHPS® in PPO settings; and one respondent represented the experience of a purchaser of care. The PPOs represented in the survey ranged in size from 170000 subscribers to >8 million (see Table 1). The PPOs represented networks throughout the United States, including Puerto Rico and other US territories. About half of the PPOs reported administering one standard benefits package, while the other half reported administering many different benefit packages. Most PPOs surveyed reported having enrollment data for subscribers: eight reported having enrollment data, two reported having claims data but not enrollment data, and one reported having neither enrollment nor claims data.
|
|
Results |
|---|
|
TopMethodsResultsDiscussionSummaryReferences |
|---|
All participants reported administering the CAHPS® survey at least once since 1999 and anticipated administering it again in 2002. Many participants used professional vendors to administer the survey and all participants rated their vendors as ‘very good’ or ‘excellent’ on a scale from ‘poor’ to ‘excellent’. Response rates to previous administrations of the CAHPS® survey ranged from 25% to 70%.
Many participants reported using the HEDIS protocol to generate the survey sample frame. When asked if they had obtained data from outside sources (i.e. payers) to generate the survey sample frame, most participants stated that they had not done so. Most participants stated that they had relied on PPO encounter data (e.g. claims data) rather than subscription or membership data to generate the sample frame.
Although some participants reported that the PPOs with which they were affiliated administered several PPO products with varying levels of benefits, none reported drawing separate samples for persons subscribed to each product. Many participants reported that they were not making the results of the CAHPS® surveys available to PPO subscribers or PPO providers.
Some participants expressed concerns about the CAHPS® survey instrument itself and the HEDIS survey protocol. These concerns included: that the CAHPS® survey instrument is too long and that the scoring is too complicated; that the response rates are too low; that the cost of administering the CAHPS® survey annually is prohibitive; and that contacting subscribers is difficult.
There were concerns about the impact of out-of-network care on reports and ratings of PPOs, specifically that poor out-of-network experiences might negatively bias reports and rating of experiences of care within PPO networks. Others requested that the utility of the CAHPS® survey questions for quality improvement efforts be improved. A specific comment in this regard was that CAHPS® survey questions are too general to be actionable.
There were several comments on the effects of benefit level and network size on survey responses and the implications for comparisons among PPOs. Specifically, participants were concerned that benefit levels, rather than experiences with care, were influencing survey responses and therefore might be biasing comparisons between PPOs. PPOs contracting with payers offering more generous benefit levels would look better on CAHPS® surveys than PPOs contracting with payers offering lower benefits, regardless of the quality of experiences provided by network providers.
Participants were also concerned that surveys of overlapping networks might eliminate the ability of CAHPS® to differentiate among PPOs in the same market. In many markets, physicians contract with multiple PPOs, resulting in PPO networks with largely overlapping providers. Because of the similarity of providers between networks, consumer surveys of these PPOs would not provide distinct information.
There were a number of suggestions for additional CAHPS® survey items. These included suggestions for items on: the numbers and types of specialists included in the PPO network; the speed of claims processing; access to after-hours care; access to routine care measured in the number of days to get an appointment; the availability of disease management, case management and other targeted programs with networks; and the quality of pharmacy services, including the quality of the pharmacy formulary.
One participant objected to referring to PPOs as health plans. Another participant felt the term ‘clinic’ had a negative connotation and that the terms ‘doctor’s office’ or ‘medical group’ should be used instead. Additional specific comments about the CAHPS® survey instrument are summarized in Table 2. A complete list of these comments is available as Appendix B (see Supplementary data, available at IJQHC Online).
|
|
Discussion |
|---|
|
TopMethodsResultsDiscussionSummaryReferences |
|---|
In this section, we discuss a number of the issues raised by our interview participants. These include concerns about the effects of benefit levels and out-of-network utilization on survey responses, the effect of overlapping networks on the ability to differentiate PPOs within markets, and the request to increase the utility of the CAHPS® surveys for quality improvement efforts. We also discuss issues raised about the CAHPS® survey items.
The effect of benefit differentials on CAHPS® responses
CAHPS® developers have acknowledged the potential effect of benefit differentials on CAHPS® survey responses. The CAHPS® Survey and Reporting Kit suggests that sponsors consider sampling each different benefit level and health plan product as a separate health insurance plan because of the positive association between generosity of benefits and favorable predisposition towards the plan [19].
The implication for PPOs that have contracts with several insurers offering different levels of benefits is that each group of consumers with a common benefits level should be treated as a distinct health insurance plan, and each group should thus be sampled separately. If, however, a PPO has contracts with several insurers offering sufficiently similar benefit levels, those groups of consumers may be regarded as a single reporting unit. Consumers with similar benefits levels pooled across insurers require only a single sampling strategy.
Further, comparisons among PPOs should be made based only on samples of consumers with similar benefits levels. Differences in co-payments and deductibles may affect access to services and perceptions of care received. Although statistical methods can be used to control the confounding effects of benefits on consumers’ assessments, this approach has not been well studied and is not generally advocated by CAHPS® investigators.
The effect of overlapping provider networks
Our study participants expressed concern about the effect of overlapping provider networks on the ability to differentiate among PPOs within markets. In an unpublished report on using the CAHPS® surveys in Oregon, Jones expressed a similar concern [23]. However, even with a high degree of provider overlap among PPOs surveyed, the results indicated a significant amount of variation among PPOs, with CAHPS® reports and ratings of PPOs ranging from one star (significantly below the average health plan in the reporting area) to three stars (significantly above the average health plan in the reporting area). In a published report, Zaslavsky et al. examined variations in consumer assessments within and among markets [24]. In this comprehensive study using Medicare CAHPS® data, ratings of health plans did discriminate among health plans within markets, but not ratings of providers, suggesting that overlapping provider networks do limit the usefulness of provider ratings for differentiating among plans within markets.
Out of network care
PPO executives have expressed concern about the effect of out-of-network utilization on the CAHPS® assessments [2]. There are three reasons why CAHPS® has not attempted to separate in-network and out-of-network utilization of medical care. Firstly, CAHPS® instruments are supposed to work across all kinds of plans. ‘Out-of-network’ is a concept that is meaningful for some plans, but not for others. The use of such a concept could confound plan comparisons. Secondly, it is difficult to construct a survey instrument that sorts care by who provided it or where it was provided. One of the challenges of designing an instrument has to do with the variety of providers and sites of care enrollees may use. Respondents are not able to distinguish consistently between care that is, and is not, provided in the network. The complexity of questions designed to draw this distinction makes such questions impractical. Finally, CAHPS® captures the experiences that members of a health plan have in getting the care they want or need. If going out of network is a significant element of the care that a member of a particular plan receives, patient reports will reflect this. Although concerns exist about the effect of out-of-network care on PPO assessments, there are also good reasons for continuing with the current strategy of not distinguishing between in-network and out-of-network care. Future research might address whether a high proportion of out-of-network claims is associated with low CAHPS® ratings. Even if so, this might be a consequence, rather than a cause, of poor experiences with the network.
Survey sampling issues in PPOs
Three types of PPOs have been described. The first type of PPO has encounter data but no enrollment data, and thus can only generate a sampling frame of eligible in-network users. This type of PPO can only generate a sample based on in-network encounter data so that the sample is necessarily limited to users of care. The CAHPS® consortium has argued that non-users of care should be sampled because health plans are responsible for their health care as well, whether or not they have used care. Further, consumers who have tried to access care but have been unsuccessful due to barriers would be missed, possibly resulting in a biased assessment of a PPO. PPOs with encounter data should only try to obtain data from insurers to include non-users of care in consumer surveys.
The second type of PPO has enrollment data that can be used to generate a sampling frame of both eligible users and non-users of care. This is the least common type of PPO. This type of PPO can comply with the current CAHPS® recommendation, which is to sample both eligible users and non-users of care who have been enrolled for at least some minimum interval.
The last type of PPO lacks encounter and enrollment data with which to generate a sampling frame for CAHPS® surveys. Without these data being available, this type of PPO requires external sources of data, such as insurers, with which it can generate a sample frame.
Once the issue of who to sample has been addressed, it is necessary to construct a sampling strategy. One appealing strategy that addresses the problem of differential benefit levels within PPOs would be a proportionately stratified random sample, with insurers serving as strata within each PPO. Total sample sizes per PPO would be set using general CAHPS® principals (e.g. 300 ‘completes’ per PPO). Within each PPO, this sample would be allocated to insurer-based strata in numbers proportionate to the share of that PPO’s consumers covered by each insurer. Such a strategy has the advantage of permitting efficient, representative, enrollment-based sampling, and also allows reporting of results by insurer (‘roll-up’) as well as more sophisticated simultaneous modeling of PPO-level and insurer-level effects.
Using CAHPS® for health plan quality improvement efforts
There is growing interest in using CAHPS® survey results in quality of care improvement efforts. Scanlon et al. found that managed care organizations use performance measures such as CAHPS® to [25]:
- target quality improvement activities;
- evaluate current performance;
- guide goal setting;
- identify the root cause of problems; and
- monitor initiative progress.
The same study found that CAHPS® survey results are primarily used to evaluate performance relative to other plans and to establish quality improvement goals. The study authors concluded that making the CAHPS® survey questions more specific, thus enabling managed care organizations to identify root causes, would increase the value of the CAHPS® measures for quality improvement efforts [25].
Submitting CAHPS® survey results currently adds five points to a PPO’s NCQA accreditation score. Commencing in July 2003, all PPOs seeking accreditation are scored using their CAHPS® results. Use of CAHPS® data also satisfies the URAC’s requirement for a PPO quality improvement project. Nonetheless, there is some uncertainty about how useful the CAHPS® data is in and of itself for quality improvement [26]. The CAHPS® team is currently conducting market interviews with key stakeholders to determine how to make the CAHPS® survey as useful as possible for quality improvement.
What CAHPS® questions might be subject to different responses based on differences in plan type, benefits, or care management strategies?
The CAHPS® surveys are designed to assess consumers’ experiences with health care at the health plan level. Most questions in CAHPS® are appropriate for consumers whether they belong to a PPO or an HMO. However, some CAHPS® questions may be problematic. Table 2 contains a review of the CAHPS® 2.0 core items (used for reporting) with commentary about the applicability of each item to consumers in PPOs.
Nationally, there is growing interest in assessing consumers’ experiences at the medical group and even the individual physician level. In California, medical group associations and the Pacific Business Group on Health (PBGH) have been collecting physician group level data for several years [27]. Currently, there is an effort to develop a standardized group level consumer survey instrument based on CAHPS® and the PBGH instrument.
There are three principal reasons for developing physician group level CAHPS®-like surveys. Firstly, consumers are often more interested in how different physicians compare with one another than in how plans compare. Secondly, health plans are often composed of distinct physician groups that can be many miles and hours apart. Only the physician group closest to a patient is of most interest. Finally, physician groups may contract with multiple health plans or PPOs, so that the performance of a health plan is really a composite of the performance of multiple physician groups. Previous research has found that there is substantial variation among physician groups [27]. In some markets, consumers have limited or no choice of plan, but do have choice of provider group. Because the proposed physician group-level survey does not include questions about services provided by health plans, the survey instrument may be more relevant than the CAHPS® core health plan survey for PPOs.
As part of the CAHPS® physician group survey project, there has been discussion about using consumer evaluations collected at the physician group level to evaluate health plans. In the state of California, about 100–150 provider groups account for 
80% of plan membership. Given 13 plans, the sample sizes that would naturally accrue to plans through simple random samples of 300 completes per provider group would be very large, perhaps an average of 3000 per plan. The problem with going no further than this is that plan members who are not enrolled in the 100–150 surveyed (larger) provider groups are systematically omitted. It is likely that the omitted patients would differ systematically in demographic characteristics (perhaps more rural, for example) and also perhaps in CAHPS® ratings. This would result in a sample that is not truly representative of the plan populations. In particular, the resultant bias is likely to be large if the proportion of members uncovered varies substantially from plan to plan.
This problem can be addressed in a straightforward and inexpensive way by stratifying plan membership into two groups: those members in surveyed provider groups, and those members not in surveyed provider groups. The simple random samples of 100–150 provider groups could be supplemented with small simple random samples from the stratum of members not in surveyed provider groups within each plan. This would allow valid inference at the plan level, while increasing total sample size by only 3–4%.
Weights would be needed for inference at the plan level. People in groups other than large provider groups would have larger weights, and this would also correct for differences in the representation of provider groups from plan to plan. This would involve some design effects at the plan level, but design effects that could be easily accommodated by the large plan-level sample sizes. Even with design effects as large as 3–4%, effective sample sizes would be 1000 per plan, resulting in excellent power for plan-level inference.
Comments on the CAHPS® instrument
Next, we discuss comments made by our interview respondents about items (questions) included in the CAHPS® survey (Table 2). One study participant commented that survey respondents might confuse a personal doctor (i.e. the doctor who knows you best) with a primary care doctor (i.e. a doctor who may not know you at all, but may serve as a gatekeeper). In the survey, respondents are provided a definition of personal doctor specifically to mitigate confusion. However, the definition currently used might be modified to specifically mention that a personal doctor is not necessarily the same doctor as an assigned primary care provider.
One study participant commented that CAHPS® question 9 does not explicitly exclude mental health providers as specialists. The reporting of mental health providers in the specialist category was not identified as a problem in cognitive testing conducted during the development of CAHPS® 1.0 or the refinement CAHPS® 2.0; however, this was not a focus of these studies. Furthermore, cognitive testing may have failed to identify this as a problem for a variety of reasons, including respondents’ discomfort in reporting use of mental health providers to the interviewer. The wording of this item in CAHPS® 3.0 may help to mitigate this problem further. In CAHPS® 3.0, the wording for this item is as follows: ‘Specialists are doctors like surgeons, heart doctors, allergy doctors, skin doctors, and others who specialize in one area of healthcare. In the last 12 months, did you or a doctor think you needed to see a specialist?’.
Because physical and mental health services are frequently provided to health plan members under separate arrangements, the CAHPS® team initially decided to have mental health care services (i.e. behavioral health care) assessed by supplemental survey items. For CAHPS® 2.0, the CAHPS® team decided to develop a separate instrument to assess the experiences of consumers with behavioral health care, the Experience of Care and Health Outcomes (ECHOTM) survey. Information about the ECHO survey can be obtained online (www.hcp.med.harvard.edu/echo).
Some study participants commented that CAHPS® questions 10 and 23 would be confusing to PPO members responding to the CAHPS® survey because referrals and health plan approvals are not required to see specialists in most, but not all, PPO settings. The item about delays in care due to pending approvals could be dropped from PPO surveys where approvals are not required. Referrals, however, whether they require approval or not, have an important role in facilitating care in both PPOs and HMOs. In CAHPS® 3.0, the latest version of the CAHPS® survey, these questions have been changed to address these problems. In CAHPS® 3.0, the wording is as follows: ‘In the last 12 months, how much of a problem, if any, was it to see a specialist that you needed to see?’. The following question is: ‘In the last 12 months, did you see a specialist?’.
One study participant commented that it was inappropriate to hold PPOs accountable for access to telephone advice during regular clinic hours (see CAHPS® question 14). The position of the CAHPS® team has been that PPOs are responsible for the care provided by contracted network providers, even though they do not directly control physicians’ office practices. If patients are finding it hard to get health advice over the phone, a PPO could act to improve access either directly through quality improvement efforts or by reporting access problems to the quality improvement officer for a physician group.
Other study participants commented that payers rather than the PPOs send the majority of written information received by PPO members, so it was not appropriate to hold PPOs accountable (CAHPS® questions 32 and 33). Other participants commented that although PPOs have member services, the majority of questions posed by members are referred to payers because they are mostly about covered benefits, which are administered by payers (CAHPS® question 34). Finally, one study participant commented that PPOs should not be held accountable for paperwork because for the most part, PPOs do not require members to complete any paperwork (CAHPS® question 36).
The CAHPS® team is now developing a modular approach to survey items that will allow different survey sponsors and users to assemble subsets of items that are most applicable and relevant to their particular market and situation. This modular approach, called Ambulatory CAHPS® (A-CAHPS®), will result in a more refined set of core measures that are relevant across the full range of care delivery systems (including PPOs) as well as a set of supplemental items that have been expanded to meet the informational needs of specific types of care delivery systems or markets. The CAHPS® Survey User Network (SUN) will maintain a library of formatted surveys covering the most common needs as communicated by sponsors. Testing and development of the A-CAHPS® modules and reporting strategies will be ongoing well into 2004 to provide adequate time to elicit comment from survey sponsors and other key CAHPS® stakeholders.
|
Summary |
|---|
|
TopMethodsResultsDiscussionSummaryReferences |
|---|
Although the CAHPS® surveys have been widely used in HMOs, little is known about the use of the CAHPS® surveys in PPO settings. An important barrier to applying the CAHPS® surveys in PPOs is generating an appropriate sample. Only a small fraction of PPOs have enrollment data (recommended by CAHPS® for sampling) or encounter data. Encounter data limits the sample frame to users of care, so that non-users of care who may experience poor access are excluded. The majority of PPOs lack both enrollment and encounter data, making sampling impossible without data from insurers.
Varying benefit levels among PPO members further complicates sampling. To apply the CAHPS® surveys, each consumer group with a common benefit level should be treated as a separate health plan. Finally, because the CAHPS® surveys ask about some services for which PPO executives do not believe their organizations are accountable, modification may be needed for administration in PPO settings. The CAHPS® physician group survey instrument may be more acceptable to PPO stakeholders. In addition, there may be opportunities to further assess and refine CAHPS® survey items for use in PPOs through additional focus group research. Finally, the modular approach being considered by the CAHPS® team now may provide the fiexibility that PPOs deserve in using the survey instrument.
The authors wish to thank Charles Darby (AHRQ), Liza Greenberg (URAC), and Jim Bost (National Council on Quality Assurance) for their helpful comments on the manuscript. They also wish to thank the 11 survey respondents for their time and effort. Support for this paper was received from the URAC and a grant from the AHRQ (5U18HS09204-05) to RAND. L.S.M. and R.D.H. also received partial support from the UCLA/DREW Project EXPORT, the National Institutes of Health, the National Center on Minority Health and Health Disparities (P20-MD00148-01), the UCLA Center for Health Improvement in Minority Elders/Resource Centers for Minority Aging Research, and the National Institute of Aging (AG-02-004). L.S.M. also received partial support from a Robert Wood Johnson Foundation Minority Medical Faculty Development Program fellowship.
|
References |
|---|
|
TopMethodsResultsDiscussionSummaryReferences |
|---|
- Greenberg L. The State of PPO Performance Measurement: Case Study Report. Washington, DC: American Accreditation Healthcare Commission/URAC, 2000.
- Hamer R, Anderson D. PPO Operations and Markets. St Paul, MN: Interstudy Publications, 2000.
- Shelton LK, Bridgeport C, Carson AF et al. Quality Evaluation for Preferred Provider Organizations: NCQA Position Paper. Washington, DC: National Committee on Quality Assurance, 2000.
- Crofton C, Lubalin JS, Darby C. Foreword. Med Care 1999; 37: MS1–MS9.[Web of Science][Medline]
- Weidmer B, Brown J, Garcia L. Translating the CAHPS 1.0 survey instrument into Spanish. Med Care 1999; 37: MS89–MS97.[CrossRef][Web of Science][Medline]
- Morales LS, Weech-Maldonado R, Elliott M, Weidmer B, Hays RD. Psychometric Properties of the Spanish Consumer Assessments of Health Plans Survey. Hisp J Behav Sci 2003; 25: 386–409.[Abstract]
- Cleary PD. The increasing importance of patient surveys. Br Med J 1999; 319: 720–721.
[Free Full Text] - Brown JA, Nederend SE, Hays RD, Short PF, Farley DO. Special issues in assessing care of Medicaid recipients. Med Care 1999; 37: MS79–MS88.[CrossRef][Web of Science][Medline]
- Millar JS, McCauley D, Hays C, Winston T, Mitchell L. Consumer Assessment of Health Plans Survey (CAHPS) results for Oklahoma managed care Medicaid, 1997, 1998, and 1999. J Okla State Med Assoc 2000; 3: 109–117.
- Goldstein E, Cleary PD, Langwell KM, Zaslavsky AM, Heller A. Medicare managed care CAHPS®: a tool for performance improvement. Health Care Financ Rev 2001; 22: 101–107.
- Lurie N, Zhan C, Sangl J, Bierman AS, Sekscenski ES. Variation in racial and ethnic differences in consumer assessments of health care. Am J Manag Care 2003; 9: 502–509.[Web of Science][Medline]
- Cox D, Lanyi B, Strabic A. Medicare health maintenance organization benefits packages and plan performance measures. Health Care Financ Rev 2002; 24: 133–144.[Medline]
- Langwell KM, Moser JW. Strategies for Medicare health plans serving racial and ethnic minorities. Health Care Financ Rev 2002; 23: 131–147.[Medline]
- Harris-Kojetin LD, McCormack LA, Jael EM, Lissy KS. Beneficiaries’ perceptions of new Medicare health plan choice print materials. Health Care Financ Rev 2001; 23: 21–35.[Medline]
- Zaslavsky AM, Cleary PD. Dimensions of plan performance for sick and healthy members on the Consumer Assessments of Health Plans Study 2.0 survey. Med Care 2002; 40: 951–964.[CrossRef][Web of Science][Medline]
- Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care 2002; 40: 485–499.[CrossRef][Web of Science][Medline]
- Schneider EC, Zaslavsky AM, Landon BE, Lied TR, Sheingold S, Cleary PD. National quality monitoring of Medicare health plans: the relationship between enrollees’ reports and the quality of clinical care. Med Care 2001; 39: 1313–1325.[CrossRef][Web of Science][Medline]
- Bender RH, Garfinkel SA. Differences in the structure of CAHPS measures among the medicare fee-for-service, medicare managed care, and privately insured populations. Health Serv Res 2001; 36: 489–508.[Web of Science][Medline]
- Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health plan quality affect health plan selection? Health Serv Res 2000; 35: 933–947.[Web of Science][Medline]
- Kanouse D, Spranca M, Uhrig J et al. Effects of CAHPS® Reports on Medicaid Recipients’ Choice of Health Plans in a Laboratory Setting. Paper presented at the meeting of the Association for Health Services Research, Los Angeles, CA, June 2000.
- Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries. Health Serv Res 2002; 37: 985–1007.[CrossRef][Web of Science][Medline]
- Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev 2002; 59: 319–336.
[Abstract/Free Full Text] - Jones HD. CAHPS® in the Real Word à la Oregon. Presented at the 5th National Users Group Meeting, Baltimore, MD, 13–15 October 1999.
- Zaslavsky AM, Beaulieu ND, Landon BE, Cleary PD. Dimensions of consumer-assessed quality of Medicare managed-care health plans. Med Care 2000; 2: 162–174.
- Scanlon D, Rolph E, Darby C, Doty H. Are managed care plans organizing for quality? Med Care Res Rev 2000; suppl. 2: 9–32.
- Garg PP, Lee P, Hays RD, Kahn KK, Cleary PD. Strategic Plan for Quality Improvement Using Medicare CAHPS®. FFS Information, 2000.
- Hays RD, Brown JA, Spritzer KL, Dixon WJ, Brook RH. Member ratings of health care provided by 48 physician groups. Arch Intern Med 1998; 158: 785–790.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||