OUP user menu

Identifying population-level indicators to measure the quality of cancer care for women

Monika K. Krzyzanowska, Lisa Barbera, Laurie Elit, Asma Razzaq, Refik Saskin, Naira Yeritsyan, Arlene S. Bierman
DOI: http://dx.doi.org/10.1093/intqhc/mzr043 554-564 First published online: 26 July 2011

Abstract

Objective Although there is interest in measuring the quality of cancer care, there has been limited effort to evaluate quality for specific subpopulations such as women or to examine differences in performance among women associated with sociodemographic characteristics. The objective of this study was to identify a comprehensive set of quality indicators for evaluation of the quality of cancer care received by women using administrative data.

Design A conceptual measurement framework developed by the study investigators was used to guide literature review to identify existing quality indicators. The list of potential indicators from the literature was first reviewed by the study investigators with respect to importance and feasibility to determine a set of indicators to present to an expert panel who used a modified Delphi process to select indicators for inclusion using predetermined explicit criteria.

Setting The Project for an Ontario Women's Health Evidence-Based Report Card.

Participants A multidisciplinary expert panel consisting of clinicians, researchers and administrators with expertise in cancer, quality of care and/or health services research.

Main Outcome Measure Set of quality indicators evaluable from administrative data.

Results The initial literature search identified 427 indicators, of which 46 were rated as important and feasible by the study investigators. Following two rounds of ratings and an in-person meeting, the expert panel recommended 31 indicators for inclusion in the final set spanning the following areas: general indicators (three indicators), cancer screening (six), colorectal cancer (four), lung cancer (three), breast cancer (five), gynecologic cancers (five), and end-of-life care (five).

Conclusions A comprehensive set of 31 indicators was identified to evaluate the quality of cancer care received by women that also allows assessment of gender and socioeconomic disparities in cancer care.

  • quality indicators
  • cancer
  • women's health
  • equity
  • modified Delphi panel

Introduction

Cancer is a leading cause of morbidity and mortality worldwide. There has been significant and increasing interest in measuring the quality of cancer care as a mechanism to identify gaps in performance and to support quality improvement efforts [1]. To date, the majority of these efforts have focused on measuring regional performance [2], performance specific to a single cancer [35] or treatment modality [6, 7]. Although a large body of evidence has found that health-care performance can vary by gender and sociodemographic factors [815], there has been limited work in developing indicators from the perspective of potentially unique or vulnerable subpopulations such as women, socially disadvantaged groups or the elderly whose quality of care may be different from that of the overall population. There is growing recognition of the need to incorporate an equity lens into performance measurement and reporting. Quality improvement interventions may narrow, maintain or widen disparities depending on the relative magnitude of improvement among different population subgroups [16]. In fact, recent US legislation has encouraged data collection and reporting of quality indicators by gender, ethnicity and socioeconomic status (Section 185, Medicare Improvements for Patients and Providers Act, 2008 [17]).

Because women and men often have very different experiences with the health-care system and different social contexts that can influence access, quality and outcomes of care, it is important to assess gender differences in performance [18]. There are biological as well as social reasons why incorporating a gender and equity perspective into quality measurement is important especially when it comes to cancer. The burden of disease as measured by cancer incidence varies between men and women [19] and there is evidence that women may have difference susceptibility to carcinogens such as tobacco and may respond to treatment differently [20, 21]. In contrast to other areas of medicine such as cardiovascular diseases where differences in the quality of care between men and women have been documented [14], much less is known about sex-based differences in the quality of cancer care. A 2006 report on the quality of cancer services among women in Ontario commissioned by the Ontario Women's Health Council which analyzed the results of the Ontario Cancer System Quality Index, a web-based annual report on the quality of cancer services in Ontario by sex, found some differences in the use of end-of-life services between women and men but minimal sex-based differences in cancer treatment [10]. However, this report did not look at differences in performance among women associated with socioeconomic position or geography. While few studies have specifically evaluated differences in the care received by women and men with cancer, there have been numerous studies that have documented differences in care among subgroups of cancer patients, including subgroups of women with respect to sociodemographic factors such as age, income, ethnicity and where one lives [2225]. For example, the majority of studies which have evaluated age or income-based differences in cancer care have found that elderly patients and patients of lower socioeconomic status have consistently different patterns of care than their younger or wealthier counterparts. Given the systematic differences in care between subgroups of cancer patients, it is only natural that any quality improvement efforts incorporate this perspective in order to succeed.

In Ontario, the Project for an Ontario Women's Health Evidence-based Report (POWER) study was designed to help policy-makers and providers to improve the health of and reduce inequities among women in Ontario by providing actionable data on women's health and health system performance in the province. As a leading cause of morbidity and mortality both in Canada and worldwide, cancer was included as one of the key diseases of interest in the POWER study. The indicators were also selected for the purpose of assessing differences in performance between women and men, and among women associated with income and geography. The perspective of the study was unique as the existing reports do not routinely examine gender differences in performance, nor do they look at differences between subgroups of women associated with socioeconomic status, ethnicity or geography. Hence, the heterogeneity of women's experiences and needs are not reflected in these reports. Furthermore, many reports focus on particular health sectors and not the continuum of care, and they rarely assess both population health and clinical care. The object of the POWER study was to address these limitations [18, 26].

In this article we present the process we undertook to identify a comprehensive set of quality indicators for evaluating the quality of cancer care at the population level with a unique focus on women and on equity. Given the lack of empiric data on differences in quality of cancer care between men and women, the final report on the selected indicators will both contribute to the evidence base in this area and serve as an example of how to incorporate an equity lens into performance measurement on a routine basis.

Methods

Identification of potential indicators

The overarching goal of the project was to identify a comprehensive set of measures to evaluate the quality of cancer care received by women in Ontario at the population level using locally available administrative health-care data. An interdisciplinary working group of 10 investigators with interest and expertise in performance measurement and cancer health services research was convened to carry out the project. The process began with two in-person meetings of the group to agree on a conceptual framework for measurement, which was then used to guide the literature review to identify candidate quality indicators. The aim of the project was to identify existing measures from the literature rather than develop novel indicators.

Based on discussions among members of the working group, it was decided that the Cancer chapter of the POWER study would look at indicators spanning the entire spectrum of the cancer disease trajectory from screening, through treatment and survivorship to end-of-life care with a specific focus on key causes of cancer morbidity and mortality in women, including cancers that occur both in men and women (colorectal and lung cancer) as well women-specific tumors such as breast cancer and gynecologic cancers of the cervix, ovary and uterus. Indicators relating to behavioral risk factors were not included in the cancer chapter as they were evaluated as part of another chapter of the POWER study focusing on population health [27].

The literature search included both the peer-reviewed literature as well as a targeted scan of the gray literature. Sources of peer-reviewed literature that were included in the search were Medline, Pubmed, Embase, Cinahl, Psychinfo and the Cochrane Database of Systematic Reviews for reviews of indicators. Gray literature sources included quality indicator gateway sites such as the National Quality Forum and National Quality Measures Clearinghouse and the Agency for Healthcare Research and Quality as well as more cancer-relevant sites such as the Cancer System Quality Index (http://csqi.cancercare.on.ca/), the American Society of Clinical Oncology (http://www.asco.org/), the National Cancer Comprehensive Network (http://www.nccn.org/index.asp) and Cancer Control Planet (http://cancercontrolplanet.cancer.gov/). This initial list was reviewed by a research coordinator with expertise in epidemiology, health services research and performance measurement to delete any duplicate measures and compile a final list of unique quality indicators to present to the working group for further deliberation.

Review process

The above search resulted in a master list of several hundred potential indicators. This list was divided into the following areas: cancer screening, colorectal cancer, lung cancer, breast cancer, gynecologic cancers and end-of-life care. At least two members from the working group with content expertise in the given area reviewed the theme-specific list and ranked each indicator according to importance and feasibility of measurement using administrative health-care data in Ontario, Canada. Indicators were felt to be important if they were able to provide key information on women's health issues, identify disparities between men and women or among women and were actionable. Indicators that were rated as both important and feasible by both working group members were retained to be presented to the external expert panel for the final selection process.

The modified Delphi technique was used for subsequent prioritization and selection of the candidate quality indicators. An expert panel was convened based on nominations from investigators in the POWER study as well as stakeholder advisory groups. In order to ensure a range of perspectives on the panel, members from the full range of the continuum of care were invited from primary care to tertiary care and across different oncologic specialties including surgical, medical and radiation oncology and palliative care. Finally, to ensure that the panel was interdisciplinary, a range of professionals were invited including not only physicians, but also health services researchers and administrators. The final panel consisted of 11 participants including three surgical oncologists, three medical oncologists, one radiation oncologist, two palliative care physicians and two cancer health services researchers. For the first round of ratings, each member of the panel was sent an electronic copy of the Cancer Indicator Reference Manual, which described each of the potential indicators in detail and was then asked to complete an internet-based survey to evaluate and rate potential indicators. The panel members were asked to rank the potential indicators with respect to the specified indicator selection criteria: important, amenable to action, valid and reliable, interpretable and comparable and relevant to equity issues which are summarized in Table 1. These criteria were developed from a comprehensive review of indicator selection criteria used by other reporting bodies and projects to meet project objectives, with specific acknowledgment of the indicator selection criteria developed by the National Committee for Quality Assurance (http://www.ncqa.org/tabid/415/Default.aspx) and those used by the Ontario ministry of health and long-term care (MOHLTC) Health System Score Card [18]. Importance was defined as reflecting an important health issue or aspect of health system functioning. Amenable to action meant that the results from the indicator could be used to inform policy or alter behavior of health services providers. Validity, reliability and interpretability reflected the scientific soundness of the indicator. Indicators were felt to be relevant to equity if previous studies suggested that there were or may be significant differences with respect to gender or socioeconomic status. This set of attributes was developed and approved by the overall group of investigators involved in the POWER study and was used by other working groups (such as cardiovascular health and depression working groups) in indicator selection to ensure consistency in indicator selection across disease areas. The panel members were also explicitly asked whether the potential quality indicator should or should not be included in the report card. Panel members were also asked to suggest other potential indicators.

View this table:
Table 1

POWER study indicator selection criteriaa (18)

Selection criterionDescription
Relevance
 Importance/usefulnessThe indicator reflects an important health issue or aspect of health system functioning that matters to the health of the population group in question
The indicator assists in monitoring and measuring health system performance over an extended period of time and is meaningful to stakeholders
 Amenable to actionThe information being collected can be used to inform and influence policy or funding, alter behavior of health services providers, or increase general understanding in the community in order to improve quality of care and population health
Scientific soundness
 ValidityThere is sufficient scientific evidence to support a link between the performance of an indicator and overall positive outcomes to patients
The indicator measures what is both intended and acceptable to the community (face validity), covers relevant content or domains (content validity), and has predictive power (criterion validity)
 ReliabilityThe same result will be obtained if measurements are repeated under identical conditions
 Risk-adjusted or stratifiedThe extent to which non-modifiable influences or factors that differ among groups being compared can be controlled or taken into account when necessary for interpretation
 InterpretabilityChanges in the indicator are commonly understood to be good or bad
 ComparabilityThe indicator can be compared over time, to other geographic areas or to other standards/benchmarks
Feasibility
 Precisely defined and specifiedThe extent to which the measure is standardized with explicit predefined requirements for data collection and calculation of the measure value or score
 Data feasibilityData required for the indicator are available and of sufficient quality for the areas and time periods indicated, such that no unreasonable obstacles or constraints exist either on access to information or restrictions on its use
 Reliability of data collectionThe data for the indicator are collected in a consistent manner by one or more agencies over time
Equity
 Health disparitiesIndicator selection considers where gender, socioeconomic and ethnic disparities in health and health care are greatest, and where there are significant gender differences in health determinants
Comprehensiveness
 Continuum of careThe indicator selection process seeks to identify health status and health care performance across the continuum of care, from population health to primary and tertiary care
  • aDeveloped from a comprehensive review of indicator selection criteria used by other reporting bodies and projects, with specific acknowledgment to the indicator selection criteria developed by the National Committee for Quality Assurance (http://www.ncqa.org/tabid/415/Default.aspx) and those used by the Ontario ministry of health and long-term care (MOHLTC) Health System Score Card.

Results of the initial survey were summarized using pre-defined decision rules and presented to the panel during an in-person meeting. Panel members' responses to the question ‘Should this indicator be included in the POWER study report card?’ were used to classify the indicators into those where there was consensus to either include or eliminate the indicator and those that needed further discussion. Results and comments from the web-based survey were distributed to inform discussion. Indicators which were recommended for inclusion by more than two-thirds of the panel (66.7%) were included in the final list, whereas those which less than one-third (33.3%) of the panel endorsed for inclusion were eliminated. At an in-person meeting, panel members were given the opportunity to recommend further discussion of these indicators if they felt a strong case could be made for their inclusion or exclusion. Then, the rating results pertaining to the remaining indicators where there was no consensus either way were presented to the panel, following which the panel members were asked to re-rate the indicators with only those recommended by the panel included in the final list.

Results

A summary of the indicator selection process is depicted in Fig. 1. The initial literature review identified 427 unique quality indicators which were divided into the following areas: cancer screening, colorectal cancer, lung cancer, breast cancer, gynecologic cancers and end-of-life care. Indicators that were pertinent to more than one cancer site (i.e. cancer incidence) were put into a ‘general’ category. The working group ranked 46 of these as important and feasible. These were subsequently presented to an external expert panel for prioritization. The panel selected 31 indicators for evaluation as part of the Cancer chapter of the POWER study.

Figure 1

Process for the selection of quality indicators for measuring quality of cancer care in women.

Among the approximately 300 indicators excluded during the preliminary review by the working group, approximately 94 were felt to be important but not feasible with currently available administrative health-care data in Ontario. The majority of these indicators were not feasible because they required detailed pathologic reports that include stage and other pathologic data (i.e. hormone receptor status in breast tumors) for evaluation which are currently not routinely available in Ontario administrative data. Most of the indicators assessing receipt of chemotherapy or hormonal therapy were excluded for this reason as were several quality indicators assessing quality of surgery or quality of pathologic reporting. The third group of not feasible indicators pertained to delays in diagnosis and management of symptoms in the advanced cancer population.

Following the first ranking of the 46 indicators by the expert panel, 20 indicators were included and none were excluded. The remaining 26 indicators were discussed during the in-person meeting and of these an additional 10 were selected by the panel for inclusion in the final set of indicators. The panel also suggested one additional indicator pertaining to evaluation of chemotherapy use in small cell lung cancer to be added to the final set. Details of the final set of 31 quality indicators selected by the expert panel are summarized in Table 2. Of the 31 indicators that were selected for inclusion in the Cancer chapter of the POWER report card, the vast majority (29/31) were process indicators. The general indicators group was composed of indicators that were applicable to multiple cancer sites such as incidence of cancer or wait times for surgery. Five indicators were selected to address breast, cervix and colorectal cancer screening including two indicators that addressed follow-up of abnormal screen results among women receiving cervical cancer screening. The disease-specific indicators focused predominantly on different treatment modalities such as surgery, radiation therapy or chemotherapy usually for early stage disease, although there were a few surveillance indicators for cancer survivors. There were very few indicators pertaining to receipt of systemic therapy as these types of indicators were generally considered not feasible with the currently available data in Ontario, specifically because of lack of detailed cancer stage data. Five indicators pertaining to quality of end-of-life care among patients who died of cancer were also chosen which included indicators that assessed both the use of hospital-based services as well as homecare services. The 16 excluded indicators (Table 3) spanned across the entire disease continuum from diagnosis through end-of-life care.

View this table:
Table 2

Description of the 31 quality indicators selected by the expert panel for evaluation

No.Indicator nameDefinitionSource
1.Cancer incidenceThis indicator measures the incidence of new cancers per 100 000 population per year. The rates for colorectal and lung cancers are reported per 100 000 population, while rates of breast, ovarian, uterine and cervical cancers are reported per 100 000 women.APHEO
CSQI
2.Five-year survival rateThis indicator measures the proportion of cancer patients who were alive 5 years after their diagnosis.CSQI
Women's Health Surveillance Report, Health Canada [36]
3.Wait times for surgery (breast, colon, ovary, uterus and cervical cancers)This indicator looks at how long patients wait for cancer surgery, measuring the time between the initial consultation with the surgeon and the date the surgery was done, reporting both the median (the time by which 50% of patients underwent surgery) and the 90th percentile (the time by which 90% of patients underwent surgery).CSQI
4.Breast cancer screeningThis indicator measures the percentage of women aged 50–69 with no previous history of breast cancer who had a mammogram in the last 2 years.Canadian Task Force on Preventive Health Care
CSQI
5.Colorectal cancer screeningThis indicator measures the percentage of adults aged 50–74, without a history of colorectal cancer who had one or more fecal occult blood tests in the previous 2 years.CSQI
6.Cervical cancer screeningThis indicator measures the percentage of women aged 18–70 with no history of cervical cancer, and who have not had a hysterectomy, who had at least one Pap test in the last 3 years.CSQI
HEDIS
7.Cervical cancer screening: follow-up after a low-grade lesion on a Pap testThis indicator measures percentage of patients whose Pap test showed a low-grade lesion (atypical squamous cells of undetermined significance—ASCUS) or low-grade squamous intraepithelial lesion and who had either a repeat Pap test or colposcopy within 6 months of the initial Pap test.RAND Quality Indicators [37]
8.Cervical cancer screening: follow-up after an ‘unsatisfactory’ Pap test resultThis indicator measures the percentage of women with an ‘unsatisfactory’ Pap test result who had another Pap test within 4 months of the original.American Society for Colposcopy and Cervical Pathology guidelines [38]
9.Cervical cancer screening: follow-up after abnormal HPV Pap testThis indicator shows the percentage of women diagnosed with an initial abnormal Pap smear of ASCUS with high-risk human papillomavirus type who have follow-up colposcopy within 6 months of abnormality identified.National Quality Measures Clearinghouse
10.Use of sphincter-sparing procedures for rectal cancerThis indicator measures the percentage of patients with rectal cancer who received a sphincter-sparing procedure at the time of surgery.Agency for Healthcare Research and Quality (AHRQ) Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer [39]
11.Referral to radiation oncology in patients with resectable rectal cancerThis indicator measures the percentage of patients with rectal cancer who had a consultation with a radiation oncologist within 6 months of diagnosis.National Initiative for Cancer Care Quality (NICCQ) [40]
12.Referral to medical oncology following resection of colon cancerThis indicator measures the percentage of patients with colon cancer who had a consultation to consider adjuvant chemotherapy with a medical oncologist in the 4 months following surgery.NICCQ [40]
AHRQ Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer [39]
13.Surveillance colonoscopy after resection of colorectal cancerThis indicator measures the percentage of colorectal cancer patients who received follow-up colonoscopy within 36 months of surgical treatment.AHRQ Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer [39]
14.Surgery for non-small cell lung cancerThis indicator measures the percentage of patients diagnosed with non-small cell lung cancer who underwent lung resection surgery.Dransfield et al. [41]
15.Percentage of non-small cell lung cancer patients treated with systemic therapy within 6 months of surgeryThis indicator measures the percentage of patients with non-small cell lung cancer who received chemotherapy within 6 months of lung resection.CSQI
16.Use of chemotherapy for small cell lung cancerThis indicator measures the percentage of patients diagnosed with small cell lung cancer who received chemotherapy within 6 months of diagnosis.CSQI
NICE Guideline: The Diagnosis and Treatment of Lung Cancer: Guideline 24 [42]
17.Breast-conserving surgeryThis indicator measures the percentage of women who had breast-conserving surgery.AHRQ Measuring the Quality of Breast Cancer Care in Women [4]
18.Axillary lymph node dissectionThis indicator measures the percentage of women who had an axillary lymph node dissection with their primary breast cancer surgery.CSQI
NICCQ [40]
19.Radiation therapy after breast-conserving surgeryThis indicator measures the percentage of women who had radiation therapy after breast-conserving surgery.CSQI
NICCQ [40]
20.Assessment by medical oncologistThis indicator measures the percentage of breast cancer patients seen by a medical oncologist in the 3 months following diagnosis.RAND Quality Indicators [43]
21.Annual surveillance mammography among breast cancer survivorsThis indicator measures the percentage of women with a history of non-metastatic breast cancer who had a surveillance mammogram in each of the first 2 years after their treatment was completed.RAND Quality Indicators [43]
22.Primary ovarian cancer surgery by a gynecologic oncologistThis indicator measures the percentage of patients whose first surgery for ovarian cancer was done by a gynecologic oncologist, within 6 months of diagnosis.Gagliardi et al. [6]
23.Postoperative chemotherapy for ovarian cancerThis indicator measures the percentage of women with ovarian cancer who received postoperative chemotherapy within 4 months of surgery.Elit et al. [44]
24.Rate of reoperationThis indicator measures the percentage of women who had a follow-up operation within 3 months of their first ovarian cancer surgery.Elit et al. [44]
25.Patients referred to a gynecologic oncologist for a staging procedureThis indicator measures the percentage of uterine cancer patients with high-risk histologies (serous and clear cell) who are referred to a gynecologic oncologist for a staging procedure.American College of Obstetricians and Gynecologists, practice guideline 65 [45]
26.Death in an acute-care bed (lung, colorectal, breast or gynaecological cancers)This indicator measures the percentage of patients with cancer who died in an acute-care bed in hospital.CSQI
27.Emergency department visit in the last two weeks of life (lung, colorectal, breast or gynaecological cancers)This indicator measures the proportion of patients who died of cancer who had at least one emergency department visit in the last 2 weeks of life.CSQI
28.Chemotherapy in the last two weeks of life (lung, colorectal, breast or gynaecological cancers)This indicator measures the percentage of cancer patients who received chemotherapy in the two weeks before they died.CSQI
29.Home care visits in the last 6 months of life (lung, colorectal, breast or gynaecological cancers)This indicator measures the percentage of cancer patients who died who received one or more home care visits in the last 6 months of life.CSQI
30.Physician house calls in the last two weeks of life (lung, colorectal, breast or gynaecological cancers)This indicator measures the percentage of cancer patients who had one or more physician house calls in the last two weeks of their lives.CSQI
31.Percentage of women with locally advanced cervical cancer who completed primary chemoradiation within 7 weeks of starting treatmentThis indicator measures the proportion of women with locally advanced cervix cancer who completed their radiation treatment within 7 weeks of starting treatment.Perez et al. [46]
  • APHEO, Association of Public Health Epidemiologists of Ontario; CSQI, Ontario Cancer System Quality Index; HEDIS, Healthcare Effectiveness Data and Information Set.

View this table:
Table 3

List of eliminated indicators

AreaIndicator
General indicatorsMortality rate
Postoperative mortality (lung, colorectal, uterus and ovarian cancers)
ScreeningPercentage of screen eligible women receiving a Pap test
Follow-up colposcopy after two abnormal Pap results
Rate of sigmoidoscopy
Complications of colonoscopy
Rate of cecal intubation during colonoscopy
Colorectal cancerAdequate lymph node retrieval and evaluation in colorectal cancer
Lung cancerRate of open/close thoracotomies for NSCLC
Breast cancerAppropriate use of preoperative mammographic evaluation
Appropriate number of therapeutic operations (≤2) for women having breast-conserving surgery
Appropriate use of guidelines for follow-up surveillance of breast cancer
Gynecologic cancersPercentage of hospitals affiliated with a cancer centre/medical school
Proportion of patients with ovarian cancer who received IV chemotherapy within 6 months of diagnosis
Improvements in 5-year relative survival among women with gynecologic cancers
End of life careICU admission in last 2 weeks of life
  • NSCLC, non-small cell lung cancer; IV, intravenous; ICU, intensive care unit.

Discussion

In this manuscript, we describe the methods and outcome of the process we used for identifying a set of population-level performance measures aimed at measuring the quality of cancer care in women, gender differences in performance and disparities in quality associated with income and geography among women. Although the potential quality indicators were chosen from among existing measures in the literature, the perspective and framework used for selection are unique as the majority of quality measurement efforts to date both within and outside cancer have been disease or procedure focused. Equity in health, which has been defined as ‘the absence of systematic and potentially remediable differences in one or more aspects of health across populations or population groups defined socially, economically, demographically or geographically’ [28], has been recognized by the Institute of Medicine as one of the six core dimensions of quality [29]. Incorporating this perspective into quality measurement by developing indicators for specific vulnerable subgroups of the population and by analyzing indicators by factors such as sex or income is very important to minimize the risk that quality improvement initiatives do not actually widen the gaps in quality of care [30].

The most unique aspect of our study is the perspective from which our indicators were selected. Cancer is a leading cause of morbidity and mortality in women. Furthermore, the diagnosis and treatment of cancer, even if the outcome is not fatal, have a significant impact on quality of life, morbidity and health-care utilization. Although relatively little is known about the differences in cancer care between women and men, gender-based differences in care have been documented in other areas of medicine. More importantly, the available literature does point to significant differences in cancer screening and care between subgroups of women with respect to factors such as age [23], socioeconomic status [31], race [32] and immigration status [33] highlighting the need for incorporating these perspectives into quality measurement. By using a gender and equity lens in measuring quality, we can identify specific subgroups of individuals whose quality of cancer care can be improved greatly. In addition, the types of interventions required to improve care are likely to differ depending on sociodemographic factors. We stratified existing performance indicators by sex and then assessed potential differences in performance among subgroups of women, as well as including gender-specific indicators for cancers that occur only in women. In the future, should biological differences in cancer associated with sex result in gender-specific differences in guidelines for cancers occurring in both men and women, there may be a need to develop gender-specific indicators that reflect these differences.

Another unique aspect of our study is the comprehensive nature of our indicator set. Our indicators span the entire spectrum of cancer from screening through end-of-life care and across several different types of cancers all of which are important causes of morbidity and mortality in women specifically. The comprehensive nature of the indicator set will allow us to look at differences in care in different sectors of the health-care system including primary care (cancer screening), tertiary care (most treatment indicators) as well as community care (end-of-life indicators). This will allow us to determine how the different sociodemographic factors affect quality of care in the different health-care sectors.

There are several limitations to our study. First, we focused on existing indicators in the literature and therefore there may be areas of cancer care received by women, which are very important to evaluate but for which indicators do not yet exist. Furthermore, the majority of our indicators were process indicators, which also reflects the current state of the performance measurement field where few validated outcome quality measures exist. Secondly, because of lack of reliable stage and limited clinical data in the available administrative health-care data in Ontario, a number of indicators felt to be important by the working group members were not feasible and therefore eliminated early in the review process. The majority of these pertained to indicators for which detailed stage and/or pathologic data were needed for measurement such as use of systemic therapy for different types of cancer or quality of pathologic diagnosis. This group of indicators could be feasible in other jurisdictions with more detailed pathology reporting. A number of cancer organizations such as the American Society of Clinical Oncology have developed indicators that could fill this gap in jurisdictions where such data are available [34]. A number of important but not feasible indicators had to do with delays in diagnosis and symptom management and are unlikely to be feasible in any jurisdiction currently but may be feasible in the future as the quality of administrative data improves and electronic health record data becomes more widely available. Thirdly, a few indicators such as cancer incidence or cancer survival are not directly actionable and therefore of limited value if used on their own as a quality measure. However, population-based interventions to reduce risk are available, and when included in a comprehensive set of measures such as those selected as part of this study they provide the context in terms of burden of disease and impact on morbidity and mortality which are crucial from a population health perspective.

In summary, this study used a systematic approach to identify a comprehensive set of indicators to assess the quality of cancer care for women in Ontario from a population health perspective. This approach included a comprehensive literature search, a cohesive working group with a range of expertise and a modified Delphi panel that represented the range of the continuum of care and various disciplines within cancer care. We have since used this indicator set to evaluate the quality of cancer care in women of Ontario and identified a number of gaps in care that are currently being addressed [35]. We hope that our conceptual framework and indicators will be of use to other jurisdictions interested in incorporating equity into performance measurement.

Funding

This study was funded as part of the POWER Study whose funding is from Echo: Improving Women's Health in Ontario, an agency of the Ontario Ministry of Health and Long-Term Care (MOHLTC). This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the MOHLTC. The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES, Echo or the Ontario MOHLTC is intended or should be inferred.

References

View Abstract