International Journal for Quality in Health Care 15:319-329 (2003)
© 2003 International Society for Quality in Health Care
Paper |
Using hierarchical models to analyse clinical indicators: a comparison of the gamma-Poisson and beta-binomial models
1 The School of Mathematical & Physical Sciences/Statistics, The University of Newcastle, Callaghan, New South Wales, 2308 Australia
2 Health Services Research Group, Faculty of Health, The University of Newcastle, Callaghan, New South Wales, 2308 Australia
Background. Clinical indicators (CIs) are used to assess, compare and determine the potential to improve the care provided by hospitals and physicians. The results for Australian hospitals in 1998–2000 have been reported using a new methodology. The gamma-Poisson hierarchical model was used to correct for the effects of sampling variation by obtaining the empirical Bayesian shrunken estimates for the CI proportions for each hospital. Then, an estimate of the potential system gains that could be achieved if the mean proportion was shifted to the 20th centile is obtained for each of the 185 CIs. The results are used to prioritize quality improvement activity.
Objectives. To describe the 20th centile method of calculating potential system gains in the health care system; to determine the impact of using the beta-binomial model rather than the gamma-Poisson model to obtain shrunken estimates for the CI proportions; and to compare the computationally simpler Method of Moments (MoM) with the maximum likelihood (ML) method for parameter estimation.
Methods. The formulae for the gamma-Poisson and beta-binomial shrinkage estimators were compared analytically. Each of the shrinkage estimators and the two methods of parameter estimation were applied to the Obstetric and Gynecological CIs, and the results compared.
Results. The comparison of the formulae for the two shrinkage estimators showed that the gamma-Poisson model results in greater shrinkage towards the overall mean. This was verified empirically using the clinical indicators. Additionally, the MoM was not a viable alternative to the ML method.
Conclusions. The gamma-Poisson model provided smaller estimates of the potential system gains by up to 6.7% of the numerator for the clinical indicators. The difference in estimation increased with increasing mean proportions and between-hospital variation. We recommend that the beta-binomial model should be used on the basis of both theoretical and empirical grounds.
Keywords: beta-binomial, clinical indicators, empirical Bayesian shrinkage estimators, gamma-Poisson, hierarchical models, system gains
Introduction
Measures of the quality of clinical care and clinical indicators (CIs) are increasingly being used to assess, compare and improve the care provided by hospitals and physicians [1–6]. Since 1993, Australian hospitals preparing for accreditation, or re-accreditation, with the Australian Council on Healthcare Standards (ACHS) have been required to provide data on CIs. Consequently, the ACHS routinely collects data on approximately 185 CIs from more than 500 acute-care Australian hospitals across many specialties [7]. This is the largest source of data that attempts to measure the quality of care in Australia.
The ACHS CIs relate to specific clinical outcomes or processes [8,9]. They are defined by the ACHS as measures of the clinical management and outcome of patient care, which are not exact standards against which hospitals must measure their clinical performance, but rather are designed as screening tools that can alert to possible problems or opportunities to improve patient care [10].
The CI data can be reported as proportions. The numerator represents the number of patients who incur an ‘event of interest’ and the denominator represents the number of patients at risk of the event. New methods for analysing and reporting the CI data have been adopted by the ACHS. Whereas past analyses of CI data have involved comparisons of individual hospital proportions with the average proportion for all hospitals or to a benchmark value set by experts [7], which aims to identify ‘outliers’, the new approach shifts the focus towards a comparison between all hospitals and the corresponding ability to make system-wide improvements.
Reporting techniques that focus on individual hospitals
The use of league tables for ranking performance measures or ‘indicators’ in institutions is commonly practiced [4,11,12]. Providing the results back to hospitals is often seen as an attempt to introduce accountability and motivate individual efforts to improve performance [5,13–15], whilst making the results public is often viewed as providing an element of competition that will encourage improvement activity [12,16,17].
Deming's philosophy, however, dictates that co-operation, not competition, is required in order to facilitate quality improvement [18] and that the components of the system must be understood and improved upon [13]. Competition arising from the publication of league tables can create perverse incentives [4,19]. For example, doctors, health care providers and hospitals may be enticed to fabricate or manipulate their data or alternatively take ‘lower risk’ patients to improve their reported performance [2,4,12,20].
Ranking hospitals is relatively simple for displaying performance measures, even when Bayesian methods have been applied to account for unequal sample sizes. However, the usefulness of hospital league tables is at best limited, if not misleading [4,20–24], because there will always be a ‘first’ and a ‘last’, irrespective of whether all are providing exceptional levels of services or not. Confidence intervals for the hospitals are often employed to identify differences in ranks; however, calculating multiple confidence intervals increases the risk of finding differences due to chance. A more conservative significance level may be used but this increases the widths of the confidence intervals. The confidence intervals for the hospitals at the top and bottom of some performance tables overlap and publishing tables ranking the performances can be meaningless [17,20,25,26].
Additionally, observing changes in rankings over time can be caused by the ‘regression to the mean’ phenomenon [27], concealing any real changes in quality. Andersson et al. [21] provided a measure of the ‘... expected change in the rank order if one were to repeat the study’ to assess the validity of the rankings and showed the ‘... tremendous uncertainty in the ordering ...’. Thus, the analysis of CIs needs to produce more information than a proportion and a rank.
Fundamentally, the league table approach fails to address the issues relating to improvements to the system of hospitals, since its purpose is to isolate and discriminate between the best and worst hospitals [5,11]. That is, reporting the performance of individual units (hospitals or people) may fail to identify system-wide improvements.
An alternative approach previously used by the ACHS was to set threshold values and perform tests of statistical significance using P values. The comparison of individual hospital proportions with a nominal threshold value does little to benefit the hospital system as a whole, because most hospitals will be below the threshold, possibly reducing incentives to improve, while hospitals with larger sample sizes are more likely to be statistically significant. The main outcome of such analyses is to classify the hospitals as either ‘satisfactory’ or ‘unsatisfactory’, and does not focus on system-wide improvements.
A new method for analysing and reporting CI data
A focus on the system of hospitals helps to identify clinical areas in which research and improvement activity may have the greatest effect. A method that uses the data to determine empirically the mean proportion that could potentially be achievable and identifies areas with large ‘potential system gains’ is required. The ACHS has achieved this by reporting the proportion surpassed by the ‘best’ 20% of hospitals, the 20th centile, which is used to provide an estimate of the proportion that is potentially achievable. The potential reduction in undesirable outcomes is estimated by assuming that the mean proportion can be shifted to the 20th centile.
The calculation of potential system gains using the 20th centile is appealing, as it does not rely on an arbitrary target value but instead is data-driven, being influenced by the current level of variation in proportions between the hospitals. The 20th centile is approximately 1 standard deviation (
) from the mean proportion, and can be viewed as a practicable or ‘best practice’ proportion.
The measure of potential gains does not focus on individual hospitals' performances but instead treats the hospitals as part of a system that may have areas for improvement. The results of this approach enable healthcare professionals to determine those clinical areas where there are potentially greater gains and hence funding for quality improvement activity would be of a higher priority.
The 20th centile and subsequent measure of the potential gains, however, should not be based on the observed proportions since they include the effects of sampling variation. Additionally, the data are reported over 6-month periods and proportions will be based on large and small sample sizes, which influences the precision of the estimated proportion. The method used to adjust the proportions involves a two-stage hierarchical model with the empirical Bayesian shrinkage estimator. The technique can be regarded as using the individual hospital's proportion plus the summary results for all hospitals to obtain an improved estimate of the hospital's underlying proportion.
The ACHS publication uses the gamma-Poisson two-stage hierarchical model to represent the between-hospital and within-hospital levels of variation in the CI proportions. This model has been used since the data were analysed in the form of ratios of observed and expected counts, the distribution of which could be represented by the gamma distribution. Additionally, the ‘rare disease’ assumption [28] allows the binomial nature of the observed counts of cases to be approximated as a Poisson distribution.
However, the Poisson approximation may affect the estimates of the parameters and centiles. Thus the primary objectives of this paper are:
- to describe the 20th centile method for calculating potential gains to help prioritize quality improvement activity in the health care system;
- to compare the results for the Obstetric and Gynecological (O&G) CIs using two shrinkage estimators, the gamma-Poisson and beta-binomial, and determine the impact of the model on the results; and
- to compare the computationally simpler Method of Moments (MoM) with the Maximum Likelihood (ML) method for parameter estimation.
Bayesian and frequentist perspectives
Bayesian methods have made inroads into statistical practice [29–32], which is however, predominantly classical, or frequentist. When estimating the parameters of an underlying model, the frequentist approach views the parameter of interest as a fixed number and infers its value from the data collected [30]. Conversely, the Bayesian approach proposes that the unknown parameter of interest is a random variable having a prior distribution summarizing the subjective belief about the plausible values for the parameter [33]. The Bayesian approach generates a posterior distribution for the parameter, conditional on the observed data and the prior distribution. The Bayesian methodology has been shown to be particularly useful in both the clinical setting and the area of public health policy when the results of a study must subsequently be used to facilitate a decision [29,34–37].
For observed data O = (O1,...,On) given a vector of unknown parameters 
= (1,...,n), where may represent the true unknown CI proportions for n hospitals, the density function may be represented by f(O|). The frequentist approach assumes is an unknown but fixed vector to be estimated by O. However, the Bayesian method places a prior distribution g(|
) on , where is a vector of parameters, or hyperparameters, for the prior. The vector represents hospital-specific proportions, whereas represents the parameters that involve the distribution of the population of all units, such as the mean and variance of the hospital proportions. With g and specified, inference concerning is then based upon the posterior distribution, p(|x,), computed using the Bayes' formula as in Equation 1.
 | (1) |
Calculating p(|O,) in Equation 1 assumes that the hyperparameters are known. However, if the value of is unknown then may be replaced by its estimate, , obtained as the value that maximizes the probability of obtaining the data, O, given . That is, select to maximize the denominator in Equation 1. Inferences are then based on the estimated posterior distribution, p(|O,), obtained by substituting into Equation 1.
This approach is referred to as an empirical Bayesian (EB) analysis because the data are used to estimate the hyperparameters. The expectation of , given p(|O,), is the EB shrinkage estimator.
Data
The ACHS collects CI data over 6-month periods. For this analysis, five consecutive 6-month periods of data spanning January 1998 to June 2000 (inclusive), in the clinical area of O&G, were used. In each period, data were collected on 16 O&G CIs. The definitions for the 16 CIs are provided in the Appendix.
There was a minimum of 45 and a maximum of 158 Australian hospitals contributing data for each of the CIs. The average hospital denominator ranged from 38 to 437. The summary statistics by CI are provided in Table 1. The low denominator was caused by some hospitals only reporting for a few weeks in the 6-month period.
|
Methods
The formulae for the gamma-Poisson and beta-binomial shrinkage estimators were compared theoretically to assess the relative shrinkage of individual hospital proportions towards the mean proportion using the two models. We then applied both the gamma-Poisson and beta-binomial models to the data to compare the models empirically. For each CI, estimates for the between-hospital variation, 20th centile and potential gains were obtained for each 6-month period. However, instead of reporting all five periods' estimates, the average value across the five periods was used. This provided an estimate having a smaller standard error than using a single 6-month period.
The observed CI proportions encompass the between-hospital, or systematic variation as well as within-hospital, or sampling variation. Thus, the observed CI proportions for an individual hospital will vary from the ‘true underlying proportions’ due to sampling variation. In the Bayesian paradigm, a two-stage hierarchical model is used to represent the distributions of the two sources of variation. The first level corresponds to the distribution of the CI proportions across all hospitals, thus representing the systematic variation. The second stage corresponds to the sampling distribution of the observed numerator at the ith hospital, Oi.
The beta-binomial hierarchical model
The proportions, pi, are assumed to be drawn from a beta distribution with parameters 
and M, where represents the mean CI proportion and the spread parameter, M, indicates the range of proportions among the hospitals and is inversely related to the variance of the proportions between hospitals, (1—)/(1 + M). Thus pi 
Beta(, M). The Oi are assumed to follow a binomial distribution, Oi binomial(Di, pi), where Di is the denominator for the ith hospital.
The choice of the beta distribution, much like the gamma distribution for the gamma-Poisson model, is for its technical convenience. However, it should not be regarded as an overly restrictive assumption as the distribution can assume a variety of shapes [33].
The ML estimate of the proportion at the ith hospital is calculated as 
. The beta-binomial Bayesian shrunken estimate of the proportion, 
, given in Equation 2 [38] is a weighted average of the prior mean proportion across all hospitals and the individual hospital's observed proportion.
 | (2) |
2, [39–41] and is given by
 | (3) |
The gamma-Poisson hierarchical model
The gamma-Poisson hierarchical model is the model currently used for ACHS reports. This model assumes that the true ratios (of observed, Oi, and expected, Ei, numerators), 
i, are drawn from a gamma distribution with mean, µ, and variance, 
, and that the Oi follow a Poisson distribution with mean iEi. That is, i Gamma(µ, ) and Oi Poisson(iEi). The parameter measures the between-hospital variation in the ratios.
The ML estimate of the ratio at the ith hospital is given by 
. The gamma-Poisson Bayesian shrunken estimate for the ratios, 
, is given by Equation 4 [40].
 | (4) |
Gamma(
) and Oi Poisson(piDi). Converting ratios to proportions by multiplying by , using 2 * variance(ratio) = variance(proportion) and Ei = Di, and setting µ = 1 (discussed later), we obtain 
with Wi given by
 | (5) |
greater than 0.5, where is 
, were converted to the complement, 1-, because the Poisson distribution is more accurate for proportions less than 0.5.
Parameter estimation
The gamma-Poisson and beta-binomial EB shrinkage estimators require estimates of µ and , and and M, respectively. These parameters were estimated using ML and the MoM.
The log-likelihood functions for the gamma-Poisson and beta-binomial models are given in Equations 6 and 7, respectively. These expressions were obtained by taking the log of the product across the n hospitals of the denominator in Equation 1 for the specific two-stage model.
 | (6) |
 | (7) |
Using this estimate ensures that the means of the shrunken and original values are equal and that the estimate of µ will be 1. To obtain the ML estimates for 2 and M, the non-linear procedure in SAS was applied to the log-likelihood functions for each of the gamma-Poisson and beta-binomial models, setting µ equal to 1 and equal to 
.
For the two-stage model, the MoM estimates the systematic variation in the proportions by subtracting the estimated sampling variation from the total variance [42,43]. The total variance in the proportions, Var(Oi/Di) is expressed as
 |
 |
2 for the proportions.
There is no unique MoM estimator [38]. We have used, since the sampling distribution for the proportions follows the binomial distribution, the unbiased MoM estimator from Martuzzi and Elliot [28], given in Equation 8, where 
represents the mean denominator size, averaged across the n hospitals. The estimate is substituted into 
to obtain 
.
 | (8) |
The MoM estimator assuming Poisson variation could also have been considered. However, because the variation in the proportions is binomial, this paper has focused on comparing the gamma-Poisson model with the beta-binomial, as well as the MoM estimation using binomial variation compared with ML estimation.
Calculating potential gains
The potential gains at the ith hospital, Gainsi, resulting from shifting the proportion to the 20th centile, p20, is calculated using Equation 9. Summed across all hospitals, this represents the number of events that would not occur if the mean proportion were p20.
 | (9) |
Figure 1 provides the distributions of 137 hospital proportions for CI 1.1 before and after shrinkage (see Appendix for definition of CIs). The 20th centile and overall mean are identified. The potential system gains identifies the reduction in the number of patients undergoing induction of labour that would occur if the mean was decreased to the 20th centile. With such a change in the mean, the distribution of the hospital proportions would then be ‘centred’ about the 20th centile. A possible distribution has been superimposed in Figure 1B to illustrate the effect. Note, however, the distribution of the proportions is not known; it is only known that the mean of all hospital proportions would be at the current 20th centile.
|
Empirical comparisons
The two hierarchical models (gamma-Poisson and beta-binomial) using ML estimation and the MoM estimator yield three shrunken proportions and three estimates of
2, 20th centiles and potential gains. A ‘crude’ fourth method is based on the original proportions, Oi/Di. This method estimates by calculating the standard deviation of these proportions. The estimated 20th centile for the crude method is also based on these proportions.
Thus, for each CI, and each of the 6-month periods of study, four estimates of 2, 20th centile, and potential gains were obtained. The shrunken proportions obtained using the beta-binomial model and ML estimation were chosen as the baseline for comparisons.
Although the empirical results presented will be the mean of the five period estimates, the results will simply be referred to as 2, 20th centiles, and potential gains, rather than specifying they are mean values.
Because the values for and potential gains vary across the CIs, the equivalent of the coefficient of variation is used to reduce the differences in scales when comparing estimates. The values /, 20th/ and Gains/Numerator each multiplied by 100 are used, where 20th, Gains and Numerator denote the estimated 20th centile, potential system gains (
), and 
, respectively.
Results
Comparison of the beta-binomial and gamma-Poisson shrinkage estimators
The proportionate shrinkages of the ML proportion toward the mean proportion using the beta-binomial and gamma-Poisson shrinkage estimators are expressed in Equations 3 and 5, respectively. Wi in Equation 5 is greater than wi in Equation 3, because /2>((1-)/2)-1. That is, the gamma-Poisson model's shrinkage will be greater than the beta-binomial's shrinkage, assuming that the estimated systematic variation in the proportions based on the gamma-Poisson and beta-binomial models are similar. The results in Table 3 suggest that these estimates are approximately equal. Consequently, the estimates for the 20th centile will be nearer the mean proportion and the potential gains will be smaller when based on the gamma-Poisson model. The difference between the two models' estimates will decrease as the mean proportion or systematic variation decreases ( 
0 or 2 0). These findings are verified empirically in the next section.
|
|
Shrinkage is unimportant for large Di [44] because Wi
0 and wi 0 for increasing Di. Thus, the difference between the two models' shrinkage towards the mean for an individual hospital will also decrease for larger Di.
Empirical results
The summary information for the 16 CIs is provided in Table 2. The results are obtained using the beta-binomial shrinkage estimator and ML estimate of . The ‘potential system gains’ column in the table focuses attention on clinical areas that exhibit a large between-hospital variation and a large number of patients. That is CIs 1.1, 1.2, and 5.1 have the greatest gains and thus quality improvement activity could have the greater potential impact in these areas. Notice, for instance, that although CIs 6.1 and 6.2 have large denominators, there is relatively small between-hospital variation (which is also a function of the low mean proportion) and therefore it is not identified as a high priority area.
Table 3 lists for each CI, /, 20th/ and the differences between Gains for a model and the beta-binomial with ML estimation, as a percentage of the Numerator. It shows the overestimation of the systematic variation when using the crude proportions. The ML estimates of using the gamma-Poisson and beta-binomial models are similar. The MoM estimates are consistently greater than the beta-binomial ML estimate.
The 20th centile of the proportions obtained using the gamma-Poisson model with ML estimation is consistently greater than the 20th centile based on the beta-binomial model. Consequently, the estimated potential gains based on the gamma-Poisson model are less than the gains based on the beta-binomial. This supports the theoretical results of the comparisons of the gamma-Poisson and beta-binomial shrinkage estimators, namely, that the former shrinks the proportions further towards the mean. Compared with the beta-binomial model's estimates, the gamma-Poisson model provides smaller estimates of the potential gains for the CIs by between 0.1% and 6.7% of the Numerator, with an average of 2.3% of the Numerator. For the CIs, that is a difference in estimated gains of 42 patients on average, with the largest difference being 214.
The six CIs yielding the largest differences between the estimated potential gains obtained using the gamma-Poisson and beta-binomial models have larger mean proportions (ranging from 9.7 to 34.7%) and larger (8.1 to 16.8%). These CIs are labelled in Figure 2. The remaining CIs have small mean proportions (six are less than 1.2%), small (seven are less than 1.6%) and have differences in estimated gains for the two models of less than 3.5% of the Numerator numerator (see Figure 2). A similar pattern existed when plotted against . These results support the theoretical results that the difference in the shrunken estimates of the potential gains decreases as and decreases.
|
Using the MoM estimate of
yields estimated potential gains that are greater, by between 0.2 and 52% of the Numerator, than the estimated gains using the ML estimate based on the beta-binomial model (see Table 3), with an average difference of 15.7% of the Numerator. For the CIs, that is a difference in estimated gains of 45 patients on average, with the largest difference being 91.
Conclusions and discussion
The application of EB shrinkage estimators uses the ensemble of hospitals to estimate the true underlying CI proportions and consequently account for sampling variation [31,39,41]. This improves the analysis and reporting of CIs, because it enables all hospitals, regardless of size, to be used in reports. Shrinkage estimators are more informative when denominator sizes differ and some are small [44]. Table 1 shows the range of denominator sizes vary considerably and warrant the use of shrinkage estimators for the CI data.
It is difficult to identify the appropriate model to use, but in this situation the second level is regarded as binomial rather than Poisson since there are a fixed number of patients with each patient having a binary outcome. Hence, the sampling distribution of the CI proportions is more likely to be represented by the binomial distribution than the Poisson. It has been shown, analytically and empirically, that the gamma-Poisson model shrinks the proportions further towards the mean than the beta-binomial model. Consequently, using the gamma-Poisson model will provide smaller estimates of the potential system gains compared with the beta-binomial model, with the difference being less when the mean proportion is small (less than 10%) and/or is small.
In general, the CI proportions are not sufficiently small nor are the samples sufficiently large to warrant the use of the Poisson approximation to the binomial sampling variation; nor is it required. Thus, although the approximation is commonly practiced in the literature because it simplifies the algebra [28], we recommend that the analysis of CI data utilize the beta-binomial model.
The MoM gave results that varied considerably from the ML estimates. The ML procedure is generally acknowledged to be superior to the MoM (in those cases in which the two lead to different estimates) [33]. Thus, although the MoM is computationally simpler than ML, its use is not advisable for the CI data.
Use of performance data for quality improvement is a productive area for further research [2]. Investing in statistical methodology for the analysis and reporting of CI data is necessary [44], and should be focused on the improvement of the health care system. The use of the 20th centile to obtain an estimate of the potential gains from quality improvement activity assists in focusing on system-wide improvements rather than allocating accountability to individual hospitals. Combining with EB shrinkage estimators, it facilitates practicable reports for healthcare providers.
Medical research applications often involve hierarchical data structures as data are collected on random samples of patients nested within each hospital [45]. Accounting for sampling variation is more important when denominators are small [44]. The proportionate shrinkage of 
i towards the prior mean using the shrinkage estimators will decrease as Di increases (i.e. when the evidence from the data overwhelms the prior information) or as the prior variance increases (i.e. when the strength of the prior mean is weak) [37]. This shrinkage shifts the proportions for the hospitals with small denominators closer towards the mean than it does for hospitals with large denominators [39–41]. The shrinkage estimators enable reports to include all hospitals regardless of size and recognize that the hospitals are not independent of each other.
The potential gains quoted in this paper are described in terms of the potential reduction in the number of ‘events of interest’. However, a monetary cost per event may be applied to enable an estimation of the potential reduction in financial terms.
This paper focused on the ACHS method of using the 20th centile and calculating potential system gains. However, ACHS reports also provide measures of stratum gains and outlier gains. Stratum gains represent a similar measure to potential gains, but in terms of the gains that would be achieved from moving the poorer performing strata to the average proportion achieved by the best performing stratum. Similarly, outlier gains provide a measure of the gains achievable simply by changing the proportions for the outlier hospitals (defined as having a difference in observed and expected counts, after shrinkage, more than three times the standard error) to that of the mean proportion. The potential system gains, stratum gains and outlier gains are reported by the ACHS for all 185 CIs [10]. The potential system and stratum gains, can be viewed as measuring ‘system-wide’ variation, since they involve multiple hospitals, whereas the outlier gains, which considers ‘outlier hospitals’, involves only the individual hospitals concerned.
Appendix: Definitions of numerators and denominators for the 16 O&G CIs
CI No. 1.1—Induction of labour other than for defined indications
Numerator: The number of patients undergoing induction of labour for indications other than those listed in the definition (excluding augmentation of labour).
Denominator: The total number of patients undergoing induction of labour for any reason (excluding augmentation of labour).
CI No. 1.2—Induction of labour other than for defined indications
Numerator: The number of patients undergoing induction of labour for indications other than those listed in the definition (excluding augmentation of labour).
Denominator: The total number of patients delivering (including augmentation of labour).
CI No. 2.1—The rate of vaginal delivery following primary Caesarean section
Numerator: The number of patients delivering vaginally following a previous primary Caesarean section, as defined above.
Denominator: The total number of patients delivering who have had a previous primary Caesarean section and no intervening pregnancies greater than 20 weeks gestation.
CI No. 3.1—Primary Caesarean section for failure to progress
Numerator: The number of patients undergoing primary Caesarean section for failure to progress after a period of labour with cervical dilatation of 3 cm or less.
Denominator: The total number of patients undergoing primary non-elective Caesarean section.
CI No. 3.2—Primary Caesarean section for failure to progress
Numerator: The number of patients undergoing primary Caesarean section for failure to progress after a period of labour with cervical dilatation of more than 3 cm.
Denominator: The total number of patients undergoing primary non-elective Caesarean section.
CI No. 4.1—Primary Caesarean section for fetal distress
Numerator: The number of patients undergoing primary Caesarean section for fetal distress as defined above.
Denominator: The total number of patients delivering, including those delivering vaginally.
CI No. 4.2—Primary Caesarean section for fetal distress
Numerator: The number of patients undergoing primary Caesarean section for fetal distress as defined above.
Denominator: The total number of patients delivering by primary Caesarean section only.
CI No. 5.1—Incidence of an intact lower genital tract in primiparous patients delivering vaginally
Numerator: The number of primiparous patients not requiring surgical repair of the lower genital tract, as defined above.
Denominator: The total number of primiparous patients delivering vaginally.
CI No. 6.1—Apgar score
Numerator: The number of babies born with an Apgar score of four or below at 5 minutes post-delivery.
Denominator: The total number of babies born.
CI No. 6.2—Apgar score
Numerator: The number of babies born with an Apgar score of six or below at 10 minutes post-delivery.
Denominator: The total number of babies born.
CI No. 7.1—Term babies transferred or admitted to a neonatal intensive care unit for reasons other than congenital abnormality
Numerator: The number of term babies transferred/admitted to a neonatal intensive care unit for reasons other than congenital abnormality.
Denominator: The total number of term live babies born.
CI No. 8.1—Hysterectomy in women below 35 years of age
Numerator: The number of patients under 35 years of age undergoing hysterectomy for an indication other than malignancy of the cervix, uterus, ovary, and/or Fallopian tube.
Denominator: The total number of patients undergoing hysterectomy for an indication other than malignancy of the cervix, uterus, ovary, and/or Fallopian tube.
CI No. 9.1—Blood transfusion for gynaecological surgery
Numerator: The number of patients receiving a blood transfusion during/post-abdominal or vaginal hysterectomy (excluding laparoscopic hysterectomy).
Denominator: The total number of patients undergoing abdominal or vaginal hysterectomy (excluding laparoscopic hysterectomy).
CI No. 9.2—Blood transfusion for gynaecological surgery
Numerator: The number of patients receiving a blood transfusion during/post-endoscopic operative procedures (including laparoscopic hysterectomy).
Denominator: The total number of patients undergoing endoscopic operative procedures (including laparoscopic hysterectomy).
CI No. 10.1—Urinary tract injury during a gynaecological operative procedure
Numerator: The number of patients suffering injury (with or without repair) to ureter/s or bladder during an abdominal or vaginal hysterectomy (excluding laparoscopic hysterectomy).
Denominator: The total number of patients undergoing abdominal or vaginal hysterectomy (excluding laparoscopic hysterectomy).
CI No. 10.2—Urinary tract injury during a gynaecological operative procedure
Numerator: The number of patients suffering injury (with or without repair) to ureter/s or bladder during an endoscopic operative procedure (including laparoscopic hysterectomy).
Denominator: The total number of patients undergoing endoscopic operative procedures (including laparoscopic hysterectomy).
The authors wish to thank the Australian Council on Healthcare Standards for allowing the use of the data to test the two methodologies. Strict confidentiality of the data was maintained at all times.
Address reprint requests to Peter P. Howley, Room v226, Mathematics Building, The School of Mathematical & Physical Sciences/Statistics, The University of Newcastle, Callaghan, New South Wales, 2308 Australia. E-mail: Peter.Howley{at}newcastle.edu.au 
Accepted for publication March 26, 2003.
References
- Schuster MA, Asch SM, McGlynn EA, Kerr EA, Hardy AM, Gifford DS. Development of a quality of care measurement system for children and adolescents: methodological considerations and comparisons with a system for adult women. Arch Pediatr Adolesc Med 1997; 151: 1085–1092.[Abstract]
- Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. J Am Med Assoc 2000; 283: 1866–1874.
[Abstract/Free Full Text] - Boyce N. Using outcome data to measure quality in health care. Int J Qual Health Care 1996; 8: 101–104.
[Free Full Text] - Marshall EC, Spiegelhalter DJ, Sanderson C, McKee M. Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. Br Med J 1998; 316: 1701–1704.
[Abstract/Free Full Text] - Leyland AH, Boddy FA. League tables and acute myocardial infarction. Lancet 1998; 351: 555–558.[CrossRef][ISI][Medline]
- Westbrook JI, Rushworth RL. Evaluating health care: What can hospital separation data tell us about the complications of hospital care? J Qual Clin Pract 1994; 14: 157–166.[Medline]
- Collopy BT. Clinical indicators in accreditation: an effective stimulus to improve patient care. Int J Qual Health Care 2000; 12: 211–216.
[Abstract/Free Full Text] - Boyce N, McNeil J, Graves D, Dunt D. Quality and Outcome Indicators for Acute Healthcare Services. Commonwealth Department of Health and Family Services, Australia 1997. Available online: http://www.health.gov.au/pubs/qualrprt/execsmry.pdf (3 February 2003, date last accessed).
- Fahey PP, Gibberd RW. Monitoring pulmonary embolisms in Australia—I. Variations between hospitals. Int J Qual Health Care 1995; 7: 373–380.
[Abstract/Free Full Text] - Australian Council on Healthcare Standards. Online: http://www.achs.org.au/content/screens/file_download/DPI2002.pdf (3 February 2003, date last accessed).
- Ashraf H. WHO compares health-care systems across the globe. Lancet 2000; 355: 2223.[ISI][Medline]
- McKee M. Indicators of clinical performance: Problematic, but poor standards of care must be tackled. Br Med J 1997; 315: 142.
[Free Full Text] - Berwick DM. A primer on leading the improvement of systems. Br Med J 1996; 312: 619–622.
[Free Full Text] - Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J Royal Stat Soc Series A 1996; 159: 385–443.
- Blumenthal D, Epstein AM. Part 6: The role of physicians in the future of quality management. N Engl J Med 1996; 335: 1328–1331.
[Free Full Text] - Anonymous. President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry, Final Report. Chapter 4: Advancing Quality Measurement & Reporting, 1998. Available online: http://www.hcqualitycommission.gov/final/chap04.html (3 February 2003, date last accessed).
- Parry GJ, Gould CR, McCabe CJ, Tarnow-Mordi WO. Annual league tables of mortality in intensive care units: longitudinal study. Br Med J 1998; 316: 1931–1935.
[Abstract/Free Full Text] - Deming WE. The New Economics. Cambridge, MA: MIT Press, 1993.
- Azeem MF, Voss S. Performance indicators for general practice. Br Med J 1995; 311: 209–210.
[Free Full Text] - Anderson P. Popularising hospital performance data. Br Med J 1999; 318: 1772.
[Free Full Text] - Andersson J, Carling K, Mattson S. Random ranking of hospitals is unsound. Chance 1998; 11: 34–39.
- Beecham L. Hospital league tables are misleading. Br Med J 2000; 320: 808.
[Free Full Text] - Gore SM. Selection to medical school in Great Britain: league tables only help if uncertainty is properly looked at. Br Med J 1999; 318: 938–939.[ISI][Medline]
- Link WA, Sauer JR. Extremes in ecology: Avoiding the misleading effects of sampling variation in summary analyses. Ecology 1996; 77: 1633–1640.
- Appleby J, Bell A. Reporting NHS performance: how did the media perform? Br Med J 2000; 321: 248.
[Free Full Text] - Langford IH. Statistical basis of public policy: bayesian analysis should be used instead of league tables of performance. Br Med J 1997; 314: 73–74.[ISI][Medline]
- Freedman D, Pisani R, Purves R. Statistics. New York, NY: W.W. Norton and Company, Inc., 1980.
- Martuzzi M, Elliott P. Empirical Bayes estimation of small area prevalence of non-rare conditions. Stat Med 1996; 15: 1867–1873.[CrossRef][ISI][Medline]
- Ashby D, Smith A. Evidence-based medicine as Bayesian decision-making. Stat Med 2000; 19: 3291–3305.[CrossRef][ISI][Medline]
- Carlin BP, Louis TA. Empirical Bayes: past, present and future. J Am Stat Assoc 2000; 95: 1286–1289.
- Eberly LE, Carlin BP. Identifiability and convergence issues for Markov chain Monte Carlo fitting of spatial models. Stat Med 2000; 19: 2279–2294.[CrossRef][ISI][Medline]
- Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res 2001; 10: 277–303.
[Abstract/Free Full Text] - Larson HJ. Introduction to Probability Theory and Statistical Inference. 3rd edn. New York, NY: John Wiley and Sons, 1982.[Medline]
- Gurrin LC, Kurinczuk JJ, Burton PR. Bayesian statistics in medical research: an intuitive alternative to conventional data analysis. J Eval Clin Pract 2000; 6: 193–204.[CrossRef][ISI][Medline]
- Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. An introduction to bayesian methods in health technology assessment. Br Med J 1999; 319: 508–512.
[Free Full Text] - Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. Br Med J 1996; 313: 603–607.
[Free Full Text] - Armitage P, Berry G. Statistical Methods in Medical Research. 3rd edn. Oxford: Blackwell Scientific Publications, 1994.
- Carlin BP, Louis TA. Bayes and Empirical Bayes Methods for Data Analysis. London: Chapman and Hall, 1996.
- Gelman A, Price PN. All maps of parameter estimates are misleading. Stat Med 1999; 18: 3221–3234.[CrossRef][ISI][Medline]
- Clayton D, Kaldor J. Empirical Bayes estimates of age-standardized relative risks for use in disease mapping. Biometrics 1987; 43: 671–681.[CrossRef][ISI][Medline]
- Louis TA, Shen W. Innovations in bayes and empirical Bayes methods: estimating parameters, populations and ranks. Stat Med 1999; 18: 2493–2505.[CrossRef][ISI][Medline]
- Cochran WG. Sampling Techniques. 3rd edn. New York, NY: John Wiley and Sons, 1977.
- McPherson K, Wennberg JE, Hovind OB, Clifford P. Small-area variations in the use of common surgical procedures: an international comparison of New England, England, and Norway. N Engl J Med 1982; 307: 1310–1314.[Abstract]
- Zaslavsky AM. Statistical issues in reporting quality data: small samples and casemix variation. Int J Qual Health Care 2001, 13: 481–487.
[Abstract/Free Full Text] - Sullivan LM, Dukes KA, Losina E. Tutorial in biostatistics: an introduction to hierarchical modelling. Stat Med 1999; 18: 855–888.[CrossRef][ISI][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M. Coory and I. Scott Analysing low-risk patient populations allows better discrimination between high-performing and low-performing hospitals: a case study using inhospital mortality from acute myocardial infarction Qual. Saf. Health Care, October 1, 2007; 16(5): 324 - 328. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R Sund and A Liski Quality effects of operative delay on mortality in hip fracture treatment Qual. Saf. Health Care, October 1, 2005; 14(5): 371 - 377. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||