OUP user menu

Validation of data and indicators in the Danish Cholecystectomy Database

Kirstine Moll Harboe, Kristian Anthonsen, Linda Bardram
DOI: http://dx.doi.org/10.1093/intqhc/mzp009 160-168 First published online: 21 March 2009

Abstract

Objectives In The Danish Cholecystectomy Database (DCD), quality indicators are derived from clinical data in combination with administrative data from the National Patient Registry. The indicators ‘Length of postoperative stay ≤1 day and no readmission’, ‘Length of stay (LOS) >3 days and/or readmission’, ‘Additional procedures within 30 days’, ‘Reconstructive bile duct surgery’, ‘Other surgery of the bile duct’ and ‘Death within 30 days’ are all derived from administrative data. This study investigates the validity of the administrative data and evaluates the association between these indicators and postoperative complications.

Research design and subjects Data from 1360 medical records of patients undergoing cholecystectomy were compared with the relevant administrative data from the National Patient Registry. The medical records served as the ‘gold standard’. The association between the individual indicators and the occurrence of a postoperative complication was assessed.

Measures Validation of administrative data against the gold standard was done by the calculation of per cent agreement (including kappa-values) sensitivity/specificity and predictive values. The association between indicators and complications was analysed with crude event rates and odds ratios.

Results The validity of the administrative data was excellent (97.1–100% agreement, κ = 0.73–1.00). All of the indicators except ‘Other bile duct surgery’ were significantly associated with postoperative complications. A subdivision of some indicators strengthened the associations.

Conclusions The DCD is a valid method for monitoring the quality of cholecystectomy in Denmark.

Keywords
  • administrative data
  • clinical database
  • cholecystectomy
  • quality indicators
  • validation

Introduction

Gallstones are a common condition in Denmark and other Western countries [1]. They can be asymptomatic, but can also give rise to disabling pain attacks and sometimes cause serious morbidity such as acute cholecystitis, pancreatitis and obstruction of the bile duct [2].

The treatment of symptomatic gallstones is surgical removal of the gallbladder, and the recommended procedure is a laparoscopic cholecystectomy. A conventional open cholecystectomy is recommended only if the indication is cancer or the procedure is part of another operation where open access is the standard procedure [3, 4].

In the early 1990s, the laparoscopic technique rapidly and based on very sparse scientific evidence replaced the conventional open procedure. The new technique was considered to give less postoperative pain, faster recovery and reduced hospitalization [5]. Critics claimed that in the new era of laparoscopy the rate of serious adverse events such as iatrogenic injury to the bile duct increased [6]. The initial increase in bile duct injuries was believed to be due to the learning curve difficulties and lack of laparoscopic surgical experience [7, 8]; the rate of bile duct injuries is now believed to be on the pre-laparoscopic level [9]. Today both the public and the medical society are interested in monitoring the quality of cholecystectomy and ensuring cholecystectomy to be a safe procedure.

The Danish Cholecystectomy Database (DCD) is a national, clinical database with mandatory report of all cholecystectomies since 1 January 2006 [10]. The database monitors the quality of cholecystectomy nationally and in each hospital. In 2003, the rate of cholecystectomy in Denmark was 143 per 100.000 populations, 7000 annual operations [11].

The aim of the DCD is to increase the proportion of cholecystectomies completed laparoscopically, decrease the postoperative LOS and reduce the frequency of surgical complications [4]. The database is founded in the Danish Surgical Society, led by a steering committee appointed by the society and funded by the Federation of Danish Regions.

The database is designed to get the utmost level of completeness and keep time-consuming data collection to a minimum in an attempt to combine the advantages of administrative data with the value of clinical data.

Clinical data are entered into a secure web site by the surgeon on a single occasion after the operation. A unique personal identification number assigned to all residents in Denmark then links these clinical data to administrative data from the National Patient Registry. The follow-up of the patients relies solely on administrative data from contacts with the health care system registered in the National Patient Registry. The variables in the DCD are listed in Box 1.

Box 1 The two sets of data used in the Danish Cholecystectomy Database

Clinical data entered into a secure web site by the surgeon:

The operation:

  Personal identification number

  Date of surgery

  Type of cholecystectomy

  Reason for primary open cholecystectomy or for conversion

  Length of incision

  Perioperatively acknowledged bile duct injury

  Method of cystic duct closure

  Completion of cholangiography

Risk factors:

  ASA-score

  Obesity

  Acute cholecystitis

  Previous acute cholecystitis

  Previous acute pancreatitis

  Previous upper abdominal surgery

  Concomitant laparoscopic removal of bile duct stones

  Surgeon's experience

  Operation used for education

Administrative data from the National Patient Registry

Admission:

  Personal identification number

  Date of surgery

  Length of postoperative stay (LOS)

  Readmission within 30 days

  Complications within 30 days:

Codes for reconstructive surgery of the bile duct:

  Cholangioduodenostomy

  Cholangiojejunostomy

  Hepaticojejunostomy extrahepatica

  Hepaticojejunostomy intrahepatica

  Bilio-entero anastomosis

Codes for other (“non-reconstructive”) surgery of the bile duct

  Cholangioraphi

  Laparoscopic cholangioraphi

  Other surgery of the bile duct

  Other laparoscopic surgery of the bile duct

Codes for additional procedures

  ERCP (Endoscopic Retrograde Cholangio-Pancreaticographi)

  ERCP with sphincterotomy/stent/stone clearance/dilatation

  Ultrasonography with puncture or drainage

  Treatment for wound rupture and wound infection

  Surgery for deep (intraperitoneal) infection

  Surgery for superficial bleeding

  Surgery for deep (intraperitoneal) bleeding

  Exploratory laparotomy

  Laparoscopy

Death within 30 days

The surgical quality of cholecystectomy is assessed using the clinical indicators listed in Box 2.

Box 2 Clinical indicators in The Danish Cholecystectomy Database

Indicators derived from the clinical data

  • Total number of cholecystectomies performed

  • Number of cholecystectomies where laparoscopic procedure is considered to be the standard operation according to Danish consensus

  • Proportion of laparoscopically completed cholecystectomiesa

  • Proportion of cholecystectomies converted from laparoscopic to open surgery due to complications

  • Proportion of cholecystectomies converted from laparoscopic to open surgery due to difficult anatomic conditions

  • Proportion of cholecystectomies used for education

Indicators derived from the administrative data

 Proportion of cholecystectomies with

  • LOS ≤ 1 day and no readmission within 30 daysa

  • LOS > 3 days and/or readmission within 30 daysa

  • Reconstructive surgery of the bile duct within 30 days

  • Other surgery of the bile duct within 30 days

  • Additional procedures within 30 days

  • 30 days mortality

aThe denominator in this proportion is the cholecystectomies where laparoscopic procedure is considered to be the standard operation, according to the general Danish consensus [4].

Indicator reports are generated with algorithms being applied to the combined dataset and presented with the statistical process control technique [12]. The reports are confidential and are sent monthly to the surgical departments and the heads of the hospitals [12]. A public report of the national and local results is published annually on a web site for Danish national health care quality assessments (www.kliniskedatabaser.dk).

It is appealing to use administrative data in quality assessment: they are inexpensive, readily accessible and unbiased by selection [13]. Administrative data are, however, collected for clerical purposes, and a disadvantage is lack of complementary clinical data with information of potential confounders. Clinical data collected with a predefined purpose are more valuable in quality assessment but also costly and time-consuming to assemble.

The use of administrative data for other purposes than intended requires a thorough evaluation of their validity as data quality is critical for the utility [14, 15].

Objective

In this study we assessed the validity of the administrative data in the National Patient Registry used in the DCD and explored the extent to which the indicators of quality derived from administrative data were associated with postoperative complications.

Methods

Medical record review

Validation of administrative data

Data from the National Patient Registry were validated against corresponding data derived from the review of medical records serving as the ‘gold standard’.

A list of all patients (n = 1363) operated with laparoscopic or open cholecystectomy in two university hospitals in Copenhagen in 2004 or 2005 was generated from the National Patient Registry. The National Patient Registry consists of data from the patient administrative systems cumulated by the National Board of Health. Administrative data about length of postoperative stay (LOS), readmission, postoperative procedures (according to predefined codes) and death were extracted. The medical records were perused for the following data: Personal identification number; date of surgery; sex; age; type of cholecystectomy; LOS; readmission; postoperative additional procedures; and death within 30 days after surgery. The day of the cholecystectomy counted as day zero, and admission prior to the operation was not included.

Evaluation of indicators

From the medical records the reviewers registered also the occurrence of a postoperative complication, defined as an unexpected event, medical or surgical, likely related to the surgery or anaesthesia. Only in-hospital complications or complications leading to readmission were included. If a patient flagged an indicator, the most likely reason for the indicator to occur was noted and grouped into the following categories: complication to surgery; treatment of bile duct stone; prolonged recovery due to open surgery; pain-related conditions; postoperative treatment with antibiotics; social circumstances; and other.

All of the medical records were reviewed by either of the two authors: KMH or KA, both senior residents at one of the surgical departments. The review followed a schematic form designed for the study. Both the medical and the care plan notes were used. The time of discharge was often more precisely found in the care plan notes. The record was discussed and consensus obtained by the reviewers in case of doubt. In case of disagreement between data in the medical records and the National Patient Registry a discharge summary was procured from the administrative systems, mainly to confirm readmissions and additional procedures in other hospitals.

Measurements

Validation of the administrative data

Validation of the administrative data in the National Patient Registry against the medical records was assessed as percent agreement with chance-corrected kappa values according to the Landis and Koch criteria (<0.00 = ‘poor agreement’; 0.00–0.20 = ‘slight agreement’; 0.21–0.40 = ‘fair agreement’; 0.41–0.60 = ‘moderate agreement’; 0.61–0.80 = ‘substantial agreement’; and 0.81–1.00 = ‘almost perfect’ [16]). Validity was also assessed with statistics of diagnostic tests with the medical records serving as the gold standard [17]. ‘Sensitivity’ measures how well the patients who flag the indicator in the medical records are captured by the National Patient Registry; ‘specificity’ how well the patients without the indicator are captured; ‘the positive predictive value’ indicates how well the National Patient Registry correctly identifies the patients with the indicator; and ‘the negative predictive value’ how well patients without the indicator are correctly identified.

Evaluation of indicators

The patients were categorized as flagged or not flagged for each indicator by the review of the medical records, and postoperative complications were noted. Crude event rates of postoperative complications in both groups were calculated. The association between the individual indicators and the occurrence of a postoperative complication was calculated with logistic regression adjusted for sex, age in three groups (<40, 40–69, ≥70) and hospital.

The confidence intervals of the agreements and test statistics were estimated with assumption of a Poisson distribution using a logit transformation to take into account the asymmetry regarding values close to zero and one.

The statistical analyses were conducted with SAS ® 9. 1, SAS Institute Inc., Cary, NC, USA.

Results

Two of the medical records from the 1363 patients were missing from the archives and one patient did not have a cholecystectomy, but a diagnostic laparoscopy miscoded as a laparoscopic cholecystectomy. Thus 1360 medical records were reviewed. There were 1014 women (74.6%) and 346 men (25.4%). The median age was 46 years (range 12–89 years).

Of 1360 cholecystectomies, 1267 (93.2%) were completed laparoscopically, and 93 (6.8%) were either converted from laparoscopic to open procedure or performed as a primary open operation. The median LOS was one day (range 0–133 days). The mean LOS was 1.57 day (SE 4.89).

Sixty-four patients (4.7%) had a surgical complication and nine patients (0.7%) died within 30 days (Table 1).

View this table:
Table 1

Postoperative complications and mortality after cholecystectomy (n = 1360)

Complicationsn (%)
Wound infection/superficial bleeding22 (1.6)
Bile leak from the cystic duct21 (1.5)
Intraperitoneal haemorrhage8 (0.6)
Intraperitoneal infection3 (0.2)
Common bile duct injury2 (0.1)
Peptic ulcer2 (0.1)
Othersa4 (0.3)
Total64 (4.7)
Mortality9 (0.7)
  • aFever, pancreatitis, stricture of the common bile duct, death after severe acute cholecystitis.

Validation of administrative data from the National Patient Registry.

The validation results are summarized in Table 2.

View this table:
Table 2

Validation of administrative data from the National Patient Registry compared to the medical record review (=gold standard) (n = 1360)

Administrative data% Agreement nSimple κSensitivitySpecificityPositive predictive valueNegative predictive value
Type of cholecystectomy (Laparoscopic/open)98.5 (97.7–99.0) 13400.88 (0.83–0.93)99.5 (98.8–99.7)86.0 (77.4–91.7)99.0(98.2–99.4)92.0 (84.1–96.1)
Date of surgery97.8 (96.9–98.5) 1330
LOS ≤1 day (yes/no)97.1 (96.1–97.9) 13210.92 (0.90–0.95)96.7 (95.4–97.6)98.5 (96.4–99.4)99.5 (98.8–99.8)90.4 (86.9–93.1)
LOS >3 days (yes/no)98.7 (97.8–99.2) 13420.94 (0.91–0.97)98.1 (94.2–99.4)98.7 (97.8–99.2)91.1 (85.7–94.5)99.8 (99.2–99.9)
Readmission (yes/no)99.2 (98.6–99.6) 13500.96 (0.93–0.98)93.9 (88.2–96.9)99.8 (99.4–100)98.4 (93.8–99.6)99.4 (98.7–99.7)
LOS ≤1 day and no readmission (yes/no)97.5 (96.5–98.2) 13260.94 (0.92–0.96)97.1 (95.8–98.0)98.5 (96.8–99.3)99.4 (98.6–99.7)93.5 (90.8–95.5)
LOS >3 days and/or readmission (yes/no)98.8 (98.0–99.2) 13430.96 (0.94–0.98)97.7 (95.1–99.0)99.0 (98.2–99.4)95.9 (94.5–97.0)99.5 (98.8–99.5)
Reconstructive surgery of bile duct (yes/no)100 13601.00 (1.00–1.00)100100100100
Other surgery of bile duct (yes/no)99.9 (99.5–100) 13590.73 (0.51–0.94)100100(99.5–100)89.0 (50.0–98.5)100
Additional procedures within 30 days (yes/no)98.1 (97.2–98.7) 13340.86 (0.81–0.91)83.7 (75.2–89.6)99.0 (98.6–99.6)91.0 (83.0–95.0)98.7 (97.8–99.2)
Death within 30 days (yes/no)100 13601.00 (1.00–1.00)100100100100
  • 95% confidence limits are written within parentheses.

  • LOS: length of stay.

The validity of the administrative data was very high (>97% agreement) with kappa values between 0.73 (‘substantial agreement’) and 1.00 (‘perfect agreement’).

Operation code

Of 1361 cases originally identified with an operation code for cholecystectomy 1360 were confirmed in the medical records (99.9%). The cholecystectomy was coded as either laparoscopic or open. No specific code for an operation converted from a laparoscopic to open exists, but the correct code will be an open cholecystectomy. Twenty (1.5%) of the 1360 cholecystectomies were coded incorrectly.

LOS

Sensitivity is the proportion of patients flagging the indicator that is correctly identified by the National Patient Registry. For LOS ≤ 1 day and LOS > 3 days the sensitivity was 96.1% and 98.3%, respectively. The positive predictive value is the proportion of patients that correctly flag the indicator in the National Patient Registry, and the negative predictive value is the proportion of patients that correctly do not flag the indicator in the registry. The National Patient Registry had a small tendency to overestimate the proportion of patients with ‘bad outcome’, that is: to overestimate the proportion of patients with LOS > 3 days (positive predictive value 91.1%) and to underestimate the proportion with LOS ≤ 1 day (negative predictive value 90.4%).

Readmission

Readmission was defined as a contact to a hospital within 30 days of the cholecystectomy classified in the medical record as an admission. Of 1360 patients, 130 were readmitted (9.5%) and only 10 (0.7%) were misclassified in the National Patient Registry. Only two patients were wrongly identified as readmitted in the registry corresponding to a specificity of 99.8%, and with a sensitivity of 93.9% 122 of 130 readmitted patients were correctly identified by the National Patient Registry.

LOS/readmission

A combination of the administrative data ‘LOS’ and ‘readmission’ is used for the indicators ‘LOS ≤ 1 day and no readmission’ and ‘LOS > 3 days and/or readmission’ in the DCD. Nearly all the patients who in the medical records flagged the indicator ‘LOS ≤ 1 day and no readmission’ was correctly identified by the National Patient Registry (sensitivity = 97.1%), and nearly all of the patients in the registry who flagged the indicator was truly positive (positive predictive value = 99.4%). Correspondingly 97.7% of the patients who flagged the indicator ‘LOS > 3 days and/or readmission’ were identified by the registry and 99.5% in the registry were correct.

Surgical procedures

The predefined list of additional procedure codes and bile duct surgery potentially related to complications to surgery is listed in Box 1. Five patients had ‘reconstructive surgery’ of the common bile duct, and all were correctly coded in the National Patient Registry (κ = 1.00). Eight patients had ‘other surgery of the bile duct’, but nine patients were identified due to one miscoded cholangiogram (κ = 0.73). In the medical records, 104 patients had an ‘additional procedure’ but 17 of these were not coded and therefore not found in the registry (sensitivity 83.7%). Of the 96 ‘additional procedures’ identified in the National Patient Registry nine ERCPs (Endoscopic Retrograde Cholangio Pancreatogram) turned out incorrect, as they were preoperative procedures (positive predictive value = 91.0%).

30 days mortality

Nine patients (0.7%) died within 30 days and all of these were identified in the National Patient Registry (κ = 1.00, all test statistics 100%).

Evaluation of the indicator's ability to detect complications

In this part of the study, the association between the clinical indicators derived from administrative data and the occurrence of a postoperative complication were analysed. The results are listed in Fig. 1 and Table 3.

Figure 1

Crude event rates of postoperative complications for each indicator in the DCD.

View this table:
Table 3

Distribution of patients meeting the indicator (flagged) and patients not meeting the indicator (not flagged) and the relationship between indicators and a postoperative complication

IndicatorFlagged, n (%)Not flagged, n (%)Patients with postoperative complicationsORa95% CI
Flagged, n (%)Not flagged, n (%)
LOS ≤1 day and no readmission950 (69.9)410 (30.1)1 (0.1)63 (15.4)0.0060.001 to 0.042
LOS >3 days and/or readmission265 (19.5)1095 (80.5)63 (23.8)1 (0.09)353.348.6 to >999
Readmission within 30 daysb130 (9.6)1230 (90.4)41 (31.5)23 (1.9)26.214.8 to 46.3
LOS >3 daysb156 (11.5)1204 (88.5)33 (21.2)31 (2.6)10.15.8 to 17.5
Reconstructive bile duct surgery5 (0.4)1355 (99.6)2 (40.0)62 (4.6)10.41.7 to 65.5
Other bile duct surgery8 (0.6)1352 (99.4)1 (12.5)63 (4.7)2.70.3 to 23.5
Additional procedures within 30 days104 (7.6)1256 (92.4)42 (40.4)22 (1.8)36.820.6 to 65.8
ERCP within 30 daysb67 (4.9)1293 (95.1)22 (32.8)42 (3.2)14.37.8 to 26.2
ERCP with stent insertion within 30 daysb20 (1.5)1340 (98.5)16 (80.0)48 (3.6)110.435.0 to 348.8
Death within 30 days9 (0.7)1351 (99.3)5 (55.6)59 (4.4)26.06.2 to 108.7
  • LOS: Length of stay ERCP: Endoscopic Retrograde Cholangio Pancreatogram.

  • aAdjusted for sex, age and hospital.

  • bNot used individually in the DCD as indicators.

As expected the indicator ‘LOS ≤ 1 day and no readmission’ was highly associated with a complication-free cause of events (OR = 0.006 [95% CI: 0.001–0.042]). Of the patients flagging this indicator only one had a postoperatively complication (a 53 year-old woman who died on the day of surgery). This patient was also the only one with a postoperative complication who did not flag the indicator ‘LOS > 3 days and/or readmission’. Out of the 265 patients, who flagged the indicator ‘LOS > 3 days and/or readmission’, 63 (23.8%) had a postoperative complication compared to 0.09% (1/1095) of those who did not flag this indicator (OR = 353.3 [95% CI: 48.6–999]).

‘LOS > 3 days’ and ‘Readmission within 30 days’ were analysed as indicators separately although they are not currently used as such in the database. ‘Readmission within 30 days’ was more strongly associated with postoperatively complications than ‘LOS > 3 days (OR 26.2 vs. 10.2). The main reasons for the patients being readmitted were postoperative pain (40.8%) and postoperative complications (31.5%). The main reasons for LOS > 3 days were prolonged recovery after an open cholecystectomy (21%), postoperative complications (18%) and bile duct stones (19%).

Reconstructive surgery of the bile ducts following a cholecystectomy is believed to be an indicator of major bile duct injury. Of the five patients who had reconstructive surgery, two had iatrogenic bile duct injuries, whereas three patients had surgery due to multiple gallstones in the bile duct. The indicator was significantly associated with general postoperative complications (OR 10.4 [95% CI: 1.7–65.5]), but the wide range of the confidence interval due to the small number of patients with reconstructive surgery limits the clinical value of the association.

‘Other surgery of the bile ducts’ was the only indicator not significantly associated with postoperative complications (OR 2.7 [95% CI: 0.3–23.5]. Of the nine patients, one had an incorrectly coded cholangiography, seven had removal of gallstones from the common bile duct and one patient had an insertion of a tube into the proximal part of the common bile duct after an accidental transection of the bile duct. The latter had reconstructive surgery at a tertiary hospital the next day. Thus, only one patient who flagged ‘other surgery of the bile duct’ had a bile duct injury.

‘An additional procedure within 30 days’ of the cholecystectomy was presumed associated with a postoperative complication. In the medical records 104 (7.6%) patients had an additional procedure and 42 (40.4%) of these were done due to a complication. The additional procedures performed in patients without complications were: 13 (12.5%) due to suspicion of complications, 35 (33.7%) ERCPs with bile duct stone removal and 1 (1%) removal of a preoperatively placed bile duct stent. In 13 of the 104 patients (12.5%) the code for ‘laparoscopy’ was added to the code for open cholecystectomy after a conversion and inappropriately counted as an additional procedure. In spite of these wrong codes and procedures done for bile duct stone removal, the indicator ‘additional procedures within 30 days’ was highly associated with postoperative complications (OR 36.8 [95% CI: 20.6–65.8]. To investigate the indicator more closely, we analysed the ERCP-procedures independently. In 67 patients an ERCP was performed within 30 days and of these 22 (33%) had a complication. An ERCP is often combined with some kind of intervention (to remove stones, insert a stent etc.). These interventions all have specific codes and some of the codes might indicate complications to cholecystectomy more than others. Twenty patients had an ERCP with insertion of a bile duct stent, and 16 (80%) of those had a complication; thus ERCP with insertion of a stent was highly associated with postoperative complications (OR 110.4 [95% CI: 35.0–348.8]).

Mortality within 30 days of surgery is commonly used as an indicator of surgical quality. In our study five of the nine patients died due to complications to the cholecystectomy (OR 26.0 [95% CI: 6.2–108.7]). In the remaining four patients the cholecystectomy was done simultaneously with other more complicated operations, judged to be primarily responsible for the complication (one Billroth II resection for perforated ulcer, two right hemicolectomies for colon cancer and one small bowel obstruction).

Discussion

With a growing interest to monitor and measure quality in the health care system a wide range of monitoring systems have been developed. Evidence of whether or not these monitoring systems have been validated is generally sparse, and the lack of evidence of an association between quality measures and clinical quality leads to a general scepticism of quality monitoring among clinicians. In a validation study like the present, the data quality of the database is ensured and the association between the quality indicators and surgical outcome is quantified. Weak indicators can be strengthened or even removed from the database, and the credibility of the conclusions based on the data in the database is increased.

In this study, we have measured the data quality of the administrative data in the National Patient Registry with medical records serving as gold standards, and we found the validity of data used in the DCD to be excellent. We consider the reliability of the medical record reviews to be adequate and usable as gold standard, since the reviews followed a simple scheme, included a limited number of data elements and were done by only two reviewers with both medical and surgical experience. Other studies of the Danish National Patient Registry have also found a high validity of administrative data concerning dates, admissions and operations [18], whereas diagnoses have been found less valid. We demonstrated that the National Patient Registry had a tendency to underestimate the number of open cholecystectomies, but the type of cholecystectomy found in the registry is not crucial as the surgeon also enters the type of cholecystectomy into the DCD, and this is considered to be the true type.

With a national registry readmissions and procedures at all Danish hospitals are included whereas many other international readmission studies and quality programmes have been restricted to single hospitals or single hospital systems due to lack of a national health care data system [19, 20]. In our study, 15% (20/130) of the readmissions were at another hospital that would not have been found in the administrative data of a single hospital. This emphasizes the potential limitations and lack of completeness in clinical databases not using data from national registries.

The validity of the administrative data in regard to surgical procedures was acceptable. Some procedures were, however, incorrectly coded by the surgeon and missing in the National Patient Registry, which underlines the importance of correct coding especially when administrative data are used for quality assessment. In our study, the code for laparoscopy was sometimes incorrectly used in addition to the code for open cholecystectomy, when the operation had to be converted. This underlines the necessity of a systematic thought-through algorithm used to extract the data. In this case, the two codes should be at different times to indicate a reoperation.

In the second part of the study, we analysed the association between the indicators of quality and the occurrence of postoperative complications. An indicator should be able to differentiate the groups of patients of interest, in our case, patients with an uneventful operation from patients with postoperative complications. If a department experiences an increase in the proportion of patients with complications this will be detected by valid quality indicator surveillance, and the department will be able to take action and investigate the process of clinical care before preventable harm is done. If an indicator fails to detect the group of patients of interest or expresses too many false positives, it is invalid as a quality indicator, and hospitals and health care professionals can be wrongly accused of delivering low quality care [12, 21].

All but one of the indicators in the DCD was significantly associated with postoperative complications. The high odds ratios of ‘LOS > 3 days and/or readmission’, in particular, is explained by the collinearity between indicators and the outcome: a patient with a postoperative complication is most likely to have either a prolonged primary admission or a readmission. The only event with a complication combined with a short stay would be death shortly after the operation – as was the case with one patient in our study. So all but one of the 64 patients with complications flagged the indicator ‘LOS > 3 days and/or readmission’, and likewise only one patient with complications flagged ‘LOS ≤ 1 day and no readmission’. The two elements of the indicators were analysed individually, and readmission had a stronger association to complications than LOS > 3 days.

The value of readmission as a quality indicator has been evaluated in several studies with different conclusions. In accordance with our results, readmission is generally considered useful as a quality indicator within specific diagnoses and surgical specialties [22]. The evidence of prolonged hospitalization as a valid measurement of clinical quality is sparse even though LOS is commonly used as a quality outcome [2328]. A short LOS might indicate premature discharge and result in a readmission, but this is taken into account in the DCD, as ‘no readmission’ is included in the quality indicator. Based on our study we can definitely recommend the use of ‘LOS ≤ 1 day and no readmission’ as a valid indicator of good quality after cholecystectomy and ‘LOS > 3 days and/or readmission’ as an indicator of complications, additional treatments for bile duct stones or conversion to open surgery. The indicators might be valuable also in the general quality monitoring of all minor elective surgical procedures.

The indicator ‘additional procedure within 30 days’ had a strong association with postoperative complications as well. Other studies of reoperation as a quality indicator support this [2931].

In our database the main disadvantage of the above-mentioned indicators is the lack of ability to distinguish between patients with real postoperative complications and patients with bile duct stones. Both groups will often have a prolonged hospitalization or readmission and be subjected to additional surgical or endoscopic procedures. To improve its association to postoperative complications analyses of the procedures ERCP with and without stent insertion showed that the indicator ‘additional procedures within 30 days’ could be refined.

To generally improve the monitoring and the global benchmarking of surgical quality, we recommend similar validation studies of selected indicators to ensure the optimal association with postoperative complications and results.

As laparoscopic cholecystectomy superseded open cholecystectomy in the 1990s, it was a general concern that the incidence of major complications such as bile duct injury increased [3234]. Thus one of the aims of the DCD is to monitor the national incidence rate of bile duct injuries. Using the codes for reconstructive and other surgery of the bile duct to monitor, these injuries was found to be suboptimal as too many of these procedures were performed for other reasons. So detailed evaluations of all such patients are now performed by looking at the discharge summaries.

‘Mortality within 30 days’ is a classic and commonly used indicator of surgical quality, but not a reliable or valuable one. In most studies of surgical mortality only in-hospital deaths are included, which makes the measure incomplete and ineffective [35, 36]. As the rate of 30-day mortality may be a valuable outcome for high-risk surgery, it is a rare event following low-risk surgery. In these cases, an intermediate outcome measure such as reoperation, readmission or prolonged admission is more valuable as a quality indicator [37]. The indication of cholecystectomy is based on a clinical judgment where the nuisance of the symptoms is balanced with the risk of surgical complications. Since gallstone disease is a benign condition, the risk of serious events must be kept to a minimum and only low to non-existent mortality rates accepted. In the DCD, discharge summaries are requested for all patients who die within 30 days after cholecystectomy to monitor and analyse this serious event on a nationwide basis.

It is well documented, for example, in the National VA Surgical Quality Improvement Program (NSQIP) that measurement of surgical quality needs to be risk-adjusted when institutions are compared [38]. In the DCD, clinical data and information of patient risk factors are registered to supplement the administrative data from the National Patient Registry (Box 1). With these data, the indicators can be risk-adjusted and hospitals are compared on a superior grounding.

Conclusion

In conclusion, we find the DCD a valid tool to monitor the quality of cholecystectomy in Denmark. The administrative data from the National Patient Registry are valid with high kappa values, sensitivities and positive predictive values. We conclude that the indicator ‘LOS ≤ 1 day and no readmission’ is associated with high-quality cholecystectomy and reflects both a low occurrence of surgical complications and a high degree of organizational quality. The indicators ‘LOS > 3 days and/or readmission’ and ‘additional procedure within 30 days’ are both highly associated with postoperative complications, but the later can be refined. To monitor and investigate the incidence of bile duct injury and other serious complications, the database has to be supplemented with discharge summaries of selected patients. More than 20 000 patients are now fully registered in the database with administrative and clinical data available for calculation of the necessary risk adjustment factors and for further studies of the quality of cholecystectomy in Denmark.

To improve validity in monitoring and global benchmarking of surgical quality in general, we find it appropriate to stress the importance of performing validation studies of the data and the indicators used in quality databases.

Acknowledgment

We wish to thank the medical secretaries Mrs Mie Rokatis and Mrs Jeannie Juul Larsen for the effort of procuring stacks of medical records needed in the validation study.

References

View Abstract