OUP user menu

Development and validation of comorbidity index in South Korea

Seol-Ryoung Kil , Sang-Il Lee , Young-Ho Khang , Moo-Song Lee , Hwa-Jung Kim , Seon-Ok Kim , Min-Woo Jo
DOI: http://dx.doi.org/10.1093/intqhc/mzs027 391-402 First published online: 20 June 2012

Abstract

Objective This study aims to develop and validate a new comorbidity index using data from hospitalized patients in South Korea.

Study design Retrospective cohort study.

Setting Hospital inpatients.

Study participants Data from 4677 hospitalized patients aged 40–79 who had admitted in a medical center in Korea between September and December in 1997 were individually linked to national mortality data through December 2007. Among them, 3274 patients (70%) were randomly included in the development data set and the other 1403 patients (30%) in the validation data set. Another 3413 liver cancer patients from the same hospital were used to validate the index.

Main measures Comorbidity index and mortality risk.

Results Based on mortality using stratified Cox regression analyses, comorbidity scores ranging from 1 to 4 were assigned for 20 comorbid conditions. Summation of the scores produced the new comorbidity index (Asan comorbidity index, ACI). C-indices in the Cox regression analyses showed the greatest increase [0.589 (95% confidence interval, 0.568–0.609) in age- and sex-adjusted model; 0.697 (0.678–0.716) in the ACI and 0.664 (0.645–0.684) in the Charlson comorbidity index, respectively] when the ACI was added in the validation models.

Conclusions A newly developed comorbidity index using Korean hospitalized patient data based on the International Classification of Disease, 10th Revision (ICD-10) was valid among both general medical inpatients and liver cancer patients. This index may well be widely used in various health-care settings in Korea where patients' information on health conditions is coded with ICD-10.

  • comorbidity index
  • inpatient care
  • patient outcomes
  • mortality

Introduction

Comorbidity means two or more medical conditions or disease processes that are additional to an initial diagnosis [1] and directly or indirectly affects health outcomes. There may be at least four important reasons to measure comorbidity: correcting confounding, identifying effect modification, using them as a predictor of study outcome and improving statistical efficiency [2]. The Charlson comorbidity index, the cumulative illness rating scale, the index of coexisting disease and the Kaplan index have been recognized as valid and reliable methods available for clinical research to measure comorbidity [2]. The Charlson comorbidity index was developed for classifying comorbid conditions which might alter the risk of mortality in longitudinal studies [3] and is now the most extensively used comorbidity index [2]. In this Charlson comorbidity index, scores from 1 to 6 for each comorbid condition are assigned according to mortality hazard ratios and an individual's index is calculated by the summation of theses scores [3]. Subsequently, several Charlson comorbidity index-based algorithms using the International Classification of Disease, 10th Revision (ICD-10) codes were also developed and used [46]. The Elixhauser index was another comorbidity adjustment index defined by 30 comorbidities [7]. This index was developed using the administrative data based on the ICD-9-CM code and algorithms using ICD-10 codes also were developed [6]. While the Charlson comorbidity index, the Elixhauser index and other related indices used mortality data decades ago, prediction abilities of these indices might have decreased because the fatality rate of each disease may well vary as therapeutic technologies have developed over time. In addition, a comorbidity index developed in a country might be sometimes irrelevant to a different population with different cause-of-death structures. It might make scores for several comorbidities non-useful. For example, in South Korea (hereafter ‘Korea’), unlike other western countries including USA, stroke, liver cancer and disease, stomach cancer and tuberculosis importantly contribute to total mortality and constitute the 10 leading causes of death, while mortality from HIV/AIDS is negligible [7]. In addition, the coronary heart disease is the leading cause of death in many western countries but only accounts for 4–6% of total mortality in Korea [810]. Several prior Korean studies were conducted to use and/or validate the Charlson comorbidity index using ICD-10 codes [1113]. Scores for each comorbid condition were based on the Charlson comorbidity index [1113], but any attempt to devise a new comorbidity index using Korean data was not made until now. Therefore, we conducted this study to develop and validate a new comorbidity index using 10-year mortality follow-up ICD-10 code-based data from Korean patients.

Methods

This study was approved by the Institutional Review Board of Asan Medical Center, Seoul, Korea.

Development and validation of comorbidity index in general medical inpatients

To develop and validate a new comorbidity index, we collected the general medical inpatient cohort data from the first admission cases (4677 patients aged 40–79) hospitalized between September and December in 1997 at the department of internal medicine in Asan Medical Centre (AMC), Seoul, which is the largest hospital in Korea. Among the general medical inpatient cohort, 70% (3274 patients) was randomly selected as a development data set and the other 30% (1403 patients) was used as one of the two validation data sets. Information on general characteristics (e.g. sex and age), administrative data (e.g. date of admission and discharge) and ICD-10 code-based diagnosis (principal diagnosis and other comorbid diseases) was obtained from patients' medical records provided by the department of medical record administration in AMC. Using 13-digit unique personal identification numbers, death status of each individual through December 2007 was followed by electronic linkage to national mortality data from the Statistics Korea. According to the registry for domestic relations act in Korea, all deaths of Koreans should be reported to the Statistics Korea and therefore death certificate data among Korean adults are known to be 100% complete [14]. A total of 1813 (55.4%) and 752 (53.6%) deaths from all causes were identified in the development and validation data set, respectively.

Validation of comorbidity index in liver cancer patients

In addition to general medical inpatient cohort data, we used data of 3413 liver cancer patients who were aged 17–86 hospitalized between January 1998 and December 2002 in the same hospital to examine the validity of the newly developed comorbidity index. The same electronic linkage to national mortality data of the Statistics Korea through December 2007 was made, and 2811 (82.4%) deaths were identified by this linkage. Information on general characteristics (e.g. sex, age) and clinical information (e.g. serum albumin, serum bilirubin and prothrombin time) was obtained from the cancer registry data of the AMC. ICD-10-based comorbidity data were also retrieved after medical record reviews which were done by medical record administrators.

Comorbidity

Principal diagnosis was defined as one diagnosis primarily related with the major cause of hospital admission, while comorbidities were other diseases recorded in the inpatient medical records at the time of discharge. Thus, one patient had one principal diagnosis and zero or over comorbidities. Diagnosis was coded with three-character categories of the ICD-10 code, but some important disease categories were divided into more homogeneous groups (e.g. metastatic cancer, acute/chronic renal failure) by authors. A total of 151 disease categories were initially grouped (Appendix Table 1). Among these categories, rare comorbidity groups (prevalence of <0.5%), ill-defined causes (R00–R99), external causes (S00–T99, V01–Y98) and codes for health services use (Z00–Z99) were excluded in our modeling process considering the future utility of the developed comorbidity index. After applying above exclusion criteria, only 70 comorbidity categories (of the initial 151 categories) were considered as independent variables in our analyses to develop the comorbidity index.

Statistical analysis

A stratified Cox model based on age group (ages 40–64 and 65–79) was developed to estimate mortality hazard ratio for each comorbid condition using a bootstrap method in the development data set. The stratified Cox model with a backward elimination procedure was repeated for each of 1000 bootstrap resamplings. The relative frequency of selection of the bootstrap resampling 50% was used as the criterion for inclusion of predictors in the final stratified Cox model [15]. Then, a score for each condition was given based on the magnitude of hazard ratio from the final model. Comorbid conditions with hazard ratio being higher than 1.2 were given scores ranging 1–4. This method was the same developing process of the Charlson comorbidity index [3]: if hazard ratio <1.2, then comorbidity score = 0; if 1.2 ≤ hazard ratio < 1.5, then comorbidity score = 1; if 1.5 ≤ hazard ratio < 2.5, then comorbidity score = 2; if 2.5 ≤ hazard ratio < 3.5, then comorbidity score = 3 and if 3.5 ≤ hazard ratio, then comorbidity score = 4.

The summed scores of comorbidities for each individual constituted the newly developed comorbidity index, Asan comorbidity index (ACI). When different scores for each condition were given by age strata (ages 40–64 and 65–79), age group-specific scores were given. Of 3274 individuals in the development data set, 59.0% scored zero and 7.5% had 1 point (Appendix Table 2). The proportion of individuals with the summed scores being 6 or over was 4.4%. The maximum summed score for each individual was limited to 5 points because the use of total scores ranging between 0 and 13 could not produce a greater predictive ability than when we used scores ranging between 0 and 5. With the developed comorbidity index (ACI), survival rates were presented using the Kaplan–Meier method and log-rank test.

In order to validate ACI, measures of discrimination were performed. C-indices in the Cox regression models including ACI were compared with those of the comorbidity count or the Charlson comorbidity index added the Cox regression models. The C-index was derived from the extension of the concept of the area under the receiver operating characteristic curve and was suggested to measure discrimination in survival analysis [16]. We used SAS macro of ‘survcstd’ for estimating the C-index and its 95% confidence interval (CI) [17]. Based on the C-index and associated 95% CI in the models with or without the ACI, we could evaluate the validity of our newly developed comorbidity index in the validation data set and other disease setting (here among liver cancer patients). For the Charlson comorbidity index, we employed the method by Quan et al. [6] among several ICD-10 code-based algorithms [46], because the method by Quan et al. [6] contained greater numbers of disease categories using ICD-10 codes and has produced a better predictive ability than other algorithms [18]. In order to compare with the Elixhauser index, we used the ICD-10 coding algorithm for the Elixhauser index [6]. For validation, we compared various models with different comorbidity scoring systems. First, the C-index in the age- and sex-adjusted model was examined and 10 prevalent diseases in principal diagnosis were added to age- and sex-adjusted models in the development data. Secondly, the same analyses were conducted in the validation data set. Finally, age-, sex- and clinical information-adjusted models were analyzed and the associated C-indices were estimated in the liver cancer patients. All P-values were two-sided and P<0.05 were regarded as statistically significant. The SAS 9.1 (SAS Institute, Cary, NC, USA) was used for all statistical analyses.

Results

Table 1 presents characteristics in data used for the development data set, validation data set and liver cancer data set. In the development and validation data set, women accounted for 38.9% of 3274 patients and 36.1% of 1403, respectively, while in the liver cancer data set, only one-fifth (19.5%) was female. The mean age (±standard deviation) was 58.2 (±9.9) and 58.0 (±9.7) years in the development and validation data set and 55.0 (±10.4) years in the liver cancer data set, respectively. In the development and validation data set, malignant neoplasm without metastasis was the most prevalent principal diagnosis (35.3 and 34.3%). Ischemic heart disease (12.3 and 12.9%), liver disease (7.3 and 9.6%), diabetes mellitus (3.8 and 2.8%) and other forms of heart disease (3.2 and 3.6%) were the leading principal diagnosis. Of total 1638 malignant neoplasms without metastasis, liver cancer was one of the leading disease categories accounting for 11.7 and 10.8% of total numbers of principal diagnosis in the development and validation data set, respectively.

View this table:
Table 1

Basic characteristics (number and percent of study subjects) in data used for development and validation of comorbidity index

Table 2 shows a total of 20 comorbidity categories to which a score is given based on the multivariate Cox regression analyses after adjusting for age and sex with backward eliminations from 70 categories. Septicemia recorded the highest score (4 points) and the score for metastatic cancer, coagulation defects, portal thrombosis and hepatic failure was 3 points. Viral hepatitis and other nine comorbidity categories had 2 points while 1 point was given to diabetes mellitus and other four comorbidity categories. For ‘chronic lower respiratory diseases (J40–47)’, scores were only assigned when a patient's age is 65 years old or above.

View this table:
Table 2

Assigned comorbidity scores for the International Classification of Disease, 10th revision (ICD-10) codes

As a result of the Kaplan–Meier method, log-rank statistics for ACI indicated a significant decrease in survival with a higher index (P< 0.001).

Results in Table 3 present discrimination abilities of ACI in the development data set. In the development data set, the C-index of the baseline model adjusting for age and sex was 0.588 (95% CI: 0.575–0.601). The C-index significantly increased with further adjustment of comorbidity index (comorbidity count, Charlson comorbidity index, Elixhauser index or ACI), but the increase in the C-index was the greatest in the ACI added model. When 10 prevalent principal diseases were additionally adjusted (baseline 2 in Table 3), the 95% CI of the C-index in the ACI model was only not overlapped with that of the C-index in the baseline 3 model. Also, the C-index in ACI models did almost not change (0.767–0.768) as the comorbidity count, Charlson comorbidity index or Elixhauser index were additionally adjusted.

View this table:
Table 3

C-index in the Cox regression models adjusting for comorbidity count (CC), Charlson comorbidity index (CCI), Elixhauser index (EI), and the newly developed comorbidity index (ACI): results in the baseline models adjusted for age, sex and 10 prevalent principal diagnoses

Results in Table 4 show discrimination abilities of ACI in the validation data set. The C-index of the baseline model adjusting for age and sex was 0.589 (95% CI: 0.568–0.609). The C-index also significantly improved with additional adjustment of the comorbidity index (comorbidity count, Charlson comorbidity index, Elixhauser index or ACI), but the improvement of the C-index was the largest in the ACI added model. When the 10 prevalent principal diseases were added to the baseline 1 model (baseline 2), the increases in the C-index were the largest when ACI was additionally included and the C-index in ACI models did almost not change (0.764–0.765) as the comorbidity count, Charlson comorbidity index or Elixhauser index were additionally adjusted.

View this table:
Table 4

C-index of the Cox regression models in the validation data adjusting for comorbidity count (CC), Charlson comorbidity index (CCI), Elixhauser index (EI) and the newly developed comorbidity index (ACI): results in the baseline models adjusted for age, sex and 10 prevalent principal diagnosis.

Table 5 shows changes in the C-index in the liver cancer patient data set. After adjustment of comorbidity count, the C-index did not significantly increase. Adjustment of the Charlson comorbidity index, Elixhauser index or ACI produced significantly increased C-indices compared with baseline models, but the greatest increase in the C-index was seen in the ACI-adjusted model. This was true after adjustment of clinical risk factors including serum albumin, serum bilirubin and prothrombin time. Moreover, as seen in the development and validation data set, additional adjustment of the comorbidity count, Charlson comorbidity index or Elixhauser index in the ACI-adjusted models made minor changes in the C-index.

View this table:
Table 5

C-index of the Cox regression models in the liver cancer data adjusting for comorbidity count (CC), Charlson comorbidity index (CCI), Elixhauser index (EI) and the newly developed comorbidity index (ACI): results in the baseline models adjusted for age, sex and clinical information

Discussion

In this study, we developed a new comorbidity index, ACI, using 10-year mortality follow-up data of 3274 hospitalized patients and validated the index against mortality follow-up data of 1403 patients who had similar conditions and 3413 liver cancer patients. Our analysis results on discriminatory abilities provided evidence that the newly developed comorbidity index is superior to other comorbidity indices in terms of comorbidity adjustment in clinical research. C-indices, a marker of discriminatory ability, in our Cox regression analyses were significantly greater in ACI added models than models adjusting for other comorbidity indices including comorbidity count, Charlson comorbidity index and Elixhauser index. In addition, additional adjustment of ACI in the development and validation data set and liver cancer patient data set made meaningful contribution to the increase in C-indices in the models adjusting for other comorbidity indices. These results have significant implications on the future use of comorbidity index because the Charlson comorbidity index has been widely used in clinical research even in Korea [1113], although the Charlson comorbidity index has been developed in the USA decades before [3].

Our analysis results also suggest possibilities that the discriminatory abilities could be better than the ability of the comorbidity index developed and used in other countries. International differences in the prevalence of a morbid condition, mortality reductions in each condition and cause-of-death structure may well contribute to the increased discriminatory ability of locally developed comorbidity indices. In fact, a score of AIDS score was 0 point because AIDS was rare disease in Korea [19, 20]. Also, metastatic cancer's score was 3 in ACI compared with 6 in the Charlson comorbidity index. This difference could be affected by mortality reduction for that period of time [21, 22] and local difference.

The C-indices from models including comorbidity index were lower than those of previous studies [6, 13, 23]. Several factors might be related with this result. The long-term follow-up or only age and sex adjustment could be related with relatively low statistics. Because various factors including management for disease, patient's compliance or other clinical or non-clinical factors could be not considered in the development process, these results might be shown. However, when development models also used 1-year mortality, the results were similar with our results; C-indices in newly developed index added models were larger than those of model including other indices. This result supports that it might need country- or region-specific comorbidity index.

We used ICD-10 codes for defining comorbid conditions in this study. We initially categorized comorbid conditions into 151 groups and then reduced the number of groups into 70 after excluding less prevalent conditions (<0.5%) and some vague or less useful codes (ill-defined causes, external causes and codes for health service use). After examining adjusted HR by each condition, scores ranging 1–4 were given to 20 conditions and for other 50 conditions zero point was assigned. Our use of ICD-10 codes for the definition of comorbid conditions would enable other researchers to use the developed comorbidity index and confirm its utility without any potential confusion about the definition of each condition. On the other hand, in the case of other indices, researchers could confuse to use the comorbidity index due to no presence of restrict disease definition [3] or definition using old version of disease classification such as ICD-9 or ICD-9-CM [6].

The ACI was developed using 10-year mortality follow-up data of 3274 patients. This sample size was large former studies. The Charlson comorbidity index was just developed from 559 patients [3] and the authors suggested further work in larger population should be needed. The cumulative illness rating scale [24], the index of coexisting disease [25] and the Kaplan index [26] were also developed using relatively small size of patients. In addition, all death cases of study participants in long term could be confirmed using death certificate data of Korean government.

There are three coding algorithms for the Charlson comorbidity index using ICD-10; however, we showed results from Quan's approach in this study, because Quan's method is reported as most comprehensive and predictive among them [18] and results from other two methods were similar with the result from Quan's which was concurrent in our data.

This study has several limitations. First, this study was conducted with patients to admitted department of internal medicine in one tertiary teaching hospital, so selection bias could affect the results and further analyses will be needed to generalize our results. However, distributions of the Charlson comorbidity index were similar with the results from former Korean studies [12, 13, 27]. Secondly, we could not differentiate comorbidity categories into complications and present of admission (POA) from non-POA because only electric information from medical record was used. Although we did not differentiate those categories, Quan et al. [6] reported that the distributions between POA and non-POA were similar, so that there was no large bias. Thirdly, we used just three clinical lab data for adjustment in liver cancer patient data set.

Despite several limitations, our results have several meaningful implications. First, ACI, a new comorbidity index, is valid to apply liver cancer patients in Korea. Secondly, ACI can be useful to apply in the ICD-10-based setting. Lastly, we suggest that each country try to develop its own comorbidity index in order to accurately evaluate their patients' status.

Funding

This study was supported by a grant (2009-480) from the Asan Institute for Life Sciences, Seoul, Korea.

Acknowledgements

We specially appreciate Prof. Sung-Cheol Yun (Department of Clinical Epidemiology and Biostatistics, Asan Medical Center, Ulsan University College of Medicine); he had many comments for this study.

Appendix

View this table:
Table 1.

Classification, distribution, hazard ratio, and score of Asan comorbidity index on ICD-10 codes

View this table:
Table 2.

Distribution of total number and death on Asan Comorbidity Index in the development dataset

References

View Abstract