International Journal for Quality in Health Care 15:147-154 (2003)
© 2003 International Society for Quality in Health Care
Paper |
Measuring readmissions: focus on the time factor
1Sintef Unimed Health Services Research, Trondheim
2Department of Community Medicine and General Practice, Norwegian University of Science and Technology, Trondheim, Norway
Objective. To assess the effects of choosing different time-intervals of observation when using unplanned readmissions as an outcome indicator.
Design. A conceptual model was developed based on the risk curve. The model assigned readmissions above a background level as related to the earlier episode of illness. The characteristics of the hazard curve were used to estimate how the rates of related and unrelated readmissions varied with time.
Setting. Patients living in a region of Middle Norway served by eight acute-care hospitals and discharged in the year 1996.
Main outcome measure. The conditional risk (hazard rate) of having an unplanned readmission. The information gathered allowed inclusion of readmissions to all hospitals in the area, and to make risk corrections for deaths.
Results. The identified proportion of readmissions judged as related to the earlier episode of illness was found to be very sensitive to changes in the time interval. With the commonly used interval of 30 days, 0.5 of all related readmissions were identified, while 0.7 of the readmissions included at this time were estimated as related ones (true positives). The hazard curve was different for medical and surgical patients, but the corresponding proportions of related and unrelated readmissions were relatively similar. Adjusting for deaths in the observation period did not result in significantly different risk curves.
Conclusion. When unplanned readmissions are used as an outcome indicator, the measure is susceptible to the choice of time interval. The operative characteristics must be interpreted in the context of where it is intended that the indicator should be used.
Keywords: acute care, hospital care, outcome assessment, patient readmission, time factors
There still remain challenges connected with the measurement and interpretation of readmission rates [13]. This study addresses some methodological issues that present themselves when using readmissions as an outcome indicator of the previous episode of illness or hospital stay. The focus of the present report is on the time factor, with the goal of assessing the effects of choosing different time intervals or periods of observation.
There are several earlier studies on how to construct the readmission measure and make appropriate refinements [46]. However, we find it worthwhile to note that the choice of observation period has rarely been explored in recent studies. When the time factor has been specifically addressed, the analyses have demonstrated a clear temporal relationship, with an early peak of readmissions within a few weeks of discharge [4,7,8]. In practice, a variety of different observation periods are used when unplanned readmission is applied as an outcome or performance indicator. An interval of 1 month (2831 days) after the discharge [911] is frequently chosen. Shorter periods such as 2 weeks are also used, and in some readmission studies patients have been followed for a much longer time, e.g. for 6 or 12 months [12].
When readmissions are considered to be an indicator of the outcome of previous hospital stay or episode of illness, it is essential to consider the link between the process of care and the outcome measure [9,13,14]. The temporal relationship is such a general link. It seems logical to choose a relatively short time interval after discharge as the study period. Such an approach appears to be a reasonable way of maximizing the chance of finding a measurable effect of, or association with, the previous event. Choosing a longer time frame would amplify the impact factors of the diseases natural course and community factors [15]. The specific question, however, is what is a reasonably short period of time? And how great is the impact of changing the time interval?
The aim of this study is to demonstrate the effects of choosing different time intervals. As an approach to address these matters we applied a conceptual model to analyse unplanned readmissions on the basis of the characteristics of the risk or hazard curve. This curve flattens out with time to a background level. The model assigned readmissions above this background level as those related to the earlier episode of illness. This implies that measurements at all points in time before this level is reached will include a mixture of those readmissions that are related and those that are unrelated to the previous event. At the same time there will be unidentified related readmissions (false negatives). The optimal situation would be to maximize both the absolute and relative number of related readmissions included at a given point in time. Based on the models assumptions, different components of readmissions could be estimated. The data used in the analyses allowed us to include readmissions to all hospitals in a geographic region and to make risk corrections for those patients who died during the observation period.
Study design
The conceptual model. The decay curve of unplanned readmission measured against time since discharge levels off exponentially from an early high occurrence rate, and flattens out towards a background level (Figure 1). One interpretation of this phenomenon is that it reflects the combination of two superimposed processes: the readmissions above the background level are those related to the earlier episode of illness and previous hospital stay (Figure 1, areas a and c), while the other readmissions have no such clear correlation (Figure 1, areas b and d) [7,8]. The related ones are the component we are interested in, and the aim would be to maximize the true positives identified in this way. The model implies that at all cut-off points in time before the background level is reached (Figure 1, point z), the readmissions recorded will include a mixture of those related and those unrelated to the previous event. Accordingly, choosing a long observation period will include a larger proportion of the unrelated or false positive measures. On the other hand, choosing a very short time interval will result in inclusion of only a small proportion of all related readmissions. When including an additional time-interval, the ratio of new-related to new-unrelated readmissions would depend on the point in time.
|
Materials and methods
The primary study material was a database of admissions to the hospitals in the Middle region of Norway in 1996. The population of the region is served by eight acute care hospitals, organized according to catchment areas. In principle, these public hospitals deliver all hospital services to the population of 630 000. In 1996, only 3.8% of all hospital admissions for this population occurred outside the region. It is reasonable to assume an even lower probability of being re-admitted outside the region.
Hospital admissions were organized chronologically in the data file, with the patient as the unit. The patients first admission in the year was regarded as the index admission. Variables indicating the time interval between admissions were calculated. In the hospital database, we identified 62 264 patients living in the region. A selection of patients, excluding cancer patients, admissions to rehabilitation units, and obstetric admissions, was used in further analyses of readmission. The obstetric departments were excluded because their registration of whether an admission was emergent or not was considered unreliable. Day-care patients, e.g. patients receiving dialysis, were not included. From this selection, 900 patients being transferred directly to another hospital when discharged were not considered as readmitted, and were excluded. In addition, 1032 patients who died during the index hospital stay were excluded from the population at risk. These procedures gave a study population of 46 779 patients for further analyses. Of these, 15% had an unplanned readmission following the index admission during the year. Some characteristics of the patient population are shown in Table 1.
|
To supply information about the time of death for patients dying during the observation period after discharge, it was necessary to obtain access to an additional data source. This information was extracted from the Central Population Register. The linkage of information from the two registers was performed using the unique personal identifier codes of the patients. Since deaths that occurred within the hospital were also registered in the hospital database, it was possible to cross-check the dates of the deaths extracted from the two sources. A very small discrepancy was found: there were no matches for 18 cases, and for 13 of these the date of the in-hospital death found in the hospital database was reported as being 1 day later than the date in the population register. In these cases the hospital dates were used.
Establishing levels of readmission risk
The probability of having an unplanned readmission was defined as the outcome event of interest. In this study, emergent readmissions were considered as unplanned. In the hospital database all admissions were categorized as emergent or not, where the definition of emergent was within 24 h.
To study the probability of unplanned readmissions with respect to length of time since discharge we used survival analyses. The conditional risk or hazard rate is defined as the instantaneous potential per unit time for the event to occur, given that the individual has survived up to time t [16]. A positive event was defined as the occurrence of an emergent readmission following the index admission. Consequently, in a case where an intervening planned readmission occurred before an emergent one, this was not accepted as a positive outcome. These procedures were included to maximize the link between the index (first) admission and the unplanned readmission.
To perform the survival analyses it was important to define the patients precise time of being at risk for a positive outcome. The calculations also had to adjust for the fact that the patients had different times of observation in our database. In the case of a planned second admission, this event would end the patients time at risk of a positive outcome, and the patient was removed from the study population at this point in time. Generally, when no emergent readmission followed the index admission, the time at risk was defined as the duration of the observation period. Accordingly, the use of survival analysis with adjustment for differences in time at risk is also a way of accounting for the time-window effect present in our database of admissions for one fiscal year.
To calculate patients time at risk for unplanned readmission as precisely as possible, the estimates also had to be adjusted for those dying during the observation period following discharge. When no unplanned readmissions occurred within the observation period, the patient was removed from the study population at their time of death, thus ending their time at risk.
Calculations based on the model
To be able to use the curve interpretation as the basis of estimations, the parameters of the hazard curve were calculated. To do this, a Weibull model was assumed. The hazard function is then defined by: h (t) =
p (
t)p-1, where p is the shape parameter,
is the scale parameter, and t is the time to readmission. To test the model assumption, the survival function S (t) was estimated using the KaplanMeier method, and a plot of ln [ln S (t)] versus ln (t) was made [17]. A linear relationship appeared, showing that the data can be modelled using a Weibull distribution. The model was estimated using the STATA program [18].
To complete the calculations it was also necessary to determine the point of minimal change with time, where the risk levels off to a constant background level (at point z in Figure 1). This was estimated by calculating the rate of change per unit time for the tangent line or derivative of the hazard function. This change was given by the ratio h'(t):h'(t-1). When this ratio is 1, the change in the tangent line is zero and the hazard curve is a straight line. Since the curve slowly approaches a straight line, we chose a cut-off point at 0.995 (corresponding to the time-point of 274 days for the total patient population).
According to the curve interpretation outlined earlier and illustrated in Figure 1, the effect of choosing various time intervals of observation was assessed by calculating the corresponding three components at time t: identified related and un-related readmissions (positives; Figure 1, areas a and b), as well as un-identified related ones (false negatives; Figure 1, area c). The area under the hazard curve or the cumulative hazard function [(
t)p] was the basis of these calculations.
Earlier analyses of the decay curves of readmissions have shown that the course seems to be condition specific [4,8]. To test whether the hazard curve was different for different specialities and patient groups, separate calculations were made for patients diagnosed as having a medical or surgical condition, defined from their attributed (DRG diagnosis related groups).
Results
Figure 2 demonstrates how the different components of readmission vary according to the chosen cut-off point in time. The proportion of the readmissions recorded at time t that was judged to be related to the index admission (true positives) was found to decrease with the length of time since discharge (Figure 2, line 2). At the same time, the proportion of all related readmissions we identified was found to increase with the time since discharge (Figure 2, line 1), so the separate optimal choices for the two proportions (close to the value of 1) would steer the cut-off time-point in opposite directions. Accordingly, choosing a very short time interval will identify a relatively small proportion of all related readmissions (more false negatives). On the other hand, including the late readmissions will include more false positives. At the 41-day time-point, adding another day to the time interval will include as many new unrelated as related readmissions.
|
Our aim was to identify a maximum number of related readmissions. In addition we wanted to minimize the amount of unrelated ones included at a chosen cut-off point. To illustrate the corresponding values that must be considered when choosing an optimum cut-off point, the results of a series of 10-day time-periods from 10 to 90 days after discharge are shown in Table 2. We identified a relatively low proportion of all the related readmissions in the shorter time intervals (e.g. 0.28 at 10 days). At the same time, the size of this proportion increased steeply over time (from 0.28 at 10 days to 0.79 at 90 days). Accordingly, this measure was found to be highly susceptible to the choice of observation period. Within the same time interval, the part of the identified readmissions that were true positives (related ones) varied from 0.81 to 0.57. The same phenomena can be seen in the data for the period 1090 days in Figure 2, where line 1 is relatively steep. In comparison, line 2 is flatter over this period, indicating that the reduction of false negatives with time is relatively larger than the increase in false positives. For the time interval of 30 days, the relative number of identified readmissions that were true positives was calculated to be 0.72, while the proportion of all related readmissions identified at this time was estimated to be 0.49.
|
The course of the hazard curve was also found to vary by patient group. The curve for surgical patients flattens out towards a background level that is lower than the background level for medical patients (Figure 3). The shapes of the curves for medical and surgical patients were also found to differ significantly when measured according to their calculated parameters p and
of the hazard function. When the areas under the curves are used to calculate the relevant components of readmission, the corresponding values were also found to be different for medical compared with surgical patients (Table 2). The differences were not considerable, however. At the 30-day time-point, the related readmissions or true positives constituted 0.69 of those identified at this point for medical patients and 0.78 for surgical patients, while the identified related readmissions calculated as a proportion of all related readmissions were estimated to be 0.47 and 0.50 for medical and surgical patients, respectively.
|
The hazard curves presented so far do not include risk adjustments for those patients who died during the observation period. Since many patients died after discharge, thus ending their time at risk of readmission, it is relevant to consider such an adjustment. If the adjusted curve has a very different shape, it would also have implications for the calculated components of readmissions. However, the estimated adjusted curve was found to be so close to the unadjusted one that it appeared to overlap (not shown).
Discussion
The results of this study show that modification of the observation period does have an effect on the calculation of readmissions. The proportion of all related readmissions identified at time t was particularly susceptible to variations in the interval. This proportion increased with the length of time since discharge. The longer the time interval, however, the greater the number of false positives or unrelated admissions included. At the commonly used observation period of 30 days, 0.49 of all related readmissions were identified, while 0.72 of the readmissions included at this time were estimated as true positives.
The objective of this study was not to select or recommend a specific time interval, but to demonstrate the effects of different choices. The definition of optimal operational characteristics of an indicator, and consequently in this case the optimal cut-off point in time, will also depend on the reason for its use. As discussed below, essential distinctions include use as a marker of patient outcome or of health care outcome, as well as use in internal processes of quality improvement compared with external comparison or ranking of hospitals.
The methodological assumptions upon which the study is based may be questioned. The intention was to single out unplanned readmissions with the greatest probability of being related to previous episodes of illness and hospital stay. The logic is based on the premise that cases related to the episode of care have an association in time with this previous event, and, furthermore, that there also exists a constant background level of probability for emergent readmission where time is not a determinant. We have, however, not tested the validity of these basic assumptions.
The association or link between the previous episode of care and the event of readmission is considered a fundamental element. To maximize such a link, we placed a condition of successive chronology between an index admission and the following unplanned readmission, accepting no intervening planned readmission. Because readmission is not a direct measure of the outcome or quality of care, it is essential that it is linked to process of care [13,19].
There are several interpretations of being related to or associated with the previous event. Alternative explanations include that of patients having a recurrence or progression of the illness, or not being cured as expected. It can mean having a complication, that in turn can be related or not related to substandard care; and it can also include patients unable to cope with their situation after discharge, for which, in turn, they might or might not have been better prepared. Unplanned readmissions are symptoms of a poor patient outcome, but not necessarily of a poor health care outcome or of poor quality. Quality of care is only one of many prognostic factors. The assessment of causality is complex, including the existence of multiple prognostic and causal factors [20]. Furthermore, when attributing an outcome to the characteristics of the care, there is also a question of concurrent care, which in this context is hospital versus community care, or the interface or cooperation between them.
To search for support for the thesis of time dependency, it is relevant to consider studies with different methodological approaches. A study that analysed the effects of hospital factors on elderly patients risk of readmission used both short and long observation periods in modelling the outcome [15]. Several hospital factors were found to have significant effects on readmission risk when a time interval of 30 days was used, but the same set of factors was found to be insignificant when the outcome measure used was readmissions that occurred in the 90180 day period following discharge.
More qualitatively based studies are also relevant in this context. Several studies have approached the matter by making retrospective assessments to select those readmissions that were caused by substandard care, and particularly the subgroup judged as preventable. Two such studies found a larger proportion of avoidable cases among the readmissions occurring very early on [21,22]. In one of these studies only 9% of the readmissions were considered preventable, whereas 75% were considered as being related to the condition causing the previous hospitalization [22]. These results were found when studying an observation period of 30 days. Even if these results cannot be compared directly to our findings, it is interesting to note that we found 0.72 of the measured readmissions to be related to the index admission using the time interval of 30 days.
In the study by Frankl et al. [22], two thirds of the readmissions were classified as related to general problems such as miscalculation of readiness for discharge, poor planning of the follow-up, or miscommunication. Such general problems do not seem to be condition-specific. However, the question of condition-specific results has to be considered. Our findings confirm that the course of the hazard curve varies significantly according to patient group for the major subgroups tested (medical versus surgical DRG). When calculating the relationship between the corresponding components of readmissions, however, the results were relatively close. Consequently, the results might be considered relatively general, and accordingly applicable when a single, general measure is wanted. However, there are certainly situations in which one may like to focus on known complications or adverse events, related to certain diseases or procedures, that would demand a very specific follow-up period.
The effect of not adjusting the readmission risk for patients dying during the observation period was also considered in the analyses and found not to be significant. But if one should add the deaths after discharge with readmissions into a combined measure of poor outcome, then the effect would be greater and would vary systematically with age.
Since the material included in this study represents a 1-year time window, there will be relatively few patients who have been observed for the longer time intervals. Even if the fact of differences in observation (risk) time is accounted for in the survival analyses, this makes the estimations in the upper period of the study time more uncertain. In particular, it may affect the estimation of the background level. It can be added that the estimations were repeated, choosing different values for the background in order to test the sensitivity of the results without finding significant effects on the major trends. However, the analyses should be repeated on material that allows more or all of the patients to be followed for 1 year post-discharge. The results may also be different for different health care systems.
In the care system used as the context for this study, the emergent readmissions are meant to be interpreted as markers of poor outcome, and to be used in a stepwise process where supplementary methods are needed to separate those cases that represent true quality problems. In the process of quality improvement, it is not sufficient to know that a problem exists; an analysis of the process of care must follow. To be able to act and find solutions, it is necessary to obtain more specific knowledge [23]. One of the consequences of using readmissions as an internal indicator would be that relatively higher proportions of false negative cases could be tolerated, while a high proportion of those cases reported as true positives would make the analytical and qualifying process more cost-effective.
These considerations may be rated differently when using readmissions to compare or rate the performance of different hospitals directly. When using such findings to identify hospitals with an increased probability of quality problems, the sensitivity becomes relatively more important since it would be desirable to detect a large proportion of the problem hospitals. At the same time, a low positive predictive value would suggest that many of the hospitals singled out were, in fact, average ones. The choice of time interval may not be the most important methodological problem when using readmission rates in the ranking of units, however. Additional challenges include factors such as threshold values and sample sizes [24]. Furthermore, in comparisons there is a need to keep all other factors of variation reasonably constant, for instance differences in case mix. There is also the question of how large quality differences have to be for the measure to discriminate between hospitals, and what the outcome would be in the case of normative or superior care given to all patients [6].
There is currently a methodological discussion concerning outcome indicators such as readmissions used in the comparison or ranking of hospital quality [2527]. A simulation study to evaluate the use of early readmissions to identify poor quality hospitals was made by Hofer and Hayward [28]. In the simulations, when systematically varying the conditions, the authors found generally low sensitivity values for the readmission indicator. In addition they found low positive predictive values that were very sensitive to other factors of variation, such as the case mix.
When considering the practical applications of our study, it is necessary to specify the setting. Our context was restricted to readmissions used as a screening tool to be analysed further in the hospitals internal processes of quality improvement. If a single measure that can be applied to a general group of patients is preferred, one question to ask is whether the results support the choice of the commonly used time interval of 1 month. It may be considered acceptable that 72% of the readmissions included at this point in time are related ones or true positives, while the identified proportion of all related readmissions would be relatively low (49%). A crucial question in these considerations would be whether the false negative cases of related readmission represent quality problems that are different from those represented by the true positive cases identified. If there is no time-based association with type of quality problem, then one could tolerate a relatively large number of false negatives. This would also reduce the cost of follow-up investigations from the screening. Furthermore, even if the estimates vary by patient group, the difference is not large when specification of different groups is as crude as medical or surgical DRG. Including information on deaths after discharge will not affect the calculation of readmission risk significantly.
We wish to thank colleagues at Sintef Unimed Norwegian Patient Register for assistance in linkage of the different data sources used in the study. The study was supported by a grant from The Norwegian Research Council (project no. 116712/330).
Address reprint requests to Dr Torhild Heggestad, Sintef Unimed Health Services Research, N-7465 Trondheim, Norway. E-mail: torhild.heggestad{at}sintef.no ![]()
Accepted for publication December 13, 2002.
References
- Hasan M. Readmission of patients to hospital: still ill-defined and poorly understood. Int J Qual Health Care 2001; 13: 177179.
[Free Full Text] - Gray L. Readmission of elderly patients to hospital: still ill-defined and poorly understood: a response. Int J Qual Health Care 2001; 13: 181182.
[Free Full Text] - Weissman JS. Readmissions: are we asking too much? Int J Qual Health Care 2001; 13: 183185.
[Free Full Text] - Chambers M, Clarke A. Measuring readmission rates. BMJ 1990; 301: 11341136.
- Henderson J, Goldacre MJ, Graveney MJ, Simmons HM. Use of medical record linkage to study readmission rates. BMJ 1989; 299: 709713.
- Milne R, Clarke A. Can readmission rates be used as an outcome indicator? BMJ 1990; 301: 11391140.
- Henderson J, Graveney MJ, Goldacre MJ. Should emergency readmissions be used as health service indicators and in medical audit? Health Serv Manage Res 1993; 6: 109116.[Medline]
- Sibbritt DW. Validation of a 28 day interval between discharge and readmission for emergency readmission rates. J Qual Clin Pract 1995; 15: 211220.[Medline]
- Ashton CM, Wray NP. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc Sci Med 1996; 43: 15331541.
- Department of Health. NHS Performance Indicators. Available at http://www.doh.gov.uk/nhsperformanceindicators/2002/trust.html (accessed September 2002).
- The Center for Performance Sciences International Quality Indicator Project. Available at http://www.internationalqip.com (accessed September 2002).
- Benbassat J, Taragin T. Hospital readmissions as a measure of quality of health care. Advantages and limitations. Arch Intern Med 2000; 160: 10741081.
[Abstract/Free Full Text] - Ashton CM, Del Junco DJ, Souchek J, Wray NP, Mansyur CL. The association between the quality of inpatient care and early readmission. A meta-analysis of the evidence. Med Care 1997; 35: 10441059.[CrossRef][Web of Science][Medline]
- Hammermeister KE, Shroyer AL, Sethi GK, Grover FL. Why it is important to demonstrate linkages between outcomes of care and processes and structures of care. Med Care 1995; 33 (suppl. 10): OS5OS16.[Web of Science][Medline]
- Heggestad T. Do hospital length of stay and staffing ratio affect elderly patients risk of readmission? A nation-wide study of Norwegian hospitals. Health Serv Res 2002; 37: 647665.[CrossRef][Web of Science][Medline]
- Kleinbaum DG. Survival Analysis. New York: Springer-Verlag, 1996.
- Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. New York: Wiley, 1980.
- STATA. Reference Manual, Release 6. College Station, TX: Stata Corporation, 1999.
- Slack R, Bucknall CE. Readmission rates are associated with differences in the process of care in acute asthma. Qual Health Care 1997; 6: 194198.
[Abstract/Free Full Text] - Gulliford MC. Evaluating prognostic factors: implications for measurement of health care outcome. J Epidemiol Community Health 1992; 46: 323326.
[Free Full Text] - Clarke A. Are readmissions avoidable? BMJ 1990; 301: 11361138.
- Frankl SE, Breeling JL, Goldman L. Preventability of emergent hospital readmission. Am J Med 1991; 90: 667674.[Web of Science][Medline]
- Kiefe C. Predicting rehospitalization after bypass surgery. Can we do it? Should we care? Med Care 1999; 37: 621624.[CrossRef][Web of Science][Medline]
- Ansari MZ, Collopy BT, McDonald IG. Establishing thresholds for adverse patient outcomes. Int J Qual Health Care 1996; 8: 223230.
[Abstract/Free Full Text] - Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc Ser A Stat Soc 1996; 159: 385409.[CrossRef]
- Russell, E. The ethics of attribution: the case of health care outcome indicators. Soc Sci Med 1998; 47: 11611169.
- Weissman JS, Ayanian JZ, Chasan-Taber S, Sherwood MJ, Roth C, Epstein AM. Hospital readmission and quality of care. Med Care 1999; 37: 490501.[CrossRef][Web of Science][Medline]
- Hofer TP, Hayward RA. Can early readmission rates accurately detect poor-quality hospitals? Med Care 1995; 33: 234245.[CrossRef][Web of Science][Medline]
This article has been cited by other articles:
![]() |
Lin Guo, E. S. Chung, D. E. Casey JR, and R. Snow Redefining Hospital Readmissions to Better Reflect Clinical Course of Care for Heart Failure Patients American Journal of Medical Quality, March 1, 2007; 22(2): 98 - 102. [Abstract] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||



