International Journal for Quality in Health Care 14:119-125 (2002)
© 2002 International Society for Quality in Health Care
In-patient interventions supported by results of randomized controlled trials in Japan
Department of General Medicine and Clinical Epidemiology, Kyoto University Hospital and Graduate School of Medicine, Kyoto City, Japan
Objective. To determine to what extent the results of randomized controlled trials (RCTs) support medical interventions for in-patients at the department of general medicine of a university hospital in Japan.
Design. Retrospective analysis. By reviewing discharge summaries, two physicians first independently decided on patients’ respective primary problems at admission and up to two secondary problems. Next, up to five interventions for the primary problem and one intervention for each of the secondary problems were selected. Differences of opinion (if any) between the two physicians regarding these selected interventions were resolved by discussion. MEDLINE and/or the Cochrane Library were used as data sources for literature regarding the selected interventions.
Setting. A ward of the department of general medicine of a Japanese university hospital, 1995–1997.
Main measures. The proportion of therapeutic interventions supported by RCT results and associated meta-analyses.
Results. For the primary problems, 103 (48.8%) of the 211 most important interventions were supported by the results of RCTs, as were 47.8% of all the interventions including the most important and the adjunctive ones. Furthermore, 56.2% of the most important interventions for the secondary problems were also supported by the RCT results.
Conclusions. Approximately half of the therapeutic interventions performed at an academic medical inpatient unit in Japan were RCT-supported. This was true not only for the most important interventions for primary problems but also for the adjunct interventions for the primary problems and the interventions for secondary problems.
Keywords: evidence-based medicine, medical practice in Japan, randomized controlled trials
 |
Introduction |
|---|
 Top Introduction MethodsResultsDiscussionReferences |
|---|
In an era of limited resources available for health care systems, the importance of evidence-based medicine (EBM) has been attracting much attention worldwide as a means to allocate resources more rationally and, at the same time, to improve the quality of care offered to patients. In spite of early pessimistic views [1, 2], recent reports have shown that most of the decisions regarding treatments provided in the context of EBM are in fact supported by evidence from the medical literature, thus further promoting EBM in clinical practice [3, 4]. However, the generalizability of these findings is debatable, not only because these reports address only the single most important treatment for patients’ primary problems, but also because they were conducted in the United Kingdom or North America, both of which have a common background in medical practice. The latter issue is perhaps of greater importance for physicians in other countries such as Japan, because of the significant differences in medical practice and attitude toward health care utilization that exist compared with western countries [5–7]. We therefore assessed retrospectively to what degree the therapeutic interventions conducted for in-patients at a department of general medicine of a university hospital in Japan were supported by the results of randomized controlled trials (RCTs).
|
Methods |
|---|
|
TopIntroductionMethodsResultsDiscussionReferences |
|---|
By reviewing discharge summaries of in-patients who were discharged between January 1995 and December 1997, two physicians (HK and KM) independently selected the patients’ primary problem at admission and up to two secondary problems. (At the time of the study, HK had 12 years of experience as a pulmonologist and several years as a general internist, while KM had worked as a general internist for 12 years. Both of them had spent several years studying clinical epidemiology.) The problems most likely to influence survival or functional status were selected first, and, in the absence of such problems, conditions that caused patients the most suffering were selected. For patients admitted more than once due to the same primary problem, only the summaries of the initial admission were examined. Two cases, whose interventions during hospitalization for their primary problems had already been initiated at an outpatient clinic, were excluded because the staff on the ward were not primarily responsible for the therapeutic decision made at an outpatient clinic. Next, the two physicians selected up to five interventions for the primary problems and one intervention for each of the secondary problems. They also designated the single most important intervention for the primary problems. Differences of opinion (if any) between the two physicians during this selection process were resolved by consensus. Neither of the physicians who selected the problems and interventions among in-patients was a member of the departmental staff during the period when the subject patients were hospitalized.
The quality of evidence was primarily classified on the basis of whether it was supported by the results of RCTs. When at least one randomized placebo-controlled trial showed statistically significant superiority of an intervention or a randomized head-to-head comparison demonstrated at least equivalent efficacy, the quality of evidence for that particular intervention was regarded as RCT-supported. Positive results of at least one meta-analysis based on RCTs were also classified as RCT-supported. When there were conflicting RCT results and no meta-analysis, the intervention was not rated as RCT-supported. Whenever the validity of the study or applicability of the evidence was called into question, the content of the full text was appraised.
There are many interventions, the effectiveness of which are thought to be clear enough for conducting RCTs to be widely considered either unnecessary or unethical, or both. Therefore, we created a category entitled ‘widely accepted’ for this study. A given intervention was judged to be in this category when all investigators agreed that it was important and effective beyond reasonable doubt, in spite of the absence of supportive RCT results.
We used MEDLINE (January 1966 to end of 1999) and/or the Cochrane Library [8] to conduct the search for evidence. When a screening search for evidence using MEDLINE indexing terms failed to detect an RCT, free text terms were used.
For the primary problems and the most important interventions for them, the coincidence rate between the two physicians (HK and KM) was defined as the number of cases for which both physicians selected the same problems or interventions divided by the total number of eligible cases. Since it was not possible to calculate kappa coefficients because of the large number of possible combinations, a coincidence score was created to assess the agreement regarding up to five for the primary problems and up to three major problems. To determine this score, an agreement rate for each case was first calculated by dividing the number of problems or interventions selected by both physicians by the larger number of problems or interventions that either of them selected, and expressed as a percentage. For example, if HK selected one problem and KM selected three problems including one that HK selected, the agreement rate for that case was 33%. The coincidence score was defined as the average of these agreement rates.
|
Results |
|---|
|
TopIntroductionMethodsResultsDiscussionReferences |
|---|
Between January 1995 and December 1997, 300 patients were admitted to the General Medicine ward; 102 males (average age 52.0 ± 19.4, range 13–90, median 54.5 years) and 132 females (average age 54.8 ± 19.2, range 14–90, median 58 years) were included in the study. The primary problems of these patients are shown in Table 1.
|
Agreement between two physicians
Coincidence rates were 88.9% for selection of the primary problems and 78.3% for deciding the most important interventions for the individual primary problems. The coincidence scores were 74.4% for choosing up to five interventions for the primary problems and 71.4% for the selection of up to three problems of individual patients.
Primary problems
For the study population of 234, 132 different types of problems were identified as primary problems, for which a total of 425 interventions (of 328 types) were performed. The number of interventions for the primary problem in each patient was one for 128 patients, two for 52, three for 32, four for 12, and five for 10 patients. Some of these interventions were not endorsed by RCT results and it was difficult to judge whether they were justifiable or not. These included rehydration for dehydration, referral to speciality departments or hospitals, and observation without treatment for several conditions. Whether these interventions are justifiable or not depends on the severity of the underlying conditions, which makes the judgement inevitably subjective. We therefore excluded these interventions from the analysis when more than half of the investigators agreed (Table 2). When doctors in charge of the ward decided to transfer a patient to another department for a particular treatment (for example, referral for chemotherapy for thyroid cancer), evidence in support of that treatment was searched for.
|
As a result, we searched for supportive evidence for 389 interventions (297 types), as shown in Figure 1. Of the 211 most important interventions for the primary problems, 103 were supported by RCT evidence (48.8%), and 39 interventions (18.5%) were judged as being widely accepted (Table 3). Of the 178 adjunct interventions for the primary problems, 83 (46.6%) were RCT-supported and 15 (8.4%) were classified as widely accepted (Table 3).
|
Secondary problems
Among the 289 secondary problems (Table 4), 63 interventions consisted of rehydration, referral, or observation without therapeutic intervention and were excluded from the analysis. Of the remaining 226 interventions, 127 (56.2%) were supported by RCT evidence, and 41 (18.1%) were classified as widely accepted. All interventions classified as widely accepted are listed in Table 5.
|
|
Table 6 lists the most important interventions for the primary and secondary problems that were supported by results of RCTs and selected by researchers more than once as the most important intervention, while Table 7 shows those that were not supported by RCTs. In addition, there were several interventions for which we found the RCTs against their use, but these are not listed in Table 7 because they were secondary interventions for the primary problems or were selected only once. They comprise intravenous aminophylline infusion for acute exacerbation of asthma (n = 5), alprostadil (prostaglandin E1) for arteriosclerosis obliterans, amphotericin B gargle for thrush, diet therapy for colonic diverticulitis, albumin administration for hypoalbuminemia, haloperidol for delusion associated with major depression, and use of a rib belt for a rib fracture (n = 1 for each).
|
|
|
Discussion |
|---|
|
TopIntroductionMethodsResultsDiscussionReferences |
|---|
The study presented here shows that approximately half of the therapeutic interventions for the primary problems among our in-patients were supported by the results of RCTs or of meta-analyses of RCTs. This was true not only for the most important interventions but also for the adjunct therapies. In addition, most of the decisions regarding the secondary problems were also supported by RCT evidence.
This study has several methodological advantages over previous studies conducted in the UK and Canada [1, 2], which assessed only the single most important intervention for the primary problem. We selected up to three important clinical problems for each patient and up to five interventions for the primary problem. This is very likely to be a more accurate reflection of actual clinical practices, thus leading to better generalizability of the study results. In addition, this is the first study of this kind performed in Japan and the findings here will prove to be more generalizable internationally when combined with the results from the UK and Canada.
The proportion of RCT-supported interventions for the primary problems in this study was somewhat lower than those previously reported; Ellis et al.[1] found that 53% of the primary treatments were supported by RCT results, and Michaud et al.[2] that this was true for 57%. This discrepancy is probably even more significant when we take into account the recent rapid increase in clinical trials, coupled with the development of search technologies. For example, approximately 128 000 papers listed in MEDLINE had been classified as RCTs by publication type by February 2000, and nearly 48 000 of these were published in the last five years. The discrepancy between previous studies and ours could be due to our incomplete search method, as discussed later. Alternatively, interventions conducted at our academic institution may be somewhat less RCT-supported at present, but not far behind.
There are a few problems regarding the generalizability of the present results. Firstly, our department accommodated only a small number of patients at the time of study, so physicians were able to spend more time on individual patients’ care than were those in community hospitals. Secondly, the staff physicians of our department were familiar with the concept and procedure of EBM because of their academic background. Thus, the results of this study do not necessarily reflect the present status of clinical practice throughout Japan. However, these findings have shown that practising EBM is feasible within the health care system unique to Japan. Finally, this study was performed in the department of general medicine of a university hospital in Japan. The health care system in Japan allows patients to visit any clinic or hospital, and most patients come to our hospital by themselves without referral. However, many did so after visiting other clinics or hospitals because our institution is widely recognized as a tertiary care hospital. As a result, distribution of diseases or conditions may differ from that found at usual hospitals in Japan.
Other shortcomings may be related to limited databases for evidence searches and inadequate critical appraisal of the literature. The databases we used were limited to MEDLINE and the Cochrane Library, which we believe to be the most practical in a normal clinical setting. Indexing in MEDLINE has been shown to be imperfect [9], and it does not record all relevant publications. For example, reports on drugs by Japanese pharmaceutical companies are less likely to be published in journals cited in MEDLINE, although this was the case in only a few instances in this study. This could have led to an underestimation of the proportion of RCT-supported interventions. On the other hand, we appraised retrieved literature critically only when we thought it necessary. We may have thus spuriously endorsed interventions as RCT-supported, when, for example, the characteristics of populations for individual RCTs were different from those of our patients. This is not a genuine practice of EBM, in which not only literature searches but also critical appraisal of the literature and judicious application of the evidence are required. However, evidence already appraised critically was used in most cases without re-appraisal. In addition, the nature of a retrospective study makes it almost impossible to determine whether every intervention was most appropriate for a particular patient in his/her specific physical and psychosocial situation at that time. In addition, as this study was based on reviews of discharge summaries, there could be a bias toward overestimation of the proportion of RCT-supported interventions, since interventions supported by solid evidence may well be more likely to be recorded. A prospective study is therefore necessary to clarify this point, although participation in such a prospective study could influence the practice pattern and might not reflect clinical reality.
Finally, there is some disparity between the time of the interventions included in this study and the dates of the RCTs used as supporting evidence. This disparity could be significant because of the previously mentioned rapid growth of reported RCTs. However, the primary purpose of this study was not to evaluate how consciously EBM was practised, but to determine to what degree the therapeutic interventions in Japan were supported by RCTs and thus to get some insights about the feasibility of EBM in Japan.
In conclusion, about half of the therapeutic interventions performed at an academic medical in-patient unit in Japan were supported by RCT results. This was true not only for the most important interventions for primary problems but also for the adjunct interventions for the primary problems or interventions for secondary problems. This study has therefore demonstrated that practising medicine based on high quality evidence supported by the literature is not just a theoretical concept but a realistic goal.
Accepted for publication November 16, 2001.
|
References |
|---|
|
TopIntroductionMethodsResultsDiscussionReferences |
|---|
- Office of Technology Assessment of the Congress of the United State. Assessing the Efficacy and Safety of Medical Technologies. Washington, DC: US Government Printing Office, 1978.
- Smith R. Where is the wisdom...? The poverty of medical evidence. Br Med J 1991; 303: 798–799.
- Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. Lancet 1995; 346: 407–410.
[Medline] - Michaud G, McGowan JL, van der Jagt R, Wells G, Tugwell P. Are therapeutic decisions supported by evidence from health care research? Arch Intern Med 1998; 158: 1665–1668.
[Medline] - Hira K, Fukui T, Endoh A, Rahman M, Maekawa M. Influence of superstition on the date of hospital discharge and medical cost in Japan: retrospective and descriptive study. Br Med J 1998; 317: 1680–1683.
[Abstract/Full Text] - Asai A, Maekawa M, Akiguchi I et al. Survey of Japanese physicians’ attitudes towards the care of adult patients in persistent vegetative state. J Med Ethics 1999; 25: 302–308.
[Abstract] - Asai A, Fukuhara S, Lo B. Attitudes of Japanese and Japanese-American physicians towards life-sustaining treatment. Lancet 1995; 346: 356–359.
[Medline] - The Cochrane Collaboration. The Cochrane Database of Systemic Reviews. In The Cochrane Library, Oxford, UK: The Cochrane Collaboration.
- Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. Br Med J 1994; 309: 1286–1291.
[Abstract/Full Text] - Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. Lancet 1995; 346: 407–410.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||