International Journal for Quality in Health Care Advance Access originally published online on November 8, 2006
International Journal for Quality in Health Care 2006 18(6):395-396; doi:10.1093/intqhc/mzl057
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Editorial |
Ten reasons to conduct a randomized study in quality improvement
In the years 2004–06, the Journal has published 192 papers (excluding editorials and letters) of which eight (4.2%) were randomized experiments [1–8]. Four per cent is not enough, considering that randomization is the most direct path to a causal inference. To be effective, we need to understand what causes what in quality improvement. This aim is not compatible with weak research methods. Some quality experts claim that the evaluation of quality projects requires less scientific rigour than ‘academic’ research. It their view, small uncontrolled before/after studies or even spontaneous experiential learning is sufficient. How could that be possible? Quality improvement deals with complex systems, multifactorial interventions, unpredictable individuals, and multifaceted outcomes. If the object of inquiry is complex, the research methods must be clever and imaginative, but also rigorous. But I digress. The bottom line is: we need more randomized studies in the field of quality improvement. Here are some reasons why you should consider performing one.Randomized studies are simple and elegant
People have the notion that randomized studies are complicated, whereas observational studies are nice and simple. It is the other way around. Whatever intervention you allocate randomly will reveal its effect, or lack thereof, without artefacts (which, granted, may not be the desired outcome on occasion). By contrast, the interpretation of an observational study will require a lot of thinking about possible confounders and a long ‘limitations’ section when you write up the findings.
You can focus on the important variables
Because you will not have to worry about confounding variables, you will not have to measure them. There is a necessary trade-off between the quantity and the quality of information that can be collected in any study. In a randomized trial, you can focus on a short list of key variables, measure them with all the necessary precision, and still have a manageable dataset.
You can do the statistical analysis yourself
Another beauty of a randomized experiment is that the main analysis is extremely simple: it is a head-to-head comparison of two (or more) groups. If you can do a chi-squared test on a 2 x 2 table, or run a t-test, you will be fine most of the time (to be honest, true statistical expertise will be necessary at times, as in the case of cluster randomization, repeated measurements, many missing observations, etc.).
It is alright if the intervention is complex
The paradigm of a randomized clinical trial, the pharmaceutical phase III trial, usually examines a simplistic intervention—an antibiotic for an infection, an antidepressant for depression, etc. However, the randomized design works just as well with complex interventions. Diagnostic strategies, strategies of diffusion of innovations, and public health programmes—for any of these, the key question is ‘does it work better than usual care’, and all can be tested in randomized trials. You may not be able to say what component of a quality improvement intervention was the most critical, but that is not the most relevant question in real life. These ‘pragmatic’ randomized trials are much underused [9,10].
Blinding and placebos are not always necessary
Most pragmatic trials do not require placebos or blinding. The placebo effect is part and parcel of any intervention that aims for systemic change. People have to be motivated to work differently. Hiding the fact that change is underway does not make sense. Another reason for having a placebo is to allow blinding, i.e. an unbiased assessment of effects. There are other ways of achieving this aim: reliance on validated instruments, separation of the roles of assessor and promoter of the intervention, or use of routinely collected outcome data.
We usually think of randomizing patients, but this was done in only one of the eight trials published in this journal [4]. Random allocation of health care providers—doctors, hospitals, etc.—is often the appropriate approach for quality improvement interventions [3,5,6]. Alternatively, within an organization, time-periods can be randomly allocated to an active or a control intervention, in the manner of N-of-1 clinical trials. Moreover, randomized experiments can be performed to compare feedback methods for policy makers [1], perceptions of medical errors by the public [2], survey methods [8], or the usability of medical devices [7].
You will be a better researcher
Because an observational study does not interfere with practice, observational research makes things a bit too easy for the researcher and hence encourages sloppiness. Measure a lot of variables, cross-tabulate them back and forth, and eventually something will pop up that will be interesting. By contrast, when you randomize, you have to think hard about what the key scientific question is—you only get that one chance. And you have to commit to that research question in writing. But the hard thinking at the start of the project pays off eventually in the relevance of the results.
It is unusual to perform a randomized study and not be able to publish the results rapidly and in a reputable journal. You get instant credibility with editors and reviewers by randomizing. No reviewer of your manuscript will be able to write in all impunity: ‘this association may be due to unmeasured confounders’—the kiss of death of an observational study, because by definition what you have not measured you cannot adjust for.
You may win the Reizenstein prize
The Reizenstein prize rewards the best paper published in the International Journal for Quality in Health Care of the previous year. It is awarded based on the votes of the members of the journal’s editorial board. Two of the last three laureate papers were based on a randomized design [2,5]. Thus, the risk of winning the prize is 25% (2 of 8) if your study is randomized, and 0.5% (1 of 184) if it is not—a relative risk of 50. But then again, we did not randomly allocate these studies to be randomized or not....
Performing your first randomized trial will be exhilarating, like the first time you could ride a bicycle. Take off these training wheels!
Quality of Care Service, University Hospitals of Geneva, Geneva, Switzerland
References
- Marshall T, Mohammed MA, Rouse A. A randomized controlled trial of league tables and control charts as aids to health service decision-making. Int J Qual Health Care 2004; 16: 309–315.
[Abstract/Free Full Text] - Schwappach DL, Koeck CM. What makes an error unacceptable? A factorial survey on the disclosure of medical errors. Int J Qual Health Care 2004; 16: 317–326.
[Abstract/Free Full Text] - Verstappen WH, van Merode F, Grimshaw J, Dubois WI, Grol RP, van Der Weijden T. Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial. Int J Qual Health Care 2004; 16: 391–398.
[Abstract/Free Full Text] - Nilsson E, Ros A, Rahmqvist M, Bäckman K, Carlsson P. Cholecystectomy. costs and health-related quality of life: a comparison of two techniques. Int J Qual Health Care 2004; 16: 473–482.
[Abstract/Free Full Text] - Preen DB, Bailey BE, Wright A et al. Effects of a multidisciplinary, post-discharge continuance of care intervention on quality of life, discharge satisfaction, and hospital length of stay: a randomized controlled trial. Int J Qual Health Care 2005; 17: 43–51.
[Abstract/Free Full Text] - Bravo G, Dubois MF, Roy PM. Using goal attainment scaling to improve the quality of long-term care: a group-randomized trial. Int J Qual Health Care 2005; 17: 511–519.
[Abstract/Free Full Text] - Anderson K, Gambhir S, Glavin R, Kinsella J. The use of an anaesthetic simulator to assess single-use laryngoscopy equipment. Int J Qual Health Care 2006; 18: 17–22.
[Abstract/Free Full Text] - Heje HN, Vedsted P, Olesen F. A cluster-randomized trial of the significance of a reminder procedure in a patient evaluation survey in general practice. Int J Qual Health Care 2006; 18: 232–237.
[Abstract/Free Full Text] - Roland M, Torgerson TJ. Understanding controlled trials: what are pragmatic trials? BMJ 1998; 316: 285.
[Free Full Text] - Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003; 290: 1624–1632.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  K. Walshe Understanding what works--and why--in quality improvement: the need for theory-driven evaluation Int. J. Qual. Health Care, April 1, 2007; 19(2): 57 - 59. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||