International Journal for Quality in Health Care 14:183-198 (2002)
© 2002 International Society for Quality in Health Care
Preventing drug-related morbidity—determining valid indicators
1School of Pharmacy and Pharmaceutical Sciences, University of Manchester, Manchester, UK and
2Department of Pharmacy Health Care Administration, University of Florida, Gainesville, FL, USA
Address reprint requests to Caroline J. Morris, The Drug Usage and Pharmacy Practice Group, School of Pharmacy and Pharmaceutical Sciences, University of Manchester, Oxford Road, Manchester M13 9PL, UK. E-mail: caroline.j.morris{at}man.ac.uk
Objective. To describe the process that is being undertaken to validate a series of indicators for preventable drug-related morbidity—originally developed in the US—for application in the UK health care system.
Design. A two-round Delphi questionnaire survey after a preliminary validation of the indicators within the University of Manchester School of Pharmacy.
Setting. A primary care study set in the UK.
Study participants. A purposively selected sample of general practitioners with a specific responsibility for prescribing-related issues (n= 6) and pharmacists actively involved in medication review in primary care (n= 10).
Main outcome measures. The establishment of consensus among the participants that an indicator reflected preventable drug-related morbidity in primary care.
Results. After preliminary validation, 37 of the original 57 US indicators were retained. The Delphi panel generated 16 additional new indicators in the first round. At the end of the second round, the pre-defined level of consensus was reached for 29 indicators (19 of the US generated indicators; ten generated by the panel in the first round).
Conclusions. The Delphi results highlighted differences in both the clinical perspective and, possibly, philosophical viewpoints of health professionals practising in the UK and US health care systems. Further work, located in both primary and secondary care, is now in progress to operationalize. This process will form a key part of the refining, and hence further validation, of the indicators. The future development of prospective medical-record-based indicators should facilitate a reduction in the human, clinical, and economic burden of drug-related morbidity.
Keywords: adverse drug events, Delphi technique, preventable drug-related morbidity, primary care, United Kingdom, United States
Measuring and assessing the quality of health care services is an issue of fundamental international importance. Although expressed in different terms, it is now firmly on the health policy agenda in both the United States of America (US) [1] and United Kingdom (UK) [2] in the form of quality improvement and clinical governance respectively. The specific areas of medical error and adverse events have attracted considerable international media attention, whilst major initiatives to facilitate improvements in patient safety have been started in the US, the UK, and Australia. It is notable that the Institute of Medicine in the US [3], the Department of Health in the UK [4], and the Australian Council for Safety and Quality in Health Care [5] have all identified the potential value of a systems approach to addressing medical error.
Reliable studies of adverse events (usually considered to be injuries caused by medical management, which result in a prolongation of hospital admission or disability) are limited. However, benchmark studies from the US [6, 7] and Australia [8] have identified adverse event rates of 3.7% and 16.6% of hospital admissions, respectively. In the latter study, half of the events were considered to be preventable [8]. Furthermore, a recent UK study [9] identified that about 10% of patients admitted to two acute hospitals experienced an adverse event, half of which were deemed preventable. The main causes of adverse events in secondary care relate to surgical procedures, medical procedures, diagnosis, and drugs. The prescription of a drug represents the most common health care intervention. Thus, targeting research into reducing medical error from the perspective of drug-related morbidity (DRM) represents one important way of improving both the safety and the quality of health care. Indeed, adverse drug events have been studied extensively. It has been estimated that in 1994 over 100 000 US hospital patients experienced fatal adverse drug reactions [10]. In a study by Bates et al. [11], 6.5 adverse drug events and 5.5 potential adverse drug events were estimated per 100 hospital admissions. Indeed, the medical literature is littered with examples of injurious outcomes resulting from drug therapy. Furthermore, drug-related problems have been identified as the reason for hospital admission in a number of studies. [12–14]. The adverse human and clinical outcomes of DRM are potentially substantial, while the consequences in ambulatory-care patients in the US have been estimated to cost 76 billion US dollars [15] and, more recently, 177 billion US dollars [16] each year.
It is notable that research focusing on the issue of drug-related hospital admissions from a primary care perspective is limited. This is despite the fact that the majority are likely to be caused by primary care prescribing. Cunningham et al. [14] used targeted, written doctor and patient educational information as intervention strategies to help reduce drug-related hospital admissions. However, these were not demonstrably effective. Otherwise, there has been little demonstrable interest in devising systems in primary care to reduce drug-related hospital admissions.
Reducing preventable drug-related morbidity (PDRM) should have a positive impact on the quality of life of patients, safety of the health care system [17], and the efficient use of health care resources. For a DRM to be preventable it must have been preceded by a drug-related problem that is both foreseeable and controllable [18].
An American research programme, embedded in the systems approach, has therefore defined PDRM and produced operational definitions of PDRM in older people [19]. However, as in the research area of the appropriateness of prescribing, direct transcription of North American findings to the UK is not necessarily valid. For example, criteria, developed in the US by Beers [20], to identify the inappropriate use of medications in older populations, have not been utilized in the UK because of their lack of relevance to UK prescribing.
In light of both the national and international importance of DRM, we aim to describe the process that is being undertaken to explore and validate US-derived indicators for application in the UK health care system.
 |
Methods |
|---|
 Top Methods ResultsConclusionsReferences |
|---|
US-derived definitions
The US definitions of PDRM are represented as clinically specific indicators. These were developed from an English-language literature review of peer-reviewed medical articles and referenced texts for types of PDRM from 1967–1998, and from a consensus panel of seven experts in geriatric medicine [19]. These indicators all take the form of an adverse therapeutic outcome and an associated pattern of care that led to the event. They are shown in full in Appendix 1.
Validation for application in the UK
Preliminary validation. In order to determine the relevance of the US-derived definitions to UK primary care prescribing, the face validity of each individual definition was initially determined within the University of Manchester School of Pharmacy. This process was undertaken by three experienced clinical pharmacists, who were asked to rate, on a three-point scale (relevant, unsure, or irrelevant), how relevant they considered each definition to be to primary care prescribing for older people in the UK. Definitions were deleted when the majority view considered it to be irrelevant. For those definitions where no clear agreement was identified, the raters met to discuss the definitions further and to establish consensus. On completion of this process, we assessed the content validity of the remaining definitions by cross-checking with the then current British National Formulary (BNF) guidelines [21]. This text, published under the authority of a Joint Formulary Committee comprising representatives of the British Medical Association, the Royal Pharmaceutical Society of Great Britain, and the Department of Health, is considered to be the UK ‘gold standard’ for the rapid reference of prescribing information. It provides clear, up-to-date, evidence-based information about the selection and use of medicines prescribed in the UK.
Delphi questionnaire survey. In order to formally assess the face and content validity of the remaining indicators, develop consensus, and identify additional indicators of specific relevance in the UK, we conducted a Delphi questionnaire survey [22, 23]. This technique is used for decision making among isolated, anonymous respondents. It helps define levels of agreement in areas prone to debate, guiding opinion towards a final decision through feedback and reflection. The Delphi process used in this study is summarized in Box 1.
A primary care group in England is a collaborative group of general practitioners (GPs), serving a population of around 100 000. Each group has a limited annual prescribing budget and a ring-fenced budget for providing general medical services and developing primary care. A designated GP (the prescribing lead) usually takes responsibility for prescribing-related issues. The Delphi panel comprised a combination of primary care group GP prescribing leads and primary care pharmacists directly involved in medication review in GP practices. We sent a letter explaining fully the Delphi technique, the time-scale involved, and an invitation to participate in the study, together with a pro-forma reply to 20 GPs, selected from a reference source for primary care contacts (Medendium)[24]. We recruited pharmacists by initially utilizing personal contacts and then employing the ‘snowballing’ technique [25]. One of the authors (CJM) approached all pharmacists by telephone and gave them full details of the study. If they were willing to participate we followed this information up by letter. In order to facilitate recruitment we made a payment of £100 to each participant on completion of the study, in recognition of their professional input and time. We provided a final summary of findings for all participants.
We anglicized the terminology and spelling of the US definitions remaining after preliminary validation before incorporating them into the Delphi questionnaire. We presented them in a ‘user-friendly’ format as a series of clinical scenarios. Each scenario specified either a drug therapy or clinical situation in a patient aged over 65 years in the form of patient management and an associated adverse therapeutic outcome(s). We asked the participants to consider each scenario and rate it on a seven-point scale, ranging from 1 = definitely PDRM to 7 = definitely not PDRM, whether they considered that it fitted the definition of PDRM provided (see Box 2). We provided space with each scenario for any additional comments by each respondent. We also added a number of reliability indicators; four were designed not to represent PDRM and two indicators were identical. In addition, we asked the participants to identify any other PDRMs, not included in the clinical scenarios, which they considered to be of particular significance in older people in the UK. We piloted and refined the questionnaire using four local pharmacists.
The number of Delphi rounds was fixed at two at the outset. A summary of the findings from the first round was prepared in the following three weeks and individualized results fed back to the participants. This comprised a grid showing the collated responses from the first questionnaire and a synopsis of any qualitative comments made, together with that individual’s own response. In light of this information, we asked the participants to re-rate each clinical scenario. In addition, we asked the participants to rate a number of new clinical scenarios that had been generated in the first questionnaire. This survey was conducted during March and April 2000.
The following definitions of consensus were fixed at the outset of the study:
- Consensus that a clinical scenario represented a PDRM was established if 75% or more of the panel scored the item 1, 2, or 3.
- Consensus that a clinical scenario did not represent a PDRM was established if 75% or more of the panel scored the item 5, 6, or 7.
No consensus was achieved if the item failed to meet the above criteria.
|
Results |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
Preliminary validation
The 51 indicators derived in the US are shown in Appendix 1. Two of the indicators (Appendix 1; numbers 7 and 24) each represented two distinct indicators and thus were treated as four different indicators. In addition, because of established monitoring practices for anti-epileptic drug therapy in the UK, indicator numbers 19 and 40 were treated as six different indicators. Therefore, the starting point for the UK work was a total of 57 indicators.
Ten indicators were immediately deleted after the preliminary validation stage, due to a unanimous opinion that they were not relevant to UK primary care prescribing (Table 1). These included indicators that encompassed drug therapies not used, little used, or irrelevant to primary care in the UK. For example, barbiturates are seldom used in the UK, while tetracycline has limited use. Aminoglycosides, being administered by injection for serious systemic infection, have no relevance to UK primary care prescribing.
|
Further discussion was required for another ten indicators, as the majority view was unsure (n = 6) or because each rater expressed a different opinion (n = 4). It became apparent that the lack of consensus stemmed from a reticence to eliminate an indicator purely on prevalence grounds. However, on discussion it was agreed that these ten indicators be deleted, as their prevalence was likely to be minimal (Table 1).
After preliminary validation, 37 of the original indicators remained. The final version of the indicators used in the Delphi questionnaire is shown in Appendix 2, together with a cross-reference to the original US indicator and comments denoting any modifications made. It can be seen from Appendix 2 that the content validity check against the BNF [21] resulted in slight changes to the wording of three indicators (numbers 6, 15, and 34).
Delphi questionnaire survey
Eleven GPs returned the pro-forma reply form, of which eight were willing to participate. One GP changed his mind on receipt of the first questionnaire, while another was unable to participate due to holiday commitments. In total, six GPs (five male and one female) participated in the study. Eleven pharmacists were approached by telephone. One pharmacist was unable to help due to time commitments, the remaining ten (two male and eight female) agreed to participate. Therefore, 16 people were recruited to the panel. All panellists completed both rounds of the questionnaire survey. Brief demographic details of the survey participants are shown in Table 2.
|
The Delphi panel generated 16 additional new indicators in the first round. These indicators, shown in Appendix 3, became indicators 43 to 58 in the second-round Delphi questionnaire.
At the end of the second round, 29 indicators reached the pre-defined level of consensus for being a PDRM (19 of the US generated indicators and ten generated by the panel in the first round). These are shown in Table 3.
|
All four reliability indicators designed not to represent PDRM (Appendix 2, numbers 4, 22, 35, and 37) were rejected by the panel as PDRMs. Apart from these, only one other indicator was rejected as being a PDRM (Appendix 2, number 2). Consensus was not achieved for the remaining indicators (n = 23). The reasons why were not explored in this study.
|
Conclusions |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
The Delphi technique
The Delphi technique has been widely used within health services research including, in recent years, the specific area of prescribing [26–28]. Despite criticisms, this technique is gaining in credibility and, providing the principles of rigorous consensus methods are adhered to, has a useful and important role to play [23].
This postal survey technique prevents bias from dominant individuals, as each panel member is able to express their own opinions away from peer pressure [23]. In this study, the panel ‘experts’ comprised a combination of pharmacists and GPs with specific roles in primary care, chosen because of their relevant knowledge, perspective, and experience. Although the optimal size of a Delphi panel has not been established [23], in this instance (as is normal), it was governed by resource constraints. Attrition rates from random samples are usually higher than from nominated samples [23]. Our study reflected the opinion of purposively selected pharmacists and GPs rather than a random sample of pharmacists and GPs. Indeed, the majority of those who agreed to take part went on to complete the first round of the questionnaire. Furthermore, all those completing the first round completed the second round.
Preventable drug-related morbidity indicators
The starting point for this UK work was PDRM indicators developed in the US [19]. Other UK workers have identified that there may be country-specific factors which preclude the direct transfer of quality indicators from the US to the UK. Factors cited included differences in provider setting, payments and litigation contexts, and clinical practice [29]. We have also identified that direct transcription of North American findings cannot be automatically assumed, due to differences in clinical practice/prescribing patterns between the US and the UK. Despite the fact that the monitoring of drug therapy in some of the US indicators may be considered a luxury by current UK practice, these indicators were retained; for example, monitoring the creatinine levels of a patient prescribed an angiotensin-converting enzyme (ACE) inhibitor every three months. Although the fact that monitoring is necessary goes without question, the issue of how, or indeed if, the frequency of monitoring in UK practice should be altered will be followed up in the next stage of this work.
It must also be noted that the composition of the Delphi panels in the US and the UK were somewhat different (experts in geriatric medicine in the US; GPs and pharmacists in the UK). Although the UK panel were specifically asked to consider both the US-derived indicators and the generation of new indicators in the context of patients aged over 65, it is likely that the UK work will be of relevance to all adults taking prescribed medication.
It is noteworthy that ten indicators were excluded at the preliminary validation stage, on the grounds that their prevalence was likely to be minimal. These were not included in the Delphi questionnaire survey for pragmatic reasons. We were conscious of the workload burden being placed upon the panellists and wanted this to remain within reasonable limits and thereby avoid compromising the Delphi process. However, these indicators are likely to be revisited in future work.
The results also highlight possible differences in philosophical viewpoints between some health professionals practising in the UK and US health care systems. It is notable that a number of definitions which related to ‘failing to prescribe’ a medicine (Appendix 2; indicator numbers 17, 19, and 36) were considered PDRM by the US panel, but no clear consensus was achieved by the UK panel. This is an area that we will also pursue in future work.
The developmental work conducted in the UK to date is only the initial phase of the validation of the indicators for application in the UK. The precise wording on the process components of these indicators are not fixed as they currently stand, for example, the frequency of monitoring required for specific drug/drug groups. Furthermore, in contrast to the US-derived indicators, the indicators generated by the Delphi panel were only included in the final round of the Delphi process. These indicators will, therefore, be subjected to further validation in future work.
As indicators based on medical records represent a way of assessing the quality of patient care, we will focus in the next stage of this work on operationalizing these indicators. Operationalization, both in a general practice database and in the context of drug-related admissions to hospital, will help to identify the most important issues. By embracing both the systems approach and quality improvement/clinical governance simultaneously, individual practice data will be shared with that practice in order to identify why specific events had occurred and what systems should be instituted to prevent them recurring in the future. These data will form a key part of the ongoing refining, modification, and hence further validation, of the indicators. Although computerization is not without problems, the continuing development of information technology related to prescribing in general practice may also ultimately help facilitate the future development of prospective medical-record-based PDRM indicators.
Appendix 1
|
Appendix 2
|
Appendix 3
|
|
Box 2 A sample from the Delphi questionnaire
For the purpose of this questionnaire preventable drug-related morbidity (PDRM) is defined as:
‘A recognizable drug-related problem with a foreseeable adverse outcome and a probable cause related to medicine use which is both identifiable and controllable.’
After considering each scenario in turn, please use the grid below to identify whether you consider each scenario fits the definition of PDRM given above. Place a circle around the number from 1 to 7 that corresponds with your opinion. Additional space is provided should you wish to make any clarifying comments or suggest minor amendments which increase the validity of the scenario e.g. frequency of investigation.
|
Scenario 1
Management: Use of an oral/topical non-steroidal anti-inflammatory drug (NSAID) for more than three months without monitoring serum creatinine at least every three months.
Outcome: Raised serum creatinine.
Scenario 2
Management: Use of a long-acting benzodiazepine in a patient with a past medical history (PMH) or current diagnosis of depression.
Outcome: GP practice or hospital contact due to depression and/or an increase in the dosage of an antidepressant.
Accepted for publication February 13, 2002.
|
References |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
- The President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Quality First: Better Health Care for all Americans. Washington, DC: US Government Printing Office, 1998.
- Department of Health. A First Class Service. London: Department of Health, 1998.
- Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. Washington, DC: Institute of Medicine, 1999.
- Department of Health. An Organisation with a Memory. London: Department of Health, 2000.
- Safety First. Report to the Australian Health Ministers’ Conference 2000. Australian Council for Safety and Quality in Health Care, May 2000.
- Brennan TA, Leape LL, Laird NM et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370– 376.
[Abstract] - Leape LL, Brennan TA, Laird N et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991; 324: 377– 384.
[Abstract] - Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian health care study. Med J Aust 1995; 163: 458– 471.
[Medline] - Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. Br Med J 2001; 322: 517– 519.
[Abstract/Full Text] - Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. J Am Med Assoc 1998; 279: 1200– 1205.
- Bates DW, Cullen DJ, Laird N et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. J Am Med Assoc 1995; 274: 29– 34.
- Nelson KM, Talbert RL. Drug-related hospital admissions. Pharmacotherapy 1996; 16: 701– 707.
[Medline] - Lindley CM, Tully MP, Paramsothy V et al. Inappropriate medication is a major cause of adverse drug reactions in elderly patients. Age and Ageing 1992; 21: 294– 300.
[Abstract] - Cunningham G, Dodd TRP, Grant DJ et al. Drug-related problems in elderly patients admitted to Tayside hospitals, methods for prevention and subsequent reassessment. Age and Ageing 1997; 26: 375– 382.
[Abstract] - Johnson JA, Bootman JL. Drug-related morbidity and mortality: A cost-of-illness model. Arch Intern Med 1995; 155: 1949– 1956.
[Medline] - Ernst FR, Grizzle AJ. Drug-related morbidity and mortality: updating the cost-of-illness model. J Am Pharm Assoc 2001; 41: 192– 199.
- Department of Health. Medicines and Older People. National Service Framework. London: Department of Health, 2001.
- Hepler CD, Strand LM. Opportunities and responsibilities in pharmaceutical care. Am J Hosp Pharm 1990; 47: 533– 543.
[Medline] - MacKinnon NJ. Risk assessment of preventable drug-related morbidity in older persons. Florida: University of Florida, Department of Pharmacy Health Care Administration, 1999 (PhD thesis).
- Beers MH. Explicit criteria for determining potentially inappropriate medication use by the elderly: An update. Arch Intern Med 1997; 157: 1531– 1536.
[Medline] - British National Formulary 38. London: British Medical Association and Royal Pharmaceutical Society of Great Britain, September 1999.
- Jones J, Hunter D, Consensus methods for medical and health services research. Br Med J 1995; 311: 376– 380.
[Full Text] - Campbell SM, Cantrill JA. Consensus methods in prescribing research. J Clin Pharm Therapeutics 2001; 26: 5– 14.
- Medendium volume 11. Location?: Medendium Group Publishing, Autumn 1999.
- Bowling A. Research methods in health. Investigating health and health services. Buckingham, UK: Open University Press, 1997.
- Cantrill JA, Sibbald B, Buetow S. Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility, and reliability. Quality in Health Care 1998; 7: 130– 135.
[Abstract] - Campbell SM, Cantrill JA, Roberts D. Prescribing indicators for UK general practice: Delphi consultation study. Br Med J 2000; 321: 425– 428.
[Abstract/Full Text] - Dean B, Barber N, Schachter M. What is a prescribing error? Quality in Health Care 2000; 9: 232– 237.
[Abstract/Full Text] - Roland M, Marshall M, Campbell S et al. Can US quality indicators for primary care be transferred to the United Kingdom? In: North American Primary Care Research Group 29th Annual Meeting volume of abstracts. Halifax, Nova Scotia, 2001; 59.
- Leape LL, Brennan TA, Laird N et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991; 324: 377– 384.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C J Morris, S Rodgers, V S Hammersley, A J Avery, and J A Cantrill Indicators for preventable drug related morbidity: application in primary care Qual. Saf. Health Care, June 1, 2004; 13(3): 181 - 185. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. S Flanagan, N. J MacKinnon, S. K Bowles, and S. A Kirkland Validation of Four Clinical Indicators of Preventable Drug-Related Morbidity Ann. Pharmacother., January 1, 2004; 38(1): 20 - 24. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. C. Earle, E. R. Park, B. Lai, J. C. Weeks, J. Z. Ayanian, and S. Block Identifying Potential Indicators of the Quality of End-of-Life Cancer Care From Administrative Data J. Clin. Oncol., March 15, 2003; 21(6): 1133 - 1138. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||