International Journal for Quality in Health Care Advance Access originally published online on October 18, 2005
International Journal for Quality in Health Care 2006 18(2):87-94; doi:10.1093/intqhc/mzi084
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Review Article
The validity of explicit indicators of prescribing appropriateness
School of Pharmacy & Pharmaceutical Sciences, University of Manchester, Manchester, UK
Objective. To assess, from the perspective of UK hospital doctors, the content validity and operational validity of a set of 14 previously developed explicit indicators of the appropriateness of long-term prescribing started during a hospital admission.
Method. A combination of data extraction from medical records and qualitative interviews with a maximum variability sample of hospital doctors.
Participants. The indicators were applied to 132 new prescriptions, intended for long-term use, prescribed for 61 patients; 36 doctors, of various grades, were purposively selected for interview.
Results. Appropriate prescribing was viewed as prescribing that was indicated, necessary, evidence based (using a broad meaning of ‘evidence’) and of acceptable cost and risk-benefit ratio. These concepts applied to individual drugs for individual patients, rather than at a more general, public health level. Where drugs had failed an indicator, rationales were explored. Often, it was missing data in the medical notes that had resulted in the drug failing the indicator.
Conclusions. The 14 indicators were considered to have content validity, reflecting all aspects of appropriate prescribing discussed by the doctors. Their operational validity was less clear-cut, due to the lack of necessary data in the medical notes. This has implications for the use of explicit indicators for assessing prescribing appropriateness, as these hospital doctors did not consider that the data required for objective, systematic assessment of prescribing would ever be recorded in hospital medical notes.
Keywords: appropriateness, indicators, prescribing, validity assessment
Address reprint requests to Mary Patricia Tully, School of Pharmacy and Pharmaceutical Sciences, University of Manchester, Oxford Road, Manchester M13 9PL, UK. E-mail: mary.tully{at}manchester.ac.uk
Accepted for publication September 18, 2005.
The evaluation and assessment of health care quality are receiving worldwide attention [1–3]. The spiralling cost of drugs, with finite resources and frequent medication errors [4], has drawn attention to prescribing quality and appropriateness. Appropriate prescribing has been defined as ‘the outcome of the process of decision-making that maximises net individual health gains within the society’s available resources’ [5].
The commonest methods for assessing appropriate or inappropriate prescribing are explicit criteria and implicit professional judgement. The former uses a consensus panel to prepare a set of standards from the literature [6,7]. They are dependent not only upon available evidence but also upon the authors’ ability to routinely include new evidence. Implicit judgement uses health care professionals’ clinical acumen to assess the appropriate prescribing for individual patients [8,9]. Consequently, implicit judgements are subject to reviewer variations. However, implicit judgements take account of the context of individual patients and easily incorporate new evidence into the assessment. They may have greater validity but poorer reliability than explicit criteria [5].
Combining these assessment types has the potential to utilize the advantages of both. Explicit indicators require each prescription to be compared with a set of published standards (e.g. the Physicians’ Desk Reference [10] or the British National Formulary [11]), but within the context of the individual patient. Such standards are used for the Medication Appropriateness Index [12] and the prescribing appropriateness indicators [13], which have been validated for ambulatory care in the US and primary care in the UK. However, there has been no similar work using explicit indicators to evaluate prescribing appropriateness in secondary care in the UK. We have previously described the development and preliminary assessment of a set of explicit indicators for this purpose [14,15]. This study aimed to assess the content and operational validity of these indicators from the perspective of UK hospital doctors.
 |
Method |
|---|
 Top Method ResultsDiscussionAppendixAcknowledgementsReferences |
|---|
A set of 14 explicit indicators (Table 1) were derived from the literature [12,13,16], applied to data in the hospital medical records about all newly prescribed long-term drugs for randomly selected patients, and suitably refined [14]. They were designed to assess prescribing started during a hospital admission, which was expected to be continued long-term in primary care. The explicit indicators require that drugs be prescribed within the standards set out in the British National Formulary [11] or, if the prescriber wished to deviate from those, that the rationale be documented in the patient’s medical record.
|
Further content and operational validity assessments were conducted in this study using two methods, data extraction from medical records and qualitative interviews. Approval for the programme of work was obtained from the relevant National Health Service Local Research Ethics Committee.
Data extraction from medical records
Patients discharged from medical or care of the elderly wards, at one large teaching hospital in England, were screened for eligibility. The inclusion criteria were that the patient was discharged on a drug that was commenced during this admission, the discharge summary did not recommend that the drug be discontinued within 3 months and the patients were suitable for interview (data not presented in this article). Post-discharge, the indicators were applied to these long-term prescriptions, using data from the patient records.
Interviews
Doctors working in general medical specialities (e.g. cardiology, endocrinology, or geriatrics) were invited to take part in one of two types of in-depth interviews. They were eligible for the first if they had either made or documented in the notes the decision to commence a drug for one of the eligible patients, where their prescription had failed one or more indicators. Additional doctors from the same departments were invited to participate in the second type of interviews, which discussed appropriate prescribing more generally, to widen the scope of the data. A maximum variability sample was sought in both cases.
The interviews were qualitative, lasting 30–60 minutes, audiotaped and transcribed in full. Doctors were not given a definition of appropriate prescribing but were asked how they would judge the appropriateness of prescribing decisions that they or their colleagues might make. Where applicable, they also discussed how their prescribing had been assessed by the indicators (a copy was provided), both specifically and as examples of what they considered ‘normal practice’.
Data Analysis
Data were recorded as to how each prescription had been judged by each of the explicit indicators. Each indicator used an ordinal scale, which was further subclassified as to whether each prescription was appropriate or inappropriate.
The interview data were analysed from an interpretivist perspective, assisted by NVivo software (QSR, La Trobe University, Melbourne, Australia). Themes were identified during a process of repeated reviewing of the transcripts and data tabulated, using the Framework method [17].
Content validity meant that the explicit indicators included all the relevant domains of appropriate prescribing. The indicators (Table 1) reflected the domains of indication (Indicators A–C), evidence for effectiveness (F, G), cost effectiveness (inclusion in the hospital formulary as a proxy, H), adverse reactions (L, M) to the drug, high risk or hazardous drug–drug or drug–disease combinations (D, N, O) and duration, dose and frequency of the prescription (E, J, K). Each indicator was judged to have content validity if the domain was discussed by the doctors as being part of how they would judge appropriate prescribing.
Operational validity meant that the indicators were capable of being implemented in practice. The operational definitions used data from the patients’ medical records (e.g. Appendix). The British National Formulary was used as the ‘gold standard’. This validity assessment used a combination of data extraction from medical records and qualitative interviews with the prescribing doctors. The indicators were judged to have operational validity if, when prescriptions were judged inappropriate by them, it was because they were actually inappropriate, rather than due to a problem with the data used to assess the prescriptions.
|
Results |
|---|
|
TopMethodResultsDiscussionAppendixAcknowledgementsReferences |
|---|
Sixty-one patients met the inclusion criteria; the indicators were applied to 132 new long-term prescriptions (Table 2). Thirteen doctors (three consultants, two specialist registrars, six senior and two pre-registration house officers) were interviewed about 38 prescriptions for 13 patients. A further 23 doctors (eight pre-registration house officers and five each of senior house officers, specialist registrars, and consultants) were purposively selected for the more general interviews. Illustrative quotations from the interviewed doctors are provided in Table 3, which were selected for their relevance to the themes.
|
|
Content validity
The term ‘appropriate prescribing’ was used by the doctors in general conversation, with an implicit, rather than explicit, understanding of its meaning. A ‘recognized’ or ‘acceptable’ indication for the drug was the paramount issue for most doctors (Indicators A–C). Evidence of, or a theoretical possibility of, benefit was seen by many as a further prerequisite for appropriateness (Indicators F, G). An appropriate drug, therefore, had to be ‘needed’, and an inappropriate drug the opposite.
Clinical trials played an important role but were not the sole source of acceptable evidence (Indicators F, G). The blind application of such evidence, ‘taking generic advice and applying it irrationally to people throughout the population’ (Consultant 1), was also considered inappropriate. Phrases such as ‘pharmacological evidence’ were used to describe the doctor’s extrapolation of a drug’s action to a putative benefit where clinical trials were lacking.
Doctors of all grades raised the ‘cost’ of prescribing drugs (Indicator H). The junior doctors considered adherence to policies, such as the hospital formulary, which incorporated this ‘cost’ into drug selection, to be important (Table 3, quote 1). Acquisition cost was mentioned by two doctors but only as a final deciding factor in decision-making. The potential ‘cost’ was most often described in terms of clinical problems, such as adverse effects (indicators L, M), onerous monitoring or the impact on patients (Table 3, quote 2). Doctors working in care of the elderly and nephrology, where polypharmacy was common, mentioned drug–drug interactions in most detail (Indicators D, N) (Table 3, quote 3). Another questioned whether only interactions that had actually caused morbidity should be classified as inappropriate prescribing. He gave as an example the co-prescription of angiotensin converting enzyme inhibitors and non-steroidal anti-inflammatory drugs (Table 3, quote 4).
There was recognition of the risk–benefit relationship in drug–disease interactions (Indicator O) that had to be weighed up, consciously or unconsciously, for each prescribing decision (Table 3, quote 5). The doctors knew that patients may need drugs that were cautioned with concomitant diseases (such as amiodarone with thyroid disease), but these were chosen based ‘on the old adage of doing least harm or doing no harm’ (specialist registrar 2). ‘Continuing medication indefinitely’ and ‘lack of review of continued need’ (Indicator E) were further examples of inappropriate prescribing. The dose and frequency of the drugs were raised (Indicators J, K), usually for drugs with narrow therapeutic indices, such as phenytoin.
There were two main areas raised in the discussions of appropriate prescribing that were not assessed by the indicators. The senior doctors raised the issues of patient acceptability and ability to take the medication as being part of the overall concept of appropriateness. They considered these important aspects that addressed the needs and circumstances of the individual patient.
Operational validity
Thirty-eight prescriptions for 13 patients, which had failed at least one indicator (Table 2), were discussed in the interviews. Additional information was provided by the doctors during the interviews, which allowed judgements of some of the indicators to be revised. Although the interviewed doctors had been involved in the prescribing process, not all knew additional information that had not been documented in the medical record. Two main themes underpinned the prescriptions failing the indicators: documentation of information relating to prescribing decisions in the medical record and the suitability of the ‘gold standard’.
The indication per se was linked closely to the doctor’s views of appropriateness. However, the existence of an indication did not necessarily mean that documentation would exist to allow independent assessment (Table 3, quote 6). Many of the doctors ‘would like to think’ that the indication for a drug would always be in the medical record, but recognized that this was unlikely to be the case (Indicator A). Reasons included the lack of available time and the questionable necessity to record the information. In particular, drugs of low perceived risk (such as laxatives) or that replaced others in the same therapeutic class (such as omeprazole replacing rabeprazole) were unlikely to be documented.
Similarly, the rationale for prescribing new drugs would often be omitted from the discharge summary (Indicator B) if this was perceived as unnecessary, not considered a priority or not known to the doctor writing the prescription. The doctors often expected that the general practitioner could deduce prescribing rationales from other available information. Less important information would be excluded from the discharge summary, to avoid compromising the important content. For example, when a patient had been admitted with a myocardial infarction and subsequently diagnosed with diabetes, problems with additional treatment for iron deficiency anemia were considered less important (Table 3, quote 7).
The impact of shift working or transferring patients between clinical teams meant that the doctor who wrote the final discharge summary may well not have been present when initial prescribing decisions were made. Therefore, if the information was not documented, it could not be transferred into the discharge summary (Table 3, quote 8).
There was recognition of the usefulness but also the limitations of the British National Formulary in some specialist areas, where the drugs would be used in unusual ways for complex patients. Five prescriptions failed Indicator C, because the drugs were used for unlicensed indications, with no rationale in the discharge summary. The interviews highlighted how common this was in certain specialities, which reduced the perceived importance of information transfer. In addition, including such information in the discharge summary requires knowledge of the drug’s license. Junior doctors, who normally write the notes and discharge summaries, described a very limited knowledge of this (Table 3, quote 9).
|
Discussion |
|---|
|
TopMethodResultsDiscussionAppendixAcknowledgementsReferences |
|---|
This study assessed the content and operational validity of a set of 14 explicit indicators of appropriate long-term prescribing in secondary care in the UK. Content validity was achieved, as all indicators assessed domains that were considered relevant to appropriate prescribing. Operational validity was not consistently achieved, as many prescriptions were incorrectly judged as inappropriate because of the lack of data in the medical record.
The indicators were used with 132 prescriptions for 61 patients; the interviews were with doctors in a single teaching hospital, working in medical or care of the elderly specialities. All of these may limit the generalizability of the findings. Doctors from other hospitals or specialities may have views on the meaning and assessment of appropriate prescribing that are not reflected here. However, data saturation was achieved for this group of doctors, with no additional themes appearing during later interviews.
Appropriate prescribing was viewed by the doctors as being indicated, necessary, evidence-based (using a broad meaning for ‘evidence’) and of acceptable cost and risk–benefit ratio. In addition, they saw appropriate and inappropriate prescribing as concepts that applied to individual prescriptions for individual patients, as proposed by Buetow and colleagues [5], rather than at a public health level. They did not class drugs or types of prescribing as inappropriate per se, as viewed by Beers and colleagues [7].
These explicit indicators can be criticized as having a purely clinical perspective. Appropriate prescribing is not merely about technical aspects but should also reflect the ways in which medicines are used by patients [18]. Sharpe and Faden [19] have contended that appropriateness is ‘fundamentally evaluative’ and that its determination is not just based on evidence but on ‘values assessment’ by all the major players. In the case of prescribing, that also includes the patient. Elderly patients consider practicality of use as an important aspect of appropriate prescribing [20]. The doctors in our study also discussed patient acceptability, but the lack of relevant data in the medical notes prevented the development of suitable explicit indicators [14].
Using the medical record was intended to facilitate assessing appropriate prescribing as explicitly as possible, minimizing the need for implicit judgements. However, medical records are known to be imperfect tools even for supporting patient care [21]. The doctors’ descriptions of appropriate prescribing did not suggest that these domains would, or even should, be explicitly documented. However, inappropriate prescribing can result from rational decision-making based on inadequate information about the patient [22]. Hence, there is an argument for a greater emphasis on training doctors in good prescribing documentation.
The doctors interviewed in this study described how they would extrapolate information from data in the medical records or not to include points they expected colleagues would already know. Detailed documentation in the medical record is contrary to normal clinical practice, where information must be quick to both write and read [23,24]. However, this ‘reading between the lines’ can often only be done by health-care professionals. For example, when non-clinically trained data analysts screened medical records for adverse drug events or medical errors, they had to pass 93% of cases to a study nurse for further review [25].
The use of these indicators is currently hampered by their operational validity, due to the lack of required data. However, the electronic patient record may provide a solution. For example, electronic prescribing software currently being implemented in UK hospitals can require the prescriber to select one of many predefined reasons for having modified or discontinued prescriptions (personal observation). Similar mandatory documentation of drug indication could be technically possible and would facilitate the use of these indicators for both clinical audit and research. It will be imperative to reassess the operational validity, however, as the way in which data are recorded electronically is likely to be different to how they are recorded on article [26].
The British National Formulary [11] was successfully used as the gold standard in this study. There were very few occasions when drugs were prescribed for indications or regimens that contravened that standard. Nonetheless, these hospital doctors commented on its limitations in a specialist health care environment. The flexibility of these indicators makes such deviations from the standard possible, but the burden of proof for the effectiveness lies with the prescriber. For such prescriptions to pass the indicators, justification needs to be documented in the patient’s medical record.
These indicators of appropriate prescribing move the research agenda from a more general consideration of appropriate drugs to a consideration of appropriate prescribing for the individual patient. Other work [6,27] in prescribing use explicit criteria founded on evidence-based literature review. Our work accepts the reality that the evidence base is lacking for much of prescribing and takes account of this in permitting the use of drugs both within and outside their licensed indication, provided the doctor is willing to provide justification. Thus, it may take greater account of the ‘complex and fluid reality’ in the implementation of evidence-based medicine for individual patients [28].
|
Appendix |
|---|
|
TopMethodResultsDiscussionAppendixAcknowledgementsReferences |
|---|
|
|
Acknowledgements |
|---|
|
TopMethodResultsDiscussionAppendixAcknowledgementsReferences |
|---|
This work was funded by a North West Region National Health Service Postdoctoral Fellowship.
|
References |
|---|
|
TopMethodResultsDiscussionAppendixAcknowledgementsReferences |
|---|
- Australian Council for Safety and Quality in Health Care. Patient Safety: Towards Sustainable Improvement. Fourth Report to the Australian Health Ministers’ Conference. Canberra: Australian Council for Safety and Quality in Health Care, 2003.
- The President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Quality First: Better Health Care for All Americans. Washington, DC: US Government Printing Office, 1998.
- Department of Health. A First Class Service. Quality in the New NHS. London: Department of Health, 1998.
- Department of Health. Building a Safer NHS for Patients – Improving Medication Safety. London: The Stationery Office, 2004.
- Buetow SA, Sibbald B, Cantrill JA, Halliwell S. Appropriateness in health care: application to prescribing. Soc Sci Med 1997; 45: 261–271.[CrossRef][Web of Science][Medline]
- Oborne CA, Hooper R, Li KC, Swift CG, Jackson SHD. An indicator of appropriate neuroleptic prescribing in nursing homes. Age Ageing 2002; 31 (6): 435–439.
[Abstract/Free Full Text] - Beers MH, Ouslander JG, Rollingher I, Reuben DB, Brooks J, Beck JC. Explicit criteria for determining inappropriate medication use in nursing home residents. Arch Intern Med 1991; 151: 1825–1832.
[Abstract/Free Full Text] - Nirodi P, Mitchell A. The quality of psychotropic drug prescribing in patients in psychiatric units for the elderly. Aging Ment Health 2002; 6 (2): 191–196.[CrossRef][Medline]
- Lindley CM, Tully MP, Parmasothy V, Tallis RC. Inappropriate medication is a major cause of adverse drug reactions in elderly patients. Age Ageing 1992; 21: 294–300.
[Abstract/Free Full Text] - Medical Economics. Physician’s Desk Reference, fifty-eighth edition. Montvale, NJ: Medical Economics Co. Inc., 2004.
- Joint Formulary Committee. British National Formulary. London: British Medical Association and Royal Pharmaceutical Society of Great Britain, 2005.
- Hanlon JT, Schmader KE, Samsa GP et al. A method for assessing drug therapy appropriateness. J Clin Epidemiol 1992; 45: 1045–1051.[CrossRef][Web of Science][Medline]
- Cantrill JA, Sibbald B, Buetow S. Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility and reliability. Qual Health Care 1998; 7: 130–135.[Abstract]
- Tully MP, Javed N, Cantrill JA. Development and face validity of explicit indicators of the appropriateness of long-term prescribing. Pharm World Sci in press.
- Tully MP, Cantrill JA. The reliability of explicit indicators of appropriateness of long term prescribing. Pharm World Sci in press.
- Tully MP, Cantrill JA. Exploring the domains of appropriateness of drug therapy, using the Nominal Group Technique. Pharm World Sci 2002; 24: 128–131.[Medline]
- Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In Bryman A, ed. Analysing Qualitative Data. London: Routledge, 1994: 173–194.
- Britten N, Jenkins L, Barber N, Bradley C, Stevenson F. Developing a measure for the appropriateness of prescribing in general practice. Qual Saf Health Care 2003; 12: 246–250.
[Abstract/Free Full Text] - Sharpe VA, Faden AI. Appropriateness in patient care: a new conceptual framework. Milbank Q 1996; 74: 115–138.[CrossRef][Web of Science][Medline]
- Volume CI, Burback LM, Farris KB. Re-assessing the MAI: elderly people’s opinions about medication appropriateness. Int J Pharm Pract 1999; 7: 129–137.
- Audit Commission. Setting the Record Straight. A Review of Progress in Health Records Services. London: Audit Commission Publications, 1999.
- Aronson JK. Rational prescribing, appropriate prescribing. Br J Clin Pharmacol 2004; 57 (3): 229–230.[CrossRef][Web of Science][Medline]
- Berg M. Medical work and the computer-based patient record: a sociological perspective. Methods Inf Med 1998; 37: 294–301.[Web of Science][Medline]
- Tully MP, Cantrill JA. Insights into creation and use of prescribing documentation in the hospital medical record. J Eval Clin Pract 2005; 11: 430–435.[Medline]
- Hope C, Overhage JM, Seger A et al. A tiered approach is more cost effective than traditional pharmacist-based review for classifying computer-detected signals as adverse drug events. J Biomed Inform 2003; 36: 92–98.[CrossRef][Web of Science][Medline]
- Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inf 1999; 55: 87–101.[CrossRef][Web of Science][Medline]
- Park RE, Fink A, Brook RH et al. Physician ratings of appropriate indications for six medical and surgical procedures. Am J Public Health 1986; 76: 766–772.
[Abstract/Free Full Text] - Freeman AC, Sweeney K. Why general practitioners do not implement evidence: qualitative study. Br Med J 2001; 323 (7321): 1100–1102.
[Abstract/Free Full Text] - British Medical Association. Doctors’ Training and Qualifications: http://www.bma.org.uk/ap.nsf/Content/doctorsqualAccessed21May2004.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C. Pulcini, S. Defres, I. Aggarwal, D. Nathwani, and P. Davey Design of a 'day 3 bundle' to improve the reassessment of inpatient empirical antibiotic prescriptions J. Antimicrob. Chemother., June 1, 2008; 61(6): 1384 - 1388. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||