OUP user menu

Risk and pharmacoeconomic analyses of the injectable medication process in the paediatric and neonatal intensive care units

Isabella De Giorgi, Caroline Fonzo-Christe, Laurence Cingria, Béatrice Caredda, Valérie Meyer, Riccardo E. Pfister, Pascal Bonnabry
DOI: http://dx.doi.org/10.1093/intqhc/mzq015 170-178 First published online: 9 April 2010

Abstract

Objective To analyse safety risks in injectable medications. To assess the potential impact and pharmacoeconomic aspects of safety tools.

Design The injectable drug process was prospectively assessed using a failure modes, effects and criticality analysis. Criticality indexes were estimated based on their likelihood of occurrence, detection probability and potential severity. The impact of 10 safety tools on the criticality index was calculated and extrapolated to all drugs injected daily. Yearly costs for a reduction in criticality by 1 point (=1 quali) per day were estimated.

Setting Paediatric and neonatal intensive care units in a University Hospital.

Participants Two paediatric nurses, a neonatologist, three hospital pharmacists.

Interventions Qualitative and quantitative risk assessment.

Main Outcome Measures Failure modes, criticality indexes, cost-efficacy ratios.

Results Thirty-one failure modes identified, with the mean of their entire criticality indexes totalling 4540. The most critical failure mode was microbial contamination. The following gains were predicted: 1292 quali (46 500 per day by extrapolation) from ready-to-use syringes, 1201 (72 060) by employing a clinical pharmacist, 996 (59 780) from double check by nurses and 984 (59 040) with computerized physician order entry. The best cost-efficacy ratios were obtained for a clinical pharmacist (1 quali = 0.54 euros), double check (1 quali = 0.71 euros) and ready-to-use syringes (1 quali = 0.72 euros). Computerized physician order entry showed the worst cost-efficacy ratio due to a very high investment costs (1 quali = 22.47 euros).

Conclusion Based on our risk and pharmacoeconomic analyses, clinical pharmacy and ready-to-use syringes appear as the most promising safety tools.

  • paediatrics
  • risk assessment
  • FMECA
  • hospital pharmacy services
  • cost-efficacy analysis
  • intensive care

Introduction

Medication errors are causing significant harm to hospitalized patients with high economic implications; the risk is particularly high in intensive care units [1]. Although medication errors and adverse drug events (ADEs) have received substantial attention in adults, relatively few published reports have addressed this issue in children. Information on paediatric medication use, particularly in neonates, is often lacking [2]. Paediatric patients need weight-based dosing, which necessitates more calculations than for adults [3]. In addition, the range of licensed medications in appropriate dosage forms is limited, thus often requiring complex dose and dilution calculations before administration. Previous studies have identified an error rate of 13–84% in hospitals when preparing and administering intravenous drugs to infants and children [46]. Dose calculations are a common contributor to medication errors, with a factor 10 error being among the most common [7]. In a 6-week prospective study in two university paediatric institutions, Kaushal et al. [8] found that medication errors occurred for 5.7% of prescriptions. Although the preventable ADEs rate was similar to that found in adult inpatients, errors with the potential to cause harm were three times more likely to occur in paediatric inpatients.

Awareness is growing that prospective risk analysis approaches as used in a number of high hazard industries need to be applied to health care [9, 10]. Among other methods, the ‘failure modes, effects and criticality analysis’ (FMECA) is a well-described tool to systematically assess a process. It identifies possible or likely errors (‘failure modes’), and gauges what their effect may be even before they take place [11]. FMECA includes a quantitative evaluation of the criticality of each failure mode, and compares the top critical events in different process organizations, allowing a simple estimation of the impact of new tools on patient safety even before their implementation.

The objectives of the current study were to perform a prospective risk analysis to quantitatively evaluate the safety of the current injectable medication process in the paediatric and neonatal intensive care units (PICU and NICU) of our University Hospital, with a special focus on the preparation and administration steps. We compared the potential impact of 10 safety tools on risk of ADEs and also classified these tools using pharmacoeconomic criteria.

Method

Setting

The study was based on the activity of the PICU (10 beds) and the NICU (15 beds) of the Geneva University Hospitals (2000 beds).

Current injectable medication process in the PICU and NICU

At the time of analysis, prescriptions were handwritten by the medical staff. Drug dispensing was performed by nurses from a ward stock, the stock being refilled by global orders to the central pharmacy. Nursing staff transcribed the orders to the nurse care plan and selected the drug from the ward stock for preparation and administration. No horizontal laminar flow hood (HFLA) was used for the preparation, but parenteral nutrition and some other drugs were compounded at the pharmacy. The name of the drug, the flow rate and the concentration of the solution were written on a sticky label for the prepared injectable. No systematic double check of the preparation (calculation, drugs used, dilution) was performed by a second nurse. Injectables were administered using syringe or volumetric pumps. No in-line filters were used with infusion lines. There was no clinical pharmacist integrated in any units but regular visits were organized by a pharmacist to discuss problems of preparation and administration of drugs with nurses.

In the meantime, a computerized physician order entry (CPOE) named Clinisoft® and including clinical decision-support systems like monitoring of vital functions, information on drugs, patient history, laboratory and radiology results has been implemented.

Design

A FMECA was performed by a multidisciplinary team (two specialized nurses, a neonatologist and three hospital pharmacists) [12]. The analysis focused on the entire medication process of injectables, from prescription to administration, with a special attention to preparation and administration steps.

FMECA risk analysis

A brainstorming strategy was used to determine all possible ways the injectable medication process might fail. Each team member had to write down all risks and possible failures they could envisage. These suggestions were then assembled and organized during a common discussion to become the failure modes. An Ishikawa's diagram was built to organize them step by step (Fig. 1).

Figure 1

The Ishikawa's diagram illustrating the five steps of the medication process with their associated failure modes.

Three frequently used drugs with different characteristics were chosen as models for injectables: gentamicin for antibiotics and other common injectables; morphine for analgesics and narcotics; dopamine for vasoactive and monitored drugs. The likelihood of occurrence of each failure mode for each model drug was classified from 1 to 10, the severity of the potential effect for the patient from 1 to 9 and the probability to detect the failure from 1 to 9. The evaluation was carried out according to standardized tables, taking care to remain coherent in ranking similar events [12]. Scores were obtained by consensual quotation in the team. In particular, occurrence was supported by data from the systematic critical incident reporting of the two ICUs. The criticality index of each failure mode was calculated by multiplying the frequency, severity and detection scores, yielding a minimum of 1 and a maximum of 810. The top 10 critical failure modes were determined by ranking the mean criticality indexes of the three model drugs.

Ten tools to improve safety were chosen empirically (Table 1). Their potential benefit on the criticality indexes of the three model drugs was again assessed by the FMECA method.

View this table:
Table 1

The 10 safety tools, their associated costs, the percentage of injectable drugs injected daily affected by each safety tool, the extrapolated number used to multiply mean criticality index, and the parameters for sensitivity analysis, together with their estimated variation

Safety toolAssociated costsPercentage of affected injectable drugs (normalized per day)Extrapolated number to multiply mean criticality index (max = 60)Sensitivity analysis: parameters with estimated variations
Current situationNo costs100%60Free
Computerized physician order entryInitial investment and maintenance100%60−964 800 euros on mean cost per year = initial investments written off after 5 years
Double check by nursesHuman resources100%60±5 s spent for a check
Clinical pharmacistHuman resources100%60±10 years of experience
Ready-to-use syringesDevelopment & production costs60% could be prepared as ready-to-use syringes36±0.31 euros on price of a syringe (R & D)
Vial of dilutionMedical supplies45% concerned by a withdrawal of a small volume (<0.5 ml)27±0.42 euros on price of a vial
Intermediate dilutionsMedical supplies45% concerned by a withdrawal of a small volume (<0.5 ml)27No variation
In-line filtersMedical supplies70% administered via a central line42±2 in-line filters used by day (filters blocked; number of patients or central line)
Simple additional measures of asepsisMedical supplies72% not ready-to-use42No variation
Horizontal laminar airflow hood in the unitInitial investment and medical supplies72% not ready-to-use42−7236 euros on mean cost per year = initial investments written off after 5 years
Drug plannerNo costs16% administered at unusual dosing interval (e.g. every 18 h)10free

The term ‘quali’ (plural: quali) was created to allow a convenient transposition from the notion of criticality to the quality gain in the medication process. One quali was defined as a reduction of the criticality index by one point. Quali for the top 10 critical failure modes were compared between the different safety tools analysed.

Generalization for economic estimate

As required in economic analyses, an extrapolation to the use of all daily injectable drugs was performed for each safety improvement tool (see Table 1), using data compiled during a large survey performed during the year 2003 in the same units [13]. On average, 7 patients with 8 drugs per day were hospitalized in the PICU and 14 patients with 3 drugs in the NICU (overall a total of 98 drugs per day). About 60 drugs were used as injectables.

Cost analysis

Cost analysis was performed from a hospital perspective. The required investment in euros per year was calculated for each tool (at the time of publication: 1 euro = 1.50 CHF = 1.43 USD = 130.90 JPY). Only direct costs were considered. Only medical supplies such as syringes, needles, in-line filters, face masks, etc. were taken into account for simple additional measures of asepsis, intermediate dilution, in-line filters and vials of dilution. Additional human resources were necessary for double check by nurses and clinical pharmacist. The nurse's double-checking time at each potential failure point was estimated at 20 s [14]. The total time for all injectable drug checks was converted into euros using the local wages scale (100% salary for nurse with intermediate level of experience = 45 000 euros per year). The daily time requirements for a clinical pharmacist for both units were estimated to 50% (half-salary for pharmacist with intermediate level of experience = 40 000 euros per year). The tasks considered for the clinical pharmacist were (i) to check the correct use of drugs by nurses through specific though not systematic interventions concerning preparation (respect of concentration; of solvent; etc.) and administration (detection of physicochemical incompatibilities; check the flow rate according to concentration of the solution, etc.) and (ii) audits of the drugs stock (organization; fridge, etc.). For the CPOE, initial investment costs and maintenance costs during 1 year were estimated at 1 million euro and 300 000 euros, respectively. The estimations of initial investment and maintenance costs have been provided by the head of medical informatics' team of the hospital. These costs are specific for a system to be used in the intensive care environment (adults and children), but cover the whole cost of the electronic patient record (not only the CPOE). For HFLA, the estimated costs included an initial investment plus the cost of aseptic preparations in the units. We estimated that 15 drugs could be provided in ready-to-use syringes (CIVAS), and we calculated the costs of development (3620 euros by drug), as well as their production cost (1.21 euros per syringe), based on our previous experience. The drug planner induced no costs [15].

Cost-efficacy analysis

The cost in euros required for an improvement of one quali per day was calculated for each safety improvement tool by dividing the investment per year by all yearly quali (extrapolation from all daily injected drugs).

Sensitivity analysis

A sensitivity analysis was performed to evaluate the influence of variations in estimated parameters on total costs (Table 1).

Results

The medication process of injectables was split up into five major steps, prescription, transcription, preparation, administration and storage. A total of 31 failure modes were determined during the brainstorming sessions (Fig. 1).

FMECA risk analysis

The criticality indexes calculated from the frequency, severity and detection scores estimated by the team for each failure mode are summarized in Table 2. The average sum of all criticality indexes was 4540. Among the model drugs, gentamicin totalized the greatest sum of criticality index, followed by morphine and dopamine. The most critical failure mode (mean criticality index = 432) was microbial contamination during the preparation of medicines. The top 10 critical failure modes concerned mainly preparation (5 failure modes), followed by administration (4) and transcription (1). Top failure modes for preparation were microbial contamination (432), dosage errors (343), dilution errors (312), labelling errors (224) and selection errors (171). Top failure modes for administration were physicochemical incompatibilities (330), wrong flow rate (317), selection error (208) and drug given twice (194). The failure mode during transcription was poor writing and reading (235).

View this table:
Table 2

Failure modes and criticality indexes for the model drugs (top 10 of failure modes in grey)

Failure modesDuring the study
GentamicinMorphineDopamineMean criticality index
PrescriptionDosage error24510548133
Prescription omitted126541866
TranscriptionPrescription omitted to be transcribed10512942
Poor writing and reading224224256235
Stop order omitted60401639
PreparationError of writing on the preparation card stuck on the pumps126126144132
Wrong selection between two different drugs (sound-alike, look-alike, etc.)16074872
Wrong selection between several dosages or salts of a drug2522528171
Wrong selection of the solvent of reconstitution or dilution1628196113
Microbial contamination432432432432
Preparation of a drug forgotten245241896
Dosage error343294392343
Dilution error336216384312
Error of calculation of the patient's parameters168120120136
Error in labelling of a prepared drug192224256224
Precipitation (high concentration)72353547
Chemical degradation of drugs in a mixed preparation1081055690
Inaccuracy of small volumes withdrawal456010871
AdministrationPump doesn't work56487259
Wrong flow rate343294315317
Physicochemical incompatibilities336360294330
Drug given twice39212664194
Wrong administration time3787256169
Wrong selection between two different drugs (sound-alike, look-alike, etc.)96563261
Wrong selection between several dosages or salts of a drug3362808208
Wrong route of administration28287042
Wrong injection site56289861
Wrong patient19619680157
Air introduced in central intravenous tubing4864192101
Air introduced in peripheral intravenous tubing24329651
StorageStorage (protection for light, temperature control of drugs, expiry date)48303638
Total5738402538574540

Generalization for economic estimate

In a total of 4540 criticality points averaged between the model drugs (272 400 criticality indexes per day by extrapolation), we expected to gain 1292 quali (46 500) with CIVAS, 1201 (72 060) with a clinical pharmacist, 996 (59 780) with double check by nurses, 984 (59 040) with CPOE, 555 (23 296) with in-line filters, 457 (12 348) with vial of dilution, 408 (17 122) with HFLA, 170 (4590) with intermediate dilution, 144 (6192) with simple additional measures of asepsis and 98 (951) with the drug planner.

The differences in quali of each safety tool for the top 10 critical failure modes are shown in Fig. 2. For the most critical failure mode, i.e. microbial contamination, five safety tools allowed a gain in quali. The greatest improvement of 384 quali was obtained with CIVAS whereas intermediate dilution was associated with a loss of quali (−72). Six safety tools gained quali for the second most critical failure mode, dosage errors during preparation. For the third most critical failure mode, the wrong flow rate, almost none of the proposed tools allowed a significant safety improvement.

Figure 2

Top 10 of failures modes and comparative gain or loss of quali for each safety tool (mean CI = mean criticality index).

Cost-efficacy ratio

Cost in euros per year and associated gain in quali for each tool are represented in Fig. 3. The 10 tools are laid out in four quadrants, according to yearly costs gained per quali and total quali gained. Intervals were calculated with the sensitivity analysis. The best cost-efficacy ratio were obtained for a clinical pharmacist (1 quali = 0.54 euros), for double check by nurses (1 quali = 0.71 euros) and for CIVAS (1 quali = 0.72 euros). The CPOE showed the worst cost-efficacy ratio due to the very high investment costs (1 quali = 22.47 euros).

Figure 3

The 10 safety tools are laid out in four quadrants, according to yearly costs to gain 1 quali per day and total quali gained per day. Intervals were calculated with a sensitivity analysis.

Discussion

Our work confirms that FMECA is a feasible tool for a proactive assessment of the injectable medication process in the PICU and NICU, even before the occurrence of adverse events. As most medication errors are multi-factorial, an FMECA was particularly useful in that it allowed consideration of the global medication process. Such errors are difficult to study and FMECA is a novel approach to predict the most cost-effective interventions. These would then need to be tested and measured in PICUs and NICUs.

Unlike other reports, we voluntarily did not focus exclusively on prescriptions by examining also the preparation and administration steps, which are seldom investigated in the literature [16]. Prot et al. [17] for instance analysed and classified drug administration errors but did not include errors occurring during the preparation step. However, such specification appears to make sense, since errors at the administration stage have been found to be the most common ones in paediatric patients [18].

In our current work, microbiological contamination during preparation was the most critical failure mode. A similar result was found in a recently published large multi-centre audit of six European hospital departments [19]. Aseptic procedures required for the safe preparation of intravenous drugs were frequently violated by staff, often unaware of the potential harm. CIVAS offer a safe alternative to reduce microbiological contamination and dilution errors and avoid drug wastage. An earlier development of ready-to-use vancomycin syringes for our NICU was very satisfying [20]. HFLA alone was unable to significantly reduce risks of microbiological contamination as it requires qualified operators, standardized protocols and a quality control procedure to be efficient.

The difficulties encountered when preparing infusions from concentrated stock solutions in intensive care units are highlighted in the literature [21, 22]. To further improve preparation in paediatrics, the most critical step in this FMECA, we have begun studying the safest procedure with the best accuracy for solutions prepared by three different techniques: (i) small volumes withdrawn as currently, (ii) intermediate dilutions and (iii) standardized vials of dilution.

The most cost-effective way of reducing the criticality of injectable drugs was expected for a clinical pharmacist in the NICU and PICU. Physicochemical incompatibilities during administration were one of the most critical failure modes, and its improvement was worthy of note. Our results agree with numerous publications demonstrating the benefit of a paediatric clinical pharmacist, particularly in the prescription step [23, 24]. One study reported savings of $9135 per year in a 10-bed PICU of a university-affiliated children's hospital [25].

The use of in-line filters was proposed in order to reduce microbiological contamination of injectable medications, but there is no convincing evidence so far of their efficiency in preventing catheter-related infections [26]. The advantages of such devices are a reported reduction of phlebitis, and the elimination of air, filtration of particulate contaminants and drugs precipitates. The fact that in-line filters cannot be used with colloids and blood and may complicate drug administration by adsorption and are expensive has to be considered in the cost-efficacy analysis [27].

Some drugs with uneven administration times, such as gentamicin, are particularly prone to wrong administration times. To avoid errors in the calculation of the correct administration time, a drug planner giving the time of next injection based on the last injection may be used [15]. This simple tool can be used for all drugs with uneven injected times. Although the impact of this tool is limited, its cost is virtually zero.

CPOE improved a considerable number of failure modes, but at high relative costs. A number of publications advocate the positive impact of a CPOE on drug prescription and administration safety in paediatric inpatients [28, 29]. However, CPOE for paediatric drug management may also introduce new errors unseen with the previous paper prescription [30]. Han et al. [31] even reported an increase in mortality from 2.8 to 6.6% in a paediatric referral centre after implementation of a commercially available CPOE system. This unexpected result was only partially attributable to the new prescription tool itself, and was associated with major changes in the organization processes. Such contradictory results highlight the difficulty of a successful implementation of a CPOE, no doubt influenced by the quality and the exhaustiveness of the program itself, but also by numerous technical, organizational, cultural and human factors. CPOEs are often included in more complete electronic patient record systems, and their high costs may be justifiable as their advantages extend well beyond the specific safety issues of injectable drugs.

Overall, the two major interests of an FMECA are its simple application and the quantitative character of the evaluation made possible by combining three complementary factors. The evaluation is easy to perform and not too time-consuming. A FMECA is therefore very helpful to decide and prioritize actions to be taken. Moreover the active brainstorming and discussion necessary to find consensual benchmarks contribute to the development of a very clear and shared vision of the process organization, accounting for all the different perspectives. The structured analysis allows constructive, objective and respectful discussions. Team members noticed that they gained extensive comprehension on each other's constraints.

The major limitation of FMECA is an unavoidable subjectivity component. As the team becomes larger and multidisciplinary this bias is minimized. In the current study, we obtained consensual benchmarks between all members of the team, thus guaranteeing the highest possible objectivity. To further limit variability, the scores were based on explicit criteria published earlier [12]. It should be noted that the specific mark found for a failure mode is not an essential result, as the main goal is to rank risks and compare orders of magnitude. A further limitation is that it is impossible to assess the impact of a combination of multiple failures on a specific outcome. However, the separate analysis of each failure mode is also an advantage as it leads to a deeper understanding of the risk associated to each step, thus allowing targeted improvements. The FMECA method has not been fully validated but is recommended by the Institute for Healthcare Improvement [32].

Our results on criticality of injectable medication are specific to our local situation. Conclusions may not be directly applicable to the injectable medication process in every PICU. However, the impact of some safety tools on the criticality can easily be generalized. Most of the reported failure modes also concern other institutions and our data appear as a good starting point to locally repeat a similar evaluation of the medication process.

Conclusion

The use of a prospective risk analysis such as FMECA has allowed a quantitative evaluation of the safety of paediatric patients in connection with the injectable medication process. The FMECA allowed generating tools for continuous safety improvement by modelling the relative safety gain for specific tools or new developments. Based on a pharmacoeconomic analysis in our local setting, the involvement of a clinical pharmacist and the introduction of ready-to-use syringes for selected drugs have been shown to be the most cost-effective tool.

Acknowledgements

The authors wish to acknowledge Prof. Bernard Testa for the careful reading of the article.

References

View Abstract