OUP user menu

Development of medical checklists for improved quality of patient care

Brigette Hales, Marius Terblanche, Robert Fowler, William Sibbald
DOI: http://dx.doi.org/10.1093/intqhc/mzm062 22-30 First published online: 11 December 2007

Abstract

Background Checklists are used in both medical and non-medical industries as cognitive aids to guide users through accurate task completion. Their development requires a systematic and comprehensive approach, particularly when implemented in high intensity fields such as medicine.

Objective A narrative review of the literature was conducted to outline the methodology to designing and implementing clear and effective medical checklists.

Methods We systematically searched for relevant English-language medical and non-medical literature both to describe where checklists have been demonstrated to improve delivery of care and also, how to develop valid checklists.

Results The MEDLINE search yielded 8303 citations of which 1042 abstracts were reviewed. On the basis of criteria for inclusion and subsequent full-manuscript review, 178 sources, including 17 non-medical publications, were included in the narrative review. This information was further supplemented by expert opinion in the area of checklist development and implementation. A small number of strategies for designing effective checklists were referenced in the literature, including utilization of pre-published guidelines, formation of expert panels and repeat pilot-testing of preliminary checklists.

Conclusion Despite currently available evidence, a highly effective, standardized methodology for the development and design of medical-specific checklists has not previously been developed and validated, which has likely contributed to their inconsistent use in several key fields of medicine, despite evidence of their fundamental role in error management.

Keywords
  • checklist
  • design methodology
  • mnemonic device
  • patient safety
  • quality improvement

Checklists are used in both medical and non-medical industries as cognitive aids to guide users through accurate task completion. A checklist is an organized tool that outlines criteria of consideration for a particular process. It functions as a support resource by delineating and categorizing items as a list—a format that simplifies conceptualization and recall of information [1]. Checklists have proven effective in various aspects of performance improvement and error prevention and management [25].

The development of technical documents, such as checklists, requires a systematic and comprehensive approach, particularly when they are to be implemented in high intensity fields such as medicine. However, there is a relative paucity of published technical instructions for medical checklists. We searched for relevant English-language medical and non-medical literature both to describe where checklists have been demonstrated to improve care and delivery and also, how to develop valid checklists. Within the medical literature, the focus lies with the development of mnemonic devices and checklists outlining current evidence-based best practices, although the checklists themselves are rarely included for publication. Of the literature available in non-medical areas, the focus lies with the design of evaluative checklists and tools for performance measurement, rather than memory aids or goals sheets. Although a small number of strategies for designing effective checklists are referred to in the literature—including utilization of pre-published guidelines, formation of expert panels and repeat pilot-testing of preliminary checklists—a highly effective, standardized protocol for checklist development and design has yet to be developed and validated [6]. To that end, we have performed a narrative review of the most effective methods used in both medicine and other disciplines to design clear and effective checklists, as well as examples of their efficacy in various environments. We also outline available sources of support information. This perspective will delineate the standard components of successful checklists that can be used as a template in the development of case-specific medical checklists.

Methods

We performed a systematic search using MEDLINE (1966–2006) using the individual terms ‘checklist’, ‘goal sheet’, as well as the combined terms ‘checklist AND “memory OR mnemonic”'. Only English language article were included. The search yielded 8303 citations (Table 1). Citations were reviewed for their relevance to the development or implementation of checklists. Papers for extraction of information were then selected on the basis of qualitative inclusion criteria such as the description of a methodology for checklist development, the use of the checklist for process improvement, clinical support or as a patient safety tool, and the description of overall outcomes following implementation in a clinical area. We then selected relevant papers to provide examples of checklists in medicine and to describe valid generation of a checklist. We found no randomized, controlled trials evaluating checklist development. Additional sources were searched to capture relevant information from non-medical industries. This included Internet searches with basic search engines using the search terms ‘checklist’, ‘memory’ and ‘mnemonic’. As there is a rich history of checklist use in aviation and aerospace, we also searched the publicly available Federal Aviation Administration (FAA) and National Aeronautics and Space Administration (NASA) literature via their respective organizational web sites (www.faa.gov; www.nasa.gov). We searched relevant information from all forms of documentation and literature that described the design, development, implementation, overall utility and study of checklists in any field or industry. Information was selected for inclusion based on the overall credibility of the source, selecting peer-reviewed, university affiliated or government published documentation where possible. In all, 178 sources were selected for detailed review (Table 1).

View this table:
Table 1

Literature search

Embedded Image

This evidence was further supplemented with expert opinion and practical advice. Key content experts in leading areas of checklist research were approached for personal interview regarding their experiences with checklist development methodology, the formatting of technical information, implementation and evaluation of checklists and additional information relevant to their respective industries. This personal communication was used to further substantiate critical points identified in the literature.

Results and interpretation

Improving patient care: the benefits of medical checklists

High intensity fields of work, such as the airline industry and the military, already employ checklists to decrease errors of omission, improper implementation of procedures and protocols, and to decrease human error under stressful conditions [2, 3]. Similar to flight crews and military personnel, healthcare providers must often analyze and manage highly complex conditions under demanding and stressful conditions [7, 8]. Several areas of healthcare in which complex medical situations require rapid systematic approaches to crisis management, such as anaesthesiology and emergency medicine, therefore already make use of the types of checklists and memory-aids shown to be beneficial and life saving in the airline industry [9, 10]. Examples of published checklists currently employed regularly in the medical field include the checklist for diagnosing brain death [11], the Anaesthesia Gas Machine Checklist [12], the Checklist for the withdrawal of life support and end-of-life care [13] and the FAST HUG checklist of Clinical Best Practices [14].

The use of these checklists and memory-aids in clinical pathways has been shown to improve the quality of medical care [4]. Examples include the prediction of successful weaning from mechanical ventilation in ICU patients [15], adherence to evidence-based best practices [16] and in the improvement of patient safety in many clinical areas [1720].

Although the implementation of checklists has not always directly correlated with significant improvements in patient care and decreases in human error [21], no published data to date indicate that checklists may contributed to adverse events, such as imposing a burden on the primary care providers, delays in treatment because of lengthy checklists, or errors of omission. Rather, they are largely considered important tools to condense large quantities of knowledge in a concise fashion, reduce the frequency of errors of omission, create reliable and reproducible evaluations and improve quality standards and use of best practices [22]. However, there are instances in which excessive use of checklists could become a hindrance in the healthcare setting. If each detail of every task were targeted for the development of a checklist, clinicians may experience ‘checklist fatigue’, whereby they become overburdened with completing these lists. Rather than fulfilling their role as a support resource and error management tool, checklist use could begin to unnecessarily complicate processes and decrease reliability by adding a secondary layer of complexity. To suggest strict adherence to checklists in all situations is impractical and to do so could compromise the efficacy of a clinical process or procedure and risk infringing on efficient clinical judgment. Careful selection of checklist topics and consideration of clinical judgement in the content design process can help avoid these potential downfalls.

Checklists have been important tools in ensuring a consistent standard of care in the medical field. Examples include the use of standardized order set checklists and daily check sheets to increase the use of best practices [17, 23]. In one such instance, the implementation of daily checklists and reminders in clinical care pathways for inpatients admitted for acute myocardial infarction or stroke led to significant improvements in compliance with various key best practices such as administration of aspirin in the emergency department (21.4%; 95% CI; 7.3–32.7%), receipt of beta-blockers within 24 h of admission (48.1%; 95% CI; 31.4–64.8%), dysphagia screening within 24 h of admission (40.7%; 95% CI; 21.0–60.2%) and administration of aspirin or clopidogrel to ischaemic stroke patients within 24 h of admission (55.4%; 95% CI; 32.9–77.9%), as compared with the period prior to study [4]. By formulating case-specific checklists using evidence-based criteria and expert judgment, healthcare providers might be more comfortable knowing they are providing the proven best standard of patient care. If the use of a checklist or memory tool is subsequently proven to be useful in decreasing errors, and improving patient or process outcomes, it can be used to standardize a procedure across an organization or field.

Methodology of checklist development

What is a checklist?

A checklist is a list of action items, tasks or behaviours arranged in a consistent manner, which allows the evaluator to record the presence or absence of the individual items listed. Typically, each item is checked off as it is completed, verified, identified or answered, by placing a mark in a designated space. A sound checklist highlights the essential criteria that should be considered in a particular area. It helps the user not forget important criteria, achieve standardization of a process, and enhances an assessment's objectivity and reproducibility.

Despite many uses, checklists are commonly employed as either mnemonic devices or evaluative tools. Mnemonic checklists are typically used as a reminder system to help standardize normal, abnormal or complex procedures by calling to mind items, tasks or behaviours typically omitted during periods of stress or crisis [24]. The benefit of employing checklists as mnemonic devices lies in the fact that they provide an organizational framework for quick recall of critical information and current best practices. Evaluative checklists can be important tools in the standardization of evaluation by providing the user certain guidelines for the assessment, adding further credibility to and consistency among the evaluators [22].

Within a strictly medical context, checklists are among several tools used for process improvement and support of the multidisciplinary team. Other resources with a similar goal include Clinical Practice Guidelines (CPGs), Standardized Order Sets and Pre-printed Protocols or FlowCharts. CPGs provide a benchmark of what evidence-based best practice should be, while protocols systematically describe a precise and detailed plan or process. Flowcharts fall within the Diagnostic Checklists category, whereby a particular path is followed to determine an outcome. Although these documents are all complimentary and share several properties, checklists typically serve the more direct purpose of memory recall and may be more appropriate for certain discrete tasks. Primarily, medical checklists are meant to be mnemonic devices aimed at supporting the clinical team through a process with a general outline. They are generally more concise than protocols and may focus on one particular error-prone area and may therefore be considered more of a support tool than a directive, as are the others forms of clinical resources.

What should a checklist look like?

The goal of the checklist will define its structure and content. For instance, checklists can be designed for a defined action (i.e. design is based on the overall purpose of the checklist), such as those employed in the field of aeronautics for the standardization of system set-up, shutdown or emergency error correction [25]. These types of checklists are commonly used to facilitate the identification of errors of omission, by having several crew/team members run through the checklist in a systematic format to ensure each checkpoint is completed appropriately with the ultimate goal of properly setting-up a system or correcting an error. Several other types of checklists based on either the sequence in which the contents of the checklist are presented, or the overall purpose and intention of the checklist exist [22]. These are listed in Table 2.

View this table:
Table 2

Types of checklists [22]

Type of checklistDescriptionExample
Laundry listItems, tasks or criteria are grouped into related categories with no particular orderMedical equipment checklist
Grocery list
Sequential or weakly sequential checklistThe grouping, order and overall flow of the items, tasks or criteria are relevant in order to obtain a valid outcomeProcedure checklist (equipment must be gathered before procedure can begin)
Iterative checklistItems, tasks or criteria on the checklist require repeated passes or review in order to obtain valid results, as early checkpoints may be altered by results entered in later checkpointsContinued re-checking of the pulse and blood pressure in algorithms or checklists for adult cardiopulmonary resuscitation
Diagnostic checklistItems, tasks or criteria on the checklist are formatted based on a ‘flowchart’ model with the ultimate goal of drawing broad conclusionsClinical algorithms
Criteria of merit checklist (COM list)Commonly used for evaluative purposes, in which the order, categorization and flow of information is paramount for the objectivity and reliability of the conclusions drawnChecklist for diagnosis of brain death
Objective structured clinical examination checklist

How do I start?

A number of guides are available outlining the proper methods for the design and formatting of successful technical documents [2629]. These documents include an outline of the components of an effective checklist, the necessary steps to create proper checklists and support tools for evaluating the final design. The primary components of designing an effective checklist include basic requirements for context, content, structure, images and usability [26, 30]. Several other sources make similar recommendations on formatting-related issues such as ensuring that all content points of the document are accurate and evidence-based, employing a correct and consistent writing style relevant to the content and ensuring it is properly organized based on the ultimate goal of the checklist (evaluative versus mnemonic), clear and emphasizing the appropriate information [30, 31]. The visual effectiveness of a document refers to how the appearance of information and the use of visual elements can directly influence the overall efficacy of the document and the ease with which users can understand and use the final product [29]. Guidelines on visual effectiveness outlined by Hargis et al. include selecting appropriate graphics, balancing the number and placement of visual elements, appropriate use of colours and shading and textual elements [32]. Other important considerations outlined by key checklist developers include readability, employing nomenclature familiar in the domain of focus for efficient communication, creating a focus on the critical process items and closely considering the expertise of the target user in the development process (Boorman, November 2005).

Medical checklists require specific considerations for successful formatting, as listed in Table 3. Importantly, when formatting a medical checklist, real-time user activities and state of mind must be factored into the design. Busy clinicians may be calling upon the checklist under emergent conditions to help recall certain life-saving best practices or critical steps of an infrequently used procedure and will require the information to be straightforward and closely representative of their normal thought pattern.

View this table:
Table 3

Considerations for formatting a medical checklist

CriteriaConsideration
Context
  • Location of the checklist should be determined prior to development. If it is to be stored in a medical record upon completion, the checklist it will need to be processed through the appropriate hospital regulatory bodies.

Content
  • When possible, synthesis of published peer-reviewed guidelines and evidence-based best practices should be considered to form the body of the checklist.

    • Literature employed for generation of criteria points should be from a broad range of peer-reviewed, reliable sources and include perspectives of all types/disciplines that represent the continuum of intended users [36]. This is particularly important if the medical checklist is to be used by all personnel within a multidisciplinary team of healthcare providers.

  • Checklists should also reflect the local hospital and institution policies and procedures.

Structure
  • Checkpoints should be presented in a logical and functional order that reflect the sequence or flow of real-time clinician activities and regular patient care routines.

  • If the checklist is to be part of standard patient care, it might be important to include a checkpoint at the end where two users can sign off that it was completed.

    • e.g. space for nurses to confirm that the checklist was completed by the physician.

Images
  • Clear, equally spaced, bold fonts are suggested for letter differentiation and reading comprehension [37].

  • If colours are to be used—ensure that they are consistent with those commonly used in the intended environment.

    • e.g. if red is commonly associated with emergency situations, it should not be used to highlight text unless it is urgent information.

  • Checklists should include appropriate institutional logos or letterheads if necessary (e.g. if it is to be included in the medical record, it will have to match the format of other forms/orders).

Usability
  • Checklist should not be so onerous or time consuming as to notably interfere with administration of patient care.

  • Overall checklist should encompass checkpoints of major importance, while still providing clinicians with the freedom to use their own judgment.

  • Members of each discipline within a unit should pilot the checklist, particularly if the checklist is to be used by all personnel within a multidisciplinary team of health care providers.

  • Validation of the checklist should occur, where possible, within the appropriate simulated clinical environment.

Owing to the lack of description of the development of medical-specific checklists, we sought expert opinion from influential members of the aeronautic, medical and academic communities to elucidate the approach used to design the checklists that are useful in these work environments. A brief summary of the dialogues is outlined in Table 4.

View this table:
Table 4

Expert opinion on the checklist development process - summarized from personal communications

DisciplineSummary of process and expert opinion
Medicine
  • Co-authors of the Pocket Medicine program ‘Checklists in Internal Medicine’ for personal digital assistants [PDA] employed a general opinion-based methodology for the design and development of the checklists.

  • Content was determined based on professional medical experience, primary literature sources and/or peer-reviewed guidelines and consensus of colleagues in the field of interest.

  • Checkpoints were designed in a ‘question’ format to engage the user in an interactive manner, with similar topics or questions repeated for several of the conditions to create flow and consistency.

  • Flow of the checkpoints was arranged to reflect the approaches used in the general evaluation of patients.

  • The final structure of the checkpoints was designed based on size constraints presented by the PDA.

  • No pilot testing of the checklists was performed prior to their approval and programming due to time constraints.

Aeronautics
  • Technical Fellow in the Boeing Flight Technical Services group & designer of the electronic checklist used on Boeing 777 Aircraft.

  • Paper-based manual of checklists was used as the basis for developing the aircraft electronic checklist system.

  • Checklist content should take into consideration: readability aspects, simplicity requirements, critical items of focus, nomenclature familiar in the domain, expertise of the user.

  • Validation is KEY. Simulators are used with the checklist to repeatedly run the checklist in the anticipated arena.

  • Each part of a checklist should be real time and follow the thought process of the user.

  • It is important to understand why the user would want the checklist—and set-up the checklist [paper or electronic] accordingly to help, not hinder, the process covered by the checklist

Academia
  • Director and members of the Evaluation Centre (Western Michigan University) and authors of numerous checklist-related documents [22, 26, 27].

  • Checklist development should include the following considerations:

    • Strong external review of format, content, design;

    • Hearings to determine content (expert consensus);

    • Testing, rigorous validation processes;

    • Advisory board to drive and guide the development process;

    • Buy-in of users is integral in success of the checklist.

A detailed process guide to developing evaluative checklists, created by Stufflebeam et al. delineates the steps required to determine, classify and refine checklist content, to review and evaluate the efficacy of the checklist and the measures to maintaining a valid checklist [27]. Other sources also touch upon the processes by which a checklist or other discipline-specific technical documents come into fruition [30, 33, 34]. Several make similar recommendations with respect to format, design and organization of technical documents. Content development will depend on the context in which the checklist is to be used. However, components of the development process consistent with all disciplines include the early identification of the desired use of the checklist, rigorous evaluation of current literature and practices to create the core content points and rigorous validation of the checklist or technical document prior to implementation.

Although the processes outlined in the literature and expert consensus can be applied to the development of checklists in any field of work, (similar to the formatting requirements) there are particular considerations when creating a checklist for use by medical professionals. For instance, designers should understand the conditions under which the clinician would be referring to or completing the checklist in order to determine the appropriate content and flow. Similar medical-related points include:

  1. Ensuring that the time required to complete the revised checklist is feasible, practical and does not interfere with time-to-delivery of appropriate and safe patient care.

  2. Ensuring that the checklists pass through appropriate administrative or regulatory authorities (e.g. Hospital Medical Advisory Committee or Policy and Procedures).

  3. Provide clinicians with freedom to use their clinical judgment.

  4. Checklists should be reviewed frequently to reflect updates in the evidence-based medicine, published guidelines and institutional policies and procedures.

Fig. 1 outlines a clinical checklist of evidence-based procedural best practices, developed based on these considerations.

Figure 1

Sample medical checklist.

Figure 2

Key lessons learned for medical checklist development.

It is also important to consider the broader organizational processes and overall improvement goals when determining the focus, purpose and placement of a checklist. They are not appropriate in all environments and where they can be selectively implemented, should focus on key areas or tasks commonly prone to error or omission, so as to improve accuracy, adherence to best practice and overall process reliability.

Final considerations

Regardless of the systematic approach used to design and develop the perfect checklist, there are subsequent measures that, if not considered, could jeopardize the implementation of the checklist into practice. For instance, users must be properly trained on the use of the checklist in order to achieve optimal results. Target users must also have a full understanding of purpose of the checklist, whether it be evaluative or mnemonic, to avoid misinterpretation of checkpoints and erroneous answers. The checklist must be extensively piloted, ideally in simulated clinical environments and amongst the population of users in order to evaluate efficacy, practicality and overall need for the checklist (Boorman, November 2005). Checklist designers or authors might also consider developing an educational plan that introduces the main concepts of the checklist to the candidate users, in combination with a promotional plan to increase awareness of the checklist. The efficacy of a checklist at altering practice is ultimately dependent on the commitment by the users to employ the checklist. Usefulness of the checklist requires support of all staff, dedicated champions and endorsement by multidisciplinary leaders in clinical care [35]. Inclusion of the multidisciplinary team during the development of checklists not only improves endorsement, but also contributes to long-term sustainability following implementation and further emphasizes a team approach to completion and ‘cross-checking’ of the checklist items, a component that is integral in the aviation industry in preventing errors of omission.

Conclusions

Checklists can serve as important tools for decreasing medical error and improving overall standards of patient care, particularly during stressful conditions when memory, vigilance and cognitive functions can be affected. The development of effective checklists involves several important steps (Fig. 2). Legitimacy of the content will depend on the process for its development, and should include a thorough review and evaluation of the literature, evaluation of current practices and consideration of expert consensus, as well as a thorough validation of the checklist in the target user population prior to implementation of the final document. Checklist development should not be static, but an ongoing process involving expert groups, up-to-date literature, and feedback from the intended users as well as the target audience. When all staff members that might interact with the checklist have been involved in the process of creating and designing the checklist, there is a feeling of ownership of the checklist. The items contained in the final checklist represent a consensus between all members of the team, and improve implementation and uptake of a checklist into daily practice [28]. The lack of literature outlining the methodology and special considerations for developing medical-specific checklists has likely contributed to their continued absence in several key fields of medicine, despite evidence of their fundamental role in error management. Further areas of research in the utilization of checklists should focus on the evaluation of checklist fatigue in healthcare, the impact of checklist usage from the patient perspective, the continued evaluation of outcome improvements and direct tracking of error rates for the respective checklist focus.

References

View Abstract