OUP user menu

Effect of crew resource management training in a multidisciplinary obstetrical setting

Guy Haller , Philippe Garnerin , Michel-Ange Morales , Ricardo Pfister , Michel Berner , Olivier Irion , François Clergue , Christian Kern
DOI: http://dx.doi.org/10.1093/intqhc/mzn018 254-263 First published online: 6 May 2008

Abstract

Objective To assess the effect of a Crew Resource Management (CRM) intervention specifically designed to improve teamwork and communication skills in a multidisciplinary obstetrical setting.

Method Design-A before-and-after cross-sectional study designed to assess participants' satisfaction, learning and change in behaviour, according to Kirkpatrick's evaluation framework for training programmes. Setting-Labour and delivery units of a large university-affiliated hospital. Participants-Two hundred and thirty nine midwives, nurses, physicians and technicians from the department of anaesthesia, obstetrics and paediatrics. Intervention-All participants took part in a CRM-based training programme specifically designed to improve teamwork and communication skills. Principal measures of outcome-We assessed participants' satisfaction by means of a 10-item standardized questionnaire. A 36-item survey was administered before and after the course to assess participants' learning. Behavioural change was assessed by a 57-item safety attitude questionnaire measuring staff's change in attitude to safety over 1 year of programme implementation.

Results Most participants valued the experience highly and 63–90% rated their level of satisfaction as being very high. Except for seven items, the 36-item survey testing participants' learning demonstrated a significant change ( <  0.05) towards better knowledge of teamwork and shared decision making after the training programme. Over the year of observation, there was a positive change in the team and safety climate in the hospital [odds ratio (OR) 2.9, 95% confidence interval (CI) (1.3–6.3) to OR 4.7, 95% CI (1.2–17.2)]. **There was also improved stress recognition [OR 2.4, 95% CI (1.2–4.8) to OR 3.0, 95% CI (1.0–8.8)].

Conclusion The implementation of a training programme based on CRM in a multidisciplinary obstetrical setting is well accepted and contributes to a significant improvement in interprofessional teamwork.

Keywords
  • patient safety
  • programme evaluation
  • teamwork
  • training programmes

Introduction

Studies have shown that from 3 to 16% of patients will experience an adverse event during their stay in hospital [14]. Between 28 and 51% of these events are considered as preventable [1, 2, 4]. Most preventable events are caused by human error [5]. Weaknesses of the human factor have been largely studied in aviation, where it is known since the early 1980s that human deficiencies, particularly, poor teamwork among cockpit crews, contribute to half of the accidents [6]. Therefore, teamwork skills and error-management teaching programmes have become a mandatory part of most flight-crew training programmes [7]. These programmes are commonly named as Crew Resource Management (CRM) training programmes. They put a strong emphasis on the process of training crews to communicate and coordinate as a team and reduce errors by making better use of human resources on the flight deck, especially during crisis situations [810]. CRM aims at developing shared behaviours to improve safety (team resources rather than individual resources). Although well known in aviation this safety-improvement method is at an early stage in the healthcare context. It has been used mainly in limited settings and virtual environments to train anaesthetists and neonatologists to deal with crisis scenarios [1113], emergency physicians to build effective teams [1417], and medical and nursing students to improve their communication and cooperation skills [18]. However, many of these CRM-based training programmes have been restricted to the use of simulated scenarios in virtual environments, or limited settings such as emergency-department units [12, 14, 1618]. Only one [15] of these CRM-based programmes has been assessed beyond participants' satisfaction [11, 12, 18]. As a consequence the true impact of CRM-based training programmes, particularly in multidisciplinary settings, is unclear.

The purpose of this study was to assess the effect of a CRM-based training programme specifically designed to improve teamwork and communication skills in the multidisciplinary setting of a large University-affiliated women's hospital. This programme aimed to increase interprofessional collaboration between anaesthetists, obstetricians, paediatricians, midwives and nurses in labour and delivery units, an area where teamwork deficiencies are the main cause of 67% of maternal deaths and 30% of neonatatal complications [19, 20].

Methods

Setting and programme

The study was performed at the women's hospital of the Geneva University Hospitals, Switzerland. The hospital performs around 4000 childbirths per year and acts as a referral centre for the surrounding area, extending also into France.

The assessed CRM-based training programme was set up in 2003 by a working party of healthcare professionals, cognitive psychologists and aviation safety experts (Dedale-Paris®). Details of its development and implementation have been described elsewhere [21]. The first sessions of the programme started in January 2004. It was set up as a 2-day inter-trade seminar for groups of 12 persons chaired by two of their peers. Nurses, physicians, midwives, and technicians from the clinics of anaesthesia, obstetrics, and paediatrics were all involved in the programme. During 2 days healthcare professionals were invited to the Hospital training centre, located outside the main hospital building. Participation was compulsory. Seminars started on the first day with a short introduction followed by a film showing some critical situations during a busy day in the delivery unit. It was used to open the discussion on common patient safety issues experienced by program participants during everyday clinical practice. This was followed by lectures aimed at improving participants understanding and theoretical knowledge on patient safety and improvement methods. This part of the seminar also included a workshop on professional roles and impact on everyday practice. The first day ended by an interactive lecture on common communication issues, using some examples from the film.

The second day started with a series of workshops and play roles aimed at highlighting each other's expectations and common misunderstandings between healthcare professionals from different specialties. This was integrated to a lecture aimed at improving participants' understanding of the impact of stress, particularly, on interprofessional communication. This was followed by an interactive course on interprofessional collaboration and team resolution of crisis scenarios. The 2 days program ended by a final session aimed at soliciting team improvement strategies to be implemented into the daily practice. Details of the 2-days training programme are provided in Fig. 1.

Figure 1

Details of the 2-days training programme.

Measurement

The Geneva University Hospitals Human Research and Ethics Committee was contacted, but as the overall project was defined as a quality-improvement activity and no patients were directly involved, it was considered to be exempt from any formal Institutional Ethics committee review. To assess the impact of this training we used the Kirkpatrick's evaluation framework for training programmes [22]. This includes four hierarchical dimensions:

  1. Reactions (do participants like the programme?)

  2. Learning (do participants learn from the programme?)

  3. Behaviour (do participants apply what they have learned from the programme?)

  4. Organizational impact (is there increased safety and are there fewer errors in the organization following implementation of the programme?)

All participants in the training programme between January 2004 and December 2004 were included in the investigation. We performed a cross-sectional study to assess participants' satisfaction, learning and change in behaviour.

Participants' reactions (Kirkpatrick's dimension 1) were measured by the standardized questionnaire developed by the Hospital training centre. This 10-item questionnaire explores four different areas of participants' satisfaction: satisfaction with the course organization (overall organization), course content (objectives reached; expectations satisfied; content adequate; learning adequate), group dynamic (number of participants; interaction with other participants; group composition) and method (overall method; technical support). It uses a 4-point Likert scale (dissatisfied to very satisfied).

For learning assessment (Kirkpatrick's dimension 2), we developed a 36-item questionnaire exploring different aspects of the course content, particularly, teamwork building, shared decision making in emergency situations, and other methods of improving patient safety. Answers were measured on a 5-point Likert scale (strongly disagree to strongly agree). The questionnaire was administered to participants before the beginning of the training course and at the end of the course.

We assessed behavioural change (Kirkpatrick's dimension 3) by measuring the change in staff attitude to safety over the training year, using the safety attitude questionnaire (SAQ-labour & delivery version©) developed by the University of Texas [23]. This was administered over one year to staff members registered for the course on their arrival at the training centre. The 57-item questionnaire is based on a 5-point Likert scale (strongly disagree to strongly agree) and it explores six dimensions of organizational safety: teamwork climate, stress climate, job satisfaction, stress recognition, perceptions of management and working conditions.

Statistical analysis

We performed all analyses using the Statistical Package for Social Sciences (SPSS-Version 12.0.1 SPSS Inc, Chicago, IL, USA). For descriptive analyses of participants' characteristics and responses to the items on the learning questionnaire we used percents and mean score. Before-and-after comparisons for participants' characteristics, learning and safety attitude were performed, for continuous data, with the t-test and Wilcoxon rank-sum test, depending on data distribution. For categorical variables we used the χ2 test and binary logistic regression. For the SAQ questionnaire, answers were dichotomized (agree/disagree) and negative items reverted. The year of observation was divided into three equal periods. The middle and end tertiles of the training year were compared with the first tertile, representing the beginning of the training programme and used as a reference. We used odds ratio (OR) with 95% confidence interval (CI); an OR > 1 represents a better agreement for the item when compared with the period of reference. Results are reported as proportions, means with standard deviation and OR, with 95% CI. A value of P < 0.05 was considered significant.

Results

Between January 2004 and April 2005, 239 participants attended the 24 seminars organized: 40.6% were nurses, 26.4% physicians, 25.9% midwives, 2.9% technicians and 3.8% observers and managers from the three departments. The majority of participants were female (79.5%). All specialties were equally represented: 75% of participants had more than 4 years of professional experience and 67% more than 3 years of professional practice in the hospital. Most were working on flexible shifts (day, evening or night shifts). There were no differences between participants' baseline characteristics at the beginning and end of the year 2004, except for the type of work shift and the number of years of professional practice in the hospital. Results are available in Table 1.

View this table:
Table 1

Participants characteristics

Reactions (Kirkpatrick's dimension 1)

The satisfaction questionnaire was completed by 74.9% of the participants. Most valued the experience and participants rated their satisfaction as very high in the proportion of 90% for course organization, 63.5–71% for course content, 79–81% for teaching method and 69–79% for items related to the group dynamics.

Learning (Kirkpatrick's dimension 2)

The 36-item questionnaire testing learning was completed by 74.1% of the participants. It demonstrated a statistically significant change (P < 0.05) towards better understanding of teamwork and shared decision-making in emergency situations. The seven unchanged items related to the assessment of knowledge regarding formal theoretical concepts of patient safety, for instance the impact of fatigue or stress, and uncertainty management. Results are described in Table 2.

View this table:
Table 2

Participants learning before and after course

Behaviour (Kirkpatrick's dimension 3)

The safety-attitude questionnaire was completed by 94.9% of the participants. After 1 year of implementation of the programme, a positive change in the team and safety climate was measured. This change was observed in items exploring the use of briefing and coordination methods as well as in the implementation of improvement initiatives following the use of incident reports [OR 2.9, 95% CI (1.3–6.3) to OR 4.7, 95% CI (1.2–17.2)]. There was also improved stress recognition [OR 2.4, 95% CI (1.2–4.8) to OR 3.0, 95% CI (1.0–8.8)], working conditions and job satisfaction. Participants reported improved availability of information [OR 2.4, 95% CI (1.0-5.7)] and the feeling of being part of a big family [OR 2.1, 95% CI (1.0-4.4)]. A negative change was observed during the second period in the dimension of perception of management of the questionnaire, in items exploring support and information provision [OR 0.2, 95% CI (0.1–0.6) to OR 0.4, 95% CI (0.1–0.8)]. This change disappeared during the third period. For the purpose of clarity, we report in Table 3, the dimensions of the safety attitude questionnaire related to the teamwork-safety climate and stress recognition. The full item list is available on the web supplement of this article (See online supplementary material).

View this table:
Table 3

Organizational safety culture change over 1 year of program implementation

Discussion

To improve interprofessional teamwork and communication, the adaptation of the CRM learning technique from aviation to a multidisciplinary healthcare setting proved to be feasible and effective. Participants were largely satisfied with the training programme developed and most valued the experience. From 63 to 90% of participants rated their level of satisfaction as very high for overall course organization, course content, teaching method and knowledge acquisition. Except for seven items related to formal theoretical concepts of stress, fatigue and uncertainty management, the 36-item questionnaire testing participants' learning demonstrated a significant change towards better understanding of teamwork and shared decision making at the end of the 2-day training programme.

Over 1 year of implementation of the programme, there was a positive change in the team and safety climate in the hospital as well as improved stress recognition.

To improve teamwork and communication in healthcare settings, a number of CRM-based programmes have been developed [1118]. However, only a few studies analysed their effectiveness and most limited their assessment to participants' reactions [11, 12, 14, 18]. In one, authors found that 100% of participants would value further similar training and indicated that the course would make them change their usual practice [14]. In another, it was found that 75–97% participants highly valued the course and rated their satisfaction with learning content at 4 and 5 on a 5-level rating scale [12]. In another, using focus groups, Kyrkjebo et al. [18] found that students were satisfied with the programme, although some of them found that the course content, particularly, videos and simulation exercises, could be more realistic. These results are comparable with our study where we found that 63–90% of participants reported being very satisfied with the overall course organization, content, teaching method and learning. This confirms that CRM-based training programmes are highly valued by participants and that this approach is popular amongst healthcare professionals. Only one programme included an assessment of further levels of the Kirpatrick's evaluation framework. Morey et al. [15] found a statistically significant improvement in the quality of team behaviour, work performance and reduction of errors following the implementation of a CRM-based training programme in emergency units. Except for error reduction (not assessed in our study), these results are similar to our findings. However, a key characteristic of their programme was the use a team-based staffing approach to day and nightshift organization. Physicians, nurses and technicians were systematically allocated to one specific team, which would be listed together. This may have introduced other factors beyond the content of their CRM-based programme, which may explain some of their findings. Our programme was developed to address routine practice communication and coordination issues amongst professionals from different specialties involved in obstetrical care. It was designed to take into account the existence of ‘temporary’ teams of professionals which may work together for 1 day or night only, every week or 2 weeks, a phenomenon amplified by the introduction of working-hours limitations for interns and residents [24, 25].

As a consequence, it emphasized the teaching of portable skills, usable in ‘temporary teams’ and various circumstances [14]. This was done by the organization of different workshops aimed at improving participants' understanding of differences in professional culture and the resulting expectations. This is how, for instance, participants discovered that a significant source of conflict between anaesthetists and midwives was that midwives expected anaesthetists to be quickly available for their patients in the delivery unit while anaesthetists expected midwives to be aware of their other commitments in the hospital.

Another aspect developed in our CRM-based training programme was the organization of workshops to train participants in improving their communication skills. For instance, participants were trained to describe and report as accurately as possible complex clinical problems. These workshops were particularly valued by participants and they led to a significant number of improvement initiatives (165 since the beginning of the programme). Some examples include the implementation of an emergency procedure to call paediatricians, the development of a multidisciplinary critical incidents review committee, and the implementation of communication standards to prepare all the personnel involved for an emergency caesarean section. This was confirmed by the analysis of the questionnaire on organizational safety climate where we found a significant improvement in items related to the use of briefing and coordination methods as well as the implementation of improvement initiatives following improvements in the communication of incident reports.

Effective communication plays a key role in the improvement of patient safety and there is some evidence in the literature that standardization of communication can improve patient outcomes [26]. However, efforts to improve communication should not be limited to the use of checklists and communication codes. The CRM approach emphasizes the use of ‘all available resources by the flight-deck crew’ [8]. This includes the development of systematic briefing sessions, the training of personnel to challenge and monitor each others' actions appropriately, and the implementation of formal education courses on recognized issues related to patient safety (i.e. fatigue, stress, workload) [27]. This latter formal educational aspect was not pre-eminent in our CRM-based training programme, which emphasized the acquisition of knowledge through practical workshops and particularly around communication issues, demonstrated to be a critical dimension of safety in labour and delivery units [28]. This may explain why participants did not demonstrate improved knowledge on formal theoretical aspects of methods to improve patient safety, except for stress recognition.

We also found only marginal change at the end of the training year for the job-satisfaction, and safety climate dimensions of the SAQ-questionnaire. Several reasons may explain this phenomenon. First, at the end of our data-collection process not all staff members of the hospital had been trained by CRM-based techniques. Overall changes observed in the organizational teamwork climate and working conditions mainly reflected the halo effect of improvement initiatives developed by the first groups of participants in the training programme. If the impact of the programme had been assessed later, other changes may have perhaps been observed. Secondly, our programme emphasized the development of teamwork skills. As a consequence it is likely that other aspects such as job satisfaction may not have been affected. Interestingly, the perception of the support provided by management got worse during the second trimester, while becoming non-significant by the end of the training year. This may be related to increased activity in the labour unit (10% by the end of year 2004) and the negative impact of the summer period, when many staff members are on holiday and the significant increase in the workload for those who remain.

This study has several limitations. First, it was undertaken in a single hospital and analysed a specific CRM-based training programme. Its generalization to other CRM-based programmes in other multidisciplinary settings may be limited. Furthermore, the participants in the programme we assessed were not randomly chosen. Results may reflect a possible selection bias of participants more inclined to participate, enjoy and learn from such a training programme. However, participation in the programme was compulsory for all staff members of the hospital working in the labour and delivery unit. Only if the workload of a participant's department was too high was he allowed to cancel his participation in a seminar and was automatically registered for another one.

Secondly, the questionnaire assessing learning is not a validated measurement tool. Results may reflect other aspects than participant's level of learning (i.e. questionnaire reliability issues). However, due to the specific characteristics of our programme there were no validated tools available to assess its content explicitly. Knowledge assessment by testing participants' acquisition of pre-defined concepts is a commonly accepted testing method and is largely used by schools and universities. Furthermore, the presence of a large number of significant changes on items testing specifically the course content suggests that changes of scores observed are more likely to be due to participants' learning than to reliability issues of the questionnaire.

Thirdly, we attributed improvement to the teamwork climate in the hospital to our CRM-based training programme. Other factors may have confounded the analysis. These may include differences in staff profiles, departmental activities, seasonal profiles and resource allocation between the two periods of assessment of the organizational safety climate. Furthermore, to measure the third dimension of Kirkpatrick's model, we used a surrogate marker of behavioural change, the modification of staff safety attitude following the training programme. This may only assess a fraction of what may represent a true change of staff behaviour. Additional approaches, such as video recording or direct observation, could have been used to assess participants' use of CRM skills in clinical settings. Finally, if it could be demonstrated that participants to a CRM-based training programme are satisfied, learn new skills and use them in their everyday practice, resulting in improvements of the organizational teamwork climate, we were unable to demonstrate that fewer human errors and improved patient safety resulted from the CRM approach (fourth dimension of Kirkpatrick's evaluation framework) [22].

Furthermore behavioural change would probably have been better assessed. Further research could look for instance at the development of safety-measurement tools able to identify, in medical charts or administrative databases, iatrogenic injuries and errors and to compare their incidence before and after implementation of the CRM-based programme.

Despite these limitations, the findings of this study indicate that the implementation of a CRM-based training programme in a multidisciplinary environment is an effective way to improve interprofessional teamwork and communication. Professionals highly value the CRM-based approach and significantly improve their knowledge of teamwork and shared decision-making. This improves the organizational teamwork and safety climates and most likely overall patient safety. CRM-based training programmes should be strongly promoted not only for the management of crisis scenarios but also for routine practice in multidisciplinary environments where a high level of coordination and collaboration is required beyond the boundaries of professional cultures.

Acknowledgements

The authors would like to thank Dr Laure Kloetzer (cognitive psychologist) and Mr Jean Pariès (aviation safety engineer) – Dédale/Paris® who advised on and guided the development of this programme. We also would like to thank Mrs M. Ariès for her help in the data-collection process, Mr Alec Hester for his editorial advices and all staff members of the three departments involved for their enthusiastic contribution.

References

View Abstract