OUP user menu

Effect of crew resource management training in a multidisciplinary obstetrical setting

Guy Haller, Philippe Garnerin, Michel-Ange Morales, Ricardo Pfister, Michel Berner, Olivier Irion, François Clergue, Christian Kern
DOI: http://dx.doi.org/10.1093/intqhc/mzn018 254-263 First published online: 6 May 2008

Abstract

Objective To assess the effect of a Crew Resource Management (CRM) intervention specifically designed to improve teamwork and communication skills in a multidisciplinary obstetrical setting.

Method Design-A before-and-after cross-sectional study designed to assess participants' satisfaction, learning and change in behaviour, according to Kirkpatrick's evaluation framework for training programmes. Setting-Labour and delivery units of a large university-affiliated hospital. Participants-Two hundred and thirty nine midwives, nurses, physicians and technicians from the department of anaesthesia, obstetrics and paediatrics. Intervention-All participants took part in a CRM-based training programme specifically designed to improve teamwork and communication skills. Principal measures of outcome-We assessed participants' satisfaction by means of a 10-item standardized questionnaire. A 36-item survey was administered before and after the course to assess participants' learning. Behavioural change was assessed by a 57-item safety attitude questionnaire measuring staff's change in attitude to safety over 1 year of programme implementation.

Results Most participants valued the experience highly and 63–90% rated their level of satisfaction as being very high. Except for seven items, the 36-item survey testing participants' learning demonstrated a significant change ( <  0.05) towards better knowledge of teamwork and shared decision making after the training programme. Over the year of observation, there was a positive change in the team and safety climate in the hospital [odds ratio (OR) 2.9, 95% confidence interval (CI) (1.3–6.3) to OR 4.7, 95% CI (1.2–17.2)]. **There was also improved stress recognition [OR 2.4, 95% CI (1.2–4.8) to OR 3.0, 95% CI (1.0–8.8)].

Conclusion The implementation of a training programme based on CRM in a multidisciplinary obstetrical setting is well accepted and contributes to a significant improvement in interprofessional teamwork.

Keywords
  • patient safety
  • programme evaluation
  • teamwork
  • training programmes

Introduction

Studies have shown that from 3 to 16% of patients will experience an adverse event during their stay in hospital [14]. Between 28 and 51% of these events are considered as preventable [1, 2, 4]. Most preventable events are caused by human error [5]. Weaknesses of the human factor have been largely studied in aviation, where it is known since the early 1980s that human deficiencies, particularly, poor teamwork among cockpit crews, contribute to half of the accidents [6]. Therefore, teamwork skills and error-management teaching programmes have become a mandatory part of most flight-crew training programmes [7]. These programmes are commonly named as Crew Resource Management (CRM) training programmes. They put a strong emphasis on the process of training crews to communicate and coordinate as a team and reduce errors by making better use of human resources on the flight deck, especially during crisis situations [810]. CRM aims at developing shared behaviours to improve safety (team resources rather than individual resources). Although well known in aviation this safety-improvement method is at an early stage in the healthcare context. It has been used mainly in limited settings and virtual environments to train anaesthetists and neonatologists to deal with crisis scenarios [1113], emergency physicians to build effective teams [1417], and medical and nursing students to improve their communication and cooperation skills [18]. However, many of these CRM-based training programmes have been restricted to the use of simulated scenarios in virtual environments, or limited settings such as emergency-department units [12, 14, 1618]. Only one [15] of these CRM-based programmes has been assessed beyond participants' satisfaction [11, 12, 18]. As a consequence the true impact of CRM-based training programmes, particularly in multidisciplinary settings, is unclear.

The purpose of this study was to assess the effect of a CRM-based training programme specifically designed to improve teamwork and communication skills in the multidisciplinary setting of a large University-affiliated women's hospital. This programme aimed to increase interprofessional collaboration between anaesthetists, obstetricians, paediatricians, midwives and nurses in labour and delivery units, an area where teamwork deficiencies are the main cause of 67% of maternal deaths and 30% of neonatatal complications [19, 20].

Methods

Setting and programme

The study was performed at the women's hospital of the Geneva University Hospitals, Switzerland. The hospital performs around 4000 childbirths per year and acts as a referral centre for the surrounding area, extending also into France.

The assessed CRM-based training programme was set up in 2003 by a working party of healthcare professionals, cognitive psychologists and aviation safety experts (Dedale-Paris®). Details of its development and implementation have been described elsewhere [21]. The first sessions of the programme started in January 2004. It was set up as a 2-day inter-trade seminar for groups of 12 persons chaired by two of their peers. Nurses, physicians, midwives, and technicians from the clinics of anaesthesia, obstetrics, and paediatrics were all involved in the programme. During 2 days healthcare professionals were invited to the Hospital training centre, located outside the main hospital building. Participation was compulsory. Seminars started on the first day with a short introduction followed by a film showing some critical situations during a busy day in the delivery unit. It was used to open the discussion on common patient safety issues experienced by program participants during everyday clinical practice. This was followed by lectures aimed at improving participants understanding and theoretical knowledge on patient safety and improvement methods. This part of the seminar also included a workshop on professional roles and impact on everyday practice. The first day ended by an interactive lecture on common communication issues, using some examples from the film.

The second day started with a series of workshops and play roles aimed at highlighting each other's expectations and common misunderstandings between healthcare professionals from different specialties. This was integrated to a lecture aimed at improving participants' understanding of the impact of stress, particularly, on interprofessional communication. This was followed by an interactive course on interprofessional collaboration and team resolution of crisis scenarios. The 2 days program ended by a final session aimed at soliciting team improvement strategies to be implemented into the daily practice. Details of the 2-days training programme are provided in Fig. 1.

Figure 1

Details of the 2-days training programme.

Measurement

The Geneva University Hospitals Human Research and Ethics Committee was contacted, but as the overall project was defined as a quality-improvement activity and no patients were directly involved, it was considered to be exempt from any formal Institutional Ethics committee review. To assess the impact of this training we used the Kirkpatrick's evaluation framework for training programmes [22]. This includes four hierarchical dimensions:

  1. Reactions (do participants like the programme?)

  2. Learning (do participants learn from the programme?)

  3. Behaviour (do participants apply what they have learned from the programme?)

  4. Organizational impact (is there increased safety and are there fewer errors in the organization following implementation of the programme?)

All participants in the training programme between January 2004 and December 2004 were included in the investigation. We performed a cross-sectional study to assess participants' satisfaction, learning and change in behaviour.

Participants' reactions (Kirkpatrick's dimension 1) were measured by the standardized questionnaire developed by the Hospital training centre. This 10-item questionnaire explores four different areas of participants' satisfaction: satisfaction with the course organization (overall organization), course content (objectives reached; expectations satisfied; content adequate; learning adequate), group dynamic (number of participants; interaction with other participants; group composition) and method (overall method; technical support). It uses a 4-point Likert scale (dissatisfied to very satisfied).

For learning assessment (Kirkpatrick's dimension 2), we developed a 36-item questionnaire exploring different aspects of the course content, particularly, teamwork building, shared decision making in emergency situations, and other methods of improving patient safety. Answers were measured on a 5-point Likert scale (strongly disagree to strongly agree). The questionnaire was administered to participants before the beginning of the training course and at the end of the course.

We assessed behavioural change (Kirkpatrick's dimension 3) by measuring the change in staff attitude to safety over the training year, using the safety attitude questionnaire (SAQ-labour & delivery version©) developed by the University of Texas [23]. This was administered over one year to staff members registered for the course on their arrival at the training centre. The 57-item questionnaire is based on a 5-point Likert scale (strongly disagree to strongly agree) and it explores six dimensions of organizational safety: teamwork climate, stress climate, job satisfaction, stress recognition, perceptions of management and working conditions.

Statistical analysis

We performed all analyses using the Statistical Package for Social Sciences (SPSS-Version 12.0.1 SPSS Inc, Chicago, IL, USA). For descriptive analyses of participants' characteristics and responses to the items on the learning questionnaire we used percents and mean score. Before-and-after comparisons for participants' characteristics, learning and safety attitude were performed, for continuous data, with the t-test and Wilcoxon rank-sum test, depending on data distribution. For categorical variables we used the χ2 test and binary logistic regression. For the SAQ questionnaire, answers were dichotomized (agree/disagree) and negative items reverted. The year of observation was divided into three equal periods. The middle and end tertiles of the training year were compared with the first tertile, representing the beginning of the training programme and used as a reference. We used odds ratio (OR) with 95% confidence interval (CI); an OR > 1 represents a better agreement for the item when compared with the period of reference. Results are reported as proportions, means with standard deviation and OR, with 95% CI. A value of P < 0.05 was considered significant.

Results

Between January 2004 and April 2005, 239 participants attended the 24 seminars organized: 40.6% were nurses, 26.4% physicians, 25.9% midwives, 2.9% technicians and 3.8% observers and managers from the three departments. The majority of participants were female (79.5%). All specialties were equally represented: 75% of participants had more than 4 years of professional experience and 67% more than 3 years of professional practice in the hospital. Most were working on flexible shifts (day, evening or night shifts). There were no differences between participants' baseline characteristics at the beginning and end of the year 2004, except for the type of work shift and the number of years of professional practice in the hospital. Results are available in Table 1.

View this table:
Table 1

Participants characteristics

Participants characteristicsOverall [N (%)] (n =239)First period [N (%)] (n =83)Second period [N (%)] (n =86)Third period [N (%)] (n =70)
Age (years)
 24–31139 (30.2)22 (26.8)25 (30.5)24 (34.3)
 32–39163 (35.4)34 (41.5)27 (32.9)22 (31.4)
  >40158 (34.3)26 (31.7)30 (36.6)24 (34.3)
Gender
 Female190 (79.5)67 (80.7)68 (79.1)55 (78.6)
 Male49 (20.5)16 (19.3)18 (20.9)15 (21.4)
Profession
 Nurse97 (40.6)35 (42.2)38 (44.7)24 (34.3)
 Physician63 (26.4)23 (27.7)21 (24.7)19 (27.1)
 Midwife62 (25.9)20 (24.1)21 (24.7)21 (30.0)
 Technician7 (2.9)5 (6.0)1 (1.2)1 (1.4)
 Other9 (3.8)04 (4.7)5 (7.1)
Specialty
 Obstetrics83 (34.7)29 (34.9)28 (32.9)26 (37.1)
 Anaesthesia75 (31.4)31 (37.3)24 (28.2)20 (28.6)
 Paediatrics71 (29.7)23 (27.7)29 (34.1)19 (27.1)
 Other9 (3.8)0 4 (4.7)5 (7.1)
Years of professional experience
 1–4117 (25.2)23 (27.7)19 (22.9)17 (24.3)
 5–9186 (40.1)33 (39.8)37 (44.6)26 (37.1)
  >10161 (34.7)27 (32.5)27 (32.5)27 (38.6)
Years working for the hospital *
 1–3153 (33.0)28 (33.7)27 (32.5)23 (32.9)
 4–7138 (29.7)28 (33.7)27 (32.5)14 (20.0)
  >8173 (37.3)27 (32.5)29 (34.9)33 (47.1)
Type of work
 Day time only (7–19 h)29 (12.1)2 (2.4)*10 (11.6) *17 (24.3) *
 Evening–Night time only (19–7 h)9 (3.8)4 (4.8)2 (2.3)3 (4.3)
 Flexible201 (84.1)77 (92.8)74 (86.0)50 (71.4)
Full-time148 (61.9)50 (60.2)55 (64.7)43 (61.4)
Part-time90 (37.7)33 (39.8)30 (35.3)27 (38.7)
  • *P < 0.05.

Reactions (Kirkpatrick's dimension 1)

The satisfaction questionnaire was completed by 74.9% of the participants. Most valued the experience and participants rated their satisfaction as very high in the proportion of 90% for course organization, 63.5–71% for course content, 79–81% for teaching method and 69–79% for items related to the group dynamics.

Learning (Kirkpatrick's dimension 2)

The 36-item questionnaire testing learning was completed by 74.1% of the participants. It demonstrated a statistically significant change (P < 0.05) towards better understanding of teamwork and shared decision-making in emergency situations. The seven unchanged items related to the assessment of knowledge regarding formal theoretical concepts of patient safety, for instance the impact of fatigue or stress, and uncertainty management. Results are described in Table 2.

View this table:
Table 2

Participants learning before and after course

ItemsBeginning of programme Mean (SD)End of programme Mean (SD)Change difference (95% CI)P-value
Q1. It is possible to perceive when colleagues start to loose control of the situation3.90 (0.88)4.34 (0.58)0.44 (0.28–0.59)<0.001
Q2. Predicting what colleagues will do, improves patient safety4.32 (0.88)4.65 (0.53)0.33 (0.17–0.48)<0.001
Q3. To ensure patient safety, a shared knowledge of workload and available human resources is necessary4.39 (0.69)4.68 (0.76)0.29 (0.14–0.43)<0.001
Q4. Human error cannot be avoided; it is important to learn how to deal with the consequences of human error4.20 (1.13)4.61 (0.76)0.41 (0.20–0.60)<0.001
Q5. Not being aware of other professionals' availabilities can negatively impact on patient safety4.58 (0.69)4.68 (0.58)0.10 (−0.02–0.23)0.11
Q6. A good professional does not make mistakes1.55 (0.79)1.21 (0.54)−0.34 (−0.20 to −0.46)<0.001
Q7. Most accidents during patient care are due to a lack of professional skills2.24 (1.0)1.90 (0.91)−0.34 (−0.14 to −0.52)<0.001
Q8. Rephrase others' sentences is a good way to ensure proper understanding4.49 (0.72)4.77 (0.54)0.28 (0.17–0.39)<0.001
Q9. Shared understanding of work processes relies on shared technical tools such as a blackboard for instance3.79 (1.04)4.51 (0.72)0.72 (0.52–0.91)<0.001
Q10. Asking for help is a sign of a lack of skills1.18 (0.57)1.18 (0.62)01.0
Q11. To ensure patient safety I must make sure others have properly understood my wording3.39 (0.75)4.72 (0.49)0.33 (0.19–0.46)<0.001
Q12. When in doubt, asking a colleague is a proof of ineptitude4.04 (1.26)3.97 (1.51)−0.07 (−0.37–0.22)0.62
Q13. To improve patient safety, I need to assess others' availabilities4.42 (0.75)4.65 (0.60)0.23 (0.08–0.37)0.01
Q14. Conflicts between healthcare professionals is deleterious to patients4.32 (0.91)4.48 (0.70)0.16 (−0.01–0.33)0.06
Q15. In emergency situations, only patients' condition matters3.15 (1.35)2.97 (1.32)−0.18 (−0.44–0.08)0.17
Q16. Understanding other professionals' tasks is a good way to improve mutual collaboration4.64 (0.68)4.83 (0.52)0.19 (0.09–0.29)<0.001
Q17. It can be justified to work in crisis situations for other reasons than mothers and newborn babies health condition3.21 (1.28)3.20 (1.36)−0.1 (−0.31–0.28)0.93
Q18. To ensure patient safety, it is needed to take into account other professionals' needs4.25 (0.74)4.48 (0.75)0.23 (0.08–0.37)0.002
Q19. To improve ones' knowledge it is necessary to be able to talk about ones' mistakes4.55 (0.92)4.77 (0.54)0.22 (0.06–0.38)0.01
Q20. Disfunctionning teams are a major cause of accidents3.81 (1.09)4.02 (0.88)0.21 (0.01–0.41)0.03
Q21. To improve patient care, each healthcare professional should be able to express ones' own perspective4.34 (1.08)4.53 (0.78)0.19 (−0.09–0.40)0.06
Q22. To develop a shared understanding of the different degrees of an emergency situation makes team coordination easier4.49 (1.06)4.87 (0.59)0.38 (0.20–0.54)<0.001
Q23. Accidents are always due to negligence2.73 (1.13)2.71 (1.19)−0.02 (−0.26–0.20)0.80
Q24. Patient safety is not affected by interruptions2.30 (1.13)2.07 (0.95)−0.23 (−0.46–0.03)0.05
Q25. Patient safety is before all a team matter4.29 (1.06)4.53 (0.75)0.24 (0.05–0.42)0.01
Q26. Shared preparation to deal with complex situations improves work processes4.41 (0.99)4.73 (0.57)0.32 (0.14–0.49)<0.001
Q27. A competent leader is able to explain his decisions4.18 (1.13)4.45 (0.91)0.26 (0.05–0.47)0.01
Q28. Accidents often result of successive misshaps4.23 (1.02)4.55 (0.80)0.32 (0.12–0.50)0.01
Q29. A good professional is not sensible to stress1.71 (0.97)1.49 (0.80)−0.22 (−0.39– − 0.03)0.02
Q30. When cases are complex, professional teams have more troubles to deal with other patients3.55 (1.17)3.83 (0.91)0.28 (0.05–0.48)0.01
Q31. Fatigue does not impair professional skills1.79 (0.96)1.64 (0.84)−0.15 (−0.33–0.03)0.11
Q32. Advanced trainees are potentially more likely to make mistakes than beginners2.06 (0.97)2.55 (1.34)0.49 (023–0.75)<0.001
Q33. Multidisciplinary meetings help to improve patient care4.19 (1.03)4.54 (0.78)0.35 (0.17–0.52)<0.001
Q34. Human communication is always ambiguous2.83 (1.26)3.66 (1.26)0.83 (0.55–1.09)<0.001
Q35. In emergency situations it is acceptable to help each others, beyond professional boundaries4.34 (1.07)4.58 (0.72)0.24 (0.04–0.44)0.01
Q36. Teams learn from their mistakes4.58 (0.98)4.79 (0.71)0.21 (0.04–0.38)0.01
  • SD, Standard deviation; the learning questionnaire summary scores range from 1 to 5 points, with higher scores indicating improved agreement with the statement tested (except for negatively worded questions Q6, Q7, Q10, Q12, Q15, Q17, Q23, Q24, Q29 and Q31).

  • Summary scores were computed for all respondents based on the un-weighted average of responses to each item.

  • P-values following paired t-test.

Behaviour (Kirkpatrick's dimension 3)

The safety-attitude questionnaire was completed by 94.9% of the participants. After 1 year of implementation of the programme, a positive change in the team and safety climate was measured. This change was observed in items exploring the use of briefing and coordination methods as well as in the implementation of improvement initiatives following the use of incident reports [OR 2.9, 95% CI (1.3–6.3) to OR 4.7, 95% CI (1.2–17.2)]. There was also improved stress recognition [OR 2.4, 95% CI (1.2–4.8) to OR 3.0, 95% CI (1.0–8.8)], working conditions and job satisfaction. Participants reported improved availability of information [OR 2.4, 95% CI (1.0-5.7)] and the feeling of being part of a big family [OR 2.1, 95% CI (1.0-4.4)]. A negative change was observed during the second period in the dimension of perception of management of the questionnaire, in items exploring support and information provision [OR 0.2, 95% CI (0.1–0.6) to OR 0.4, 95% CI (0.1–0.8)]. This change disappeared during the third period. For the purpose of clarity, we report in Table 3, the dimensions of the safety attitude questionnaire related to the teamwork-safety climate and stress recognition. The full item list is available on the web supplement of this article (See online supplementary material).

View this table:
Table 3

Organizational safety culture change over 1 year of program implementation

ItemsSecond period change OR (95% CI)Third period change OR (95% CI)
Teamwork climate
Q1. Nurse input is well received in this clinical area0.71 (0.20–2.46)1.04 (0.25–3.91)
Q2. Decision making in this clinical area utilizes input from relevant personnel1.12 (0.55–2.26)1.80 (0.82–3.95)
Q3. In this clinical area it is difficult to speak up if I perceive a problem with patient care0.70 (0.36–1.37)1.40 (0.67–2.90)
Q4. Disagreement here are resolved appropriately0.81 (0.39–1.67)1.71 (0.72–4.06)
Q5. I have the support I need form other personnel to care for patients0.61 (0.24–1.51)0.73 (0.28–1.94)
Q6. It is easy for personnel in this clinical area to ask questions when there is something that they do not understand0.40 (0.11–1.38)1.15 (0.24–5.38)
Q7. During emergencies, I can predict what other personnel are going to do next0.56 (0.24–1.29)0.71 (0.29–1.75)
Q8. Briefings are common in this clinical area0.89 (0.46–1.74)2.92 (1.35–6.33)
Q9. The physicians and nurses here work together as a well-coordinated team1.10 (0.56–2.15)2.21 (1.00–4.87)
Q10. I know the first and last names of all the personnel I worked with during my last shift1.50 (0.79–2.84)1.17 (0.60–2.28)
Q11. Important issues are well communicated at shift changes1.44 (0.43–4.75)1.90 (0.47–7.68)
Q12. Communication breakdowns which lead to delays in staring surgical procedures are common0.25 (0.02–2.36)0.90 (0.59–5.18)
Safety climate
Q1. I would feel safe being treated here as a patient0.46 (0.11–1.93)2.46 (0.25–24.32)
Q2. Medical errors are handled appropriately in this clinical area0.41 (0.14–1.14)1.21 (0.32–4.49)
Q3 I receive appropriate feedback about my performance0.73 (0.38–1.37)1.15 (0.58–2.28)
Q4. In this clinical area, it is difficult to discuss errors0.85 (0.44–1.67)1.05 (0.51–2.12)
Q5. Briefing other personnel before a procedure is important for patient safety0.90 (0.17–4.63)0
Q6. I am encouraged by my colleagues to report any patient safety concerns I may have0.65 (0.23–1.80)0.94 (0.29–2.95)
Q7. The culture in this clinical area makes it easy to learn from errors of others0.65 (0.31–1.34)1.27 (0.57–2.83)
Q8. I have seen others make errors that had the potential to harm patients1.15 (0.55–2.39)1.17 (0.53–2.54)
Q9. I know the proper channels to direct questions regarding patient safety in this clinical area0.48 (0.20–1.13)0.96 (0.36–2.55)
Q10. I am frequently unable to express disagreement with staff physicians1.27 (0.65–2.47)0.72 (0.36–1.43)
Q11. All the personnel in my clinical area take responsibility for patient safety0.47 (0.13–1.66)1.73 (0.30–9.83)
Q12. Patient safety is constantly reinforced as the priority in this clinical area1.21 (0.58–2.52)2.37 (0.99–5.67)
Q13. There is widespread adherence to clinical guidelines and evidence-based criteria regarding patient safety here0.57 (0.16–2.04)1.79 (0.31–10.13)
Q14. Information obtained through incident reports is used to make patient care safer in this clinical area0.84 (0.38–1.88)4.71 (1.28–17.28)
Q15. Personnel frequently disregards rules or guidelines0.69 (0.35–1.34)1.00 (0.50–2.00)
Stress recognition
Q1. Fatigue impairs my performance during emergency situations3.04 (1.12–8.26)3.04 (1.05–8.83)
Q2. When my workload becomes excessive, my performance is impaired0.98 (0.30–3.20)1.25 (0.33–4.65)
Q3. I am less effective at work when fatigued2.05 (0.49–8.51)1.23 (0.33–4.58)
Q4. I am more likely to make errors in tense or hostile situations1.11 (0.58–2.10)1.05 (0.53–2.08)
Q5. Stress from personal problems adversely affects my performance1.29 (0.68–2.42)2.42 (1.20–4.87)
Q6. Disruptions in the continuity of care can be detrimental to patient safety0.94 (0.48–1.83)0.65 (0.33–1.21)
Q7. Truly professionals personnel can leave personal problems behind when working1.06 (0.45–2.51)2.48 (1.10–5.59)
Q8. I have made errors that had the potential to harm patients0.84 (0.44–1.62)0.72 (0.36–1.44)
  • Overall odds ratio (OR) for each item is presented. The overall OR was calculated by transforming individual item values comprised between 1, strongly disagree and 5, strongly disagree into a dichotomic variable 0, disagree and 1, agree. Final results were obtained by comparing for all participants and each item the second and third tertiles of the year to the initial tertile, used as a reference. They indicate participants' change of agreement (or disagreement) with the item between the two periods compared.

Discussion

To improve interprofessional teamwork and communication, the adaptation of the CRM learning technique from aviation to a multidisciplinary healthcare setting proved to be feasible and effective. Participants were largely satisfied with the training programme developed and most valued the experience. From 63 to 90% of participants rated their level of satisfaction as very high for overall course organization, course content, teaching method and knowledge acquisition. Except for seven items related to formal theoretical concepts of stress, fatigue and uncertainty management, the 36-item questionnaire testing participants' learning demonstrated a significant change towards better understanding of teamwork and shared decision making at the end of the 2-day training programme.

Over 1 year of implementation of the programme, there was a positive change in the team and safety climate in the hospital as well as improved stress recognition.

To improve teamwork and communication in healthcare settings, a number of CRM-based programmes have been developed [1118]. However, only a few studies analysed their effectiveness and most limited their assessment to participants' reactions [11, 12, 14, 18]. In one, authors found that 100% of participants would value further similar training and indicated that the course would make them change their usual practice [14]. In another, it was found that 75–97% participants highly valued the course and rated their satisfaction with learning content at 4 and 5 on a 5-level rating scale [12]. In another, using focus groups, Kyrkjebo et al. [18] found that students were satisfied with the programme, although some of them found that the course content, particularly, videos and simulation exercises, could be more realistic. These results are comparable with our study where we found that 63–90% of participants reported being very satisfied with the overall course organization, content, teaching method and learning. This confirms that CRM-based training programmes are highly valued by participants and that this approach is popular amongst healthcare professionals. Only one programme included an assessment of further levels of the Kirpatrick's evaluation framework. Morey et al. [15] found a statistically significant improvement in the quality of team behaviour, work performance and reduction of errors following the implementation of a CRM-based training programme in emergency units. Except for error reduction (not assessed in our study), these results are similar to our findings. However, a key characteristic of their programme was the use a team-based staffing approach to day and nightshift organization. Physicians, nurses and technicians were systematically allocated to one specific team, which would be listed together. This may have introduced other factors beyond the content of their CRM-based programme, which may explain some of their findings. Our programme was developed to address routine practice communication and coordination issues amongst professionals from different specialties involved in obstetrical care. It was designed to take into account the existence of ‘temporary’ teams of professionals which may work together for 1 day or night only, every week or 2 weeks, a phenomenon amplified by the introduction of working-hours limitations for interns and residents [24, 25].

As a consequence, it emphasized the teaching of portable skills, usable in ‘temporary teams’ and various circumstances [14]. This was done by the organization of different workshops aimed at improving participants' understanding of differences in professional culture and the resulting expectations. This is how, for instance, participants discovered that a significant source of conflict between anaesthetists and midwives was that midwives expected anaesthetists to be quickly available for their patients in the delivery unit while anaesthetists expected midwives to be aware of their other commitments in the hospital.

Another aspect developed in our CRM-based training programme was the organization of workshops to train participants in improving their communication skills. For instance, participants were trained to describe and report as accurately as possible complex clinical problems. These workshops were particularly valued by participants and they led to a significant number of improvement initiatives (165 since the beginning of the programme). Some examples include the implementation of an emergency procedure to call paediatricians, the development of a multidisciplinary critical incidents review committee, and the implementation of communication standards to prepare all the personnel involved for an emergency caesarean section. This was confirmed by the analysis of the questionnaire on organizational safety climate where we found a significant improvement in items related to the use of briefing and coordination methods as well as the implementation of improvement initiatives following improvements in the communication of incident reports.

Effective communication plays a key role in the improvement of patient safety and there is some evidence in the literature that standardization of communication can improve patient outcomes [26]. However, efforts to improve communication should not be limited to the use of checklists and communication codes. The CRM approach emphasizes the use of ‘all available resources by the flight-deck crew’ [8]. This includes the development of systematic briefing sessions, the training of personnel to challenge and monitor each others' actions appropriately, and the implementation of formal education courses on recognized issues related to patient safety (i.e. fatigue, stress, workload) [27]. This latter formal educational aspect was not pre-eminent in our CRM-based training programme, which emphasized the acquisition of knowledge through practical workshops and particularly around communication issues, demonstrated to be a critical dimension of safety in labour and delivery units [28]. This may explain why participants did not demonstrate improved knowledge on formal theoretical aspects of methods to improve patient safety, except for stress recognition.

We also found only marginal change at the end of the training year for the job-satisfaction, and safety climate dimensions of the SAQ-questionnaire. Several reasons may explain this phenomenon. First, at the end of our data-collection process not all staff members of the hospital had been trained by CRM-based techniques. Overall changes observed in the organizational teamwork climate and working conditions mainly reflected the halo effect of improvement initiatives developed by the first groups of participants in the training programme. If the impact of the programme had been assessed later, other changes may have perhaps been observed. Secondly, our programme emphasized the development of teamwork skills. As a consequence it is likely that other aspects such as job satisfaction may not have been affected. Interestingly, the perception of the support provided by management got worse during the second trimester, while becoming non-significant by the end of the training year. This may be related to increased activity in the labour unit (10% by the end of year 2004) and the negative impact of the summer period, when many staff members are on holiday and the significant increase in the workload for those who remain.

This study has several limitations. First, it was undertaken in a single hospital and analysed a specific CRM-based training programme. Its generalization to other CRM-based programmes in other multidisciplinary settings may be limited. Furthermore, the participants in the programme we assessed were not randomly chosen. Results may reflect a possible selection bias of participants more inclined to participate, enjoy and learn from such a training programme. However, participation in the programme was compulsory for all staff members of the hospital working in the labour and delivery unit. Only if the workload of a participant's department was too high was he allowed to cancel his participation in a seminar and was automatically registered for another one.

Secondly, the questionnaire assessing learning is not a validated measurement tool. Results may reflect other aspects than participant's level of learning (i.e. questionnaire reliability issues). However, due to the specific characteristics of our programme there were no validated tools available to assess its content explicitly. Knowledge assessment by testing participants' acquisition of pre-defined concepts is a commonly accepted testing method and is largely used by schools and universities. Furthermore, the presence of a large number of significant changes on items testing specifically the course content suggests that changes of scores observed are more likely to be due to participants' learning than to reliability issues of the questionnaire.

Thirdly, we attributed improvement to the teamwork climate in the hospital to our CRM-based training programme. Other factors may have confounded the analysis. These may include differences in staff profiles, departmental activities, seasonal profiles and resource allocation between the two periods of assessment of the organizational safety climate. Furthermore, to measure the third dimension of Kirkpatrick's model, we used a surrogate marker of behavioural change, the modification of staff safety attitude following the training programme. This may only assess a fraction of what may represent a true change of staff behaviour. Additional approaches, such as video recording or direct observation, could have been used to assess participants' use of CRM skills in clinical settings. Finally, if it could be demonstrated that participants to a CRM-based training programme are satisfied, learn new skills and use them in their everyday practice, resulting in improvements of the organizational teamwork climate, we were unable to demonstrate that fewer human errors and improved patient safety resulted from the CRM approach (fourth dimension of Kirkpatrick's evaluation framework) [22].

Furthermore behavioural change would probably have been better assessed. Further research could look for instance at the development of safety-measurement tools able to identify, in medical charts or administrative databases, iatrogenic injuries and errors and to compare their incidence before and after implementation of the CRM-based programme.

Despite these limitations, the findings of this study indicate that the implementation of a CRM-based training programme in a multidisciplinary environment is an effective way to improve interprofessional teamwork and communication. Professionals highly value the CRM-based approach and significantly improve their knowledge of teamwork and shared decision-making. This improves the organizational teamwork and safety climates and most likely overall patient safety. CRM-based training programmes should be strongly promoted not only for the management of crisis scenarios but also for routine practice in multidisciplinary environments where a high level of coordination and collaboration is required beyond the boundaries of professional cultures.

Acknowledgements

The authors would like to thank Dr Laure Kloetzer (cognitive psychologist) and Mr Jean Pariès (aviation safety engineer) – Dédale/Paris® who advised on and guided the development of this programme. We also would like to thank Mrs M. Ariès for her help in the data-collection process, Mr Alec Hester for his editorial advices and all staff members of the three departments involved for their enthusiastic contribution.

References

View Abstract