International Journal for Quality in Health Care Advance Access originally published online on March 4, 2008
International Journal for Quality in Health Care 2008 20(3):211-220; doi:10.1093/intqhc/mzn003
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Development of a patient safety climate scale in Japan
1 Department of Health Services Management and Policy, Kyushu University Graduate School of Medicine, Fukuoka, Japan
2 Department of Health Care Administration and Management, Kyushu University Graduate School of Medicine, Fukuoka, Japan
Address reprint requests to: Akihito Hagihara, Department of Health Care Administration and Management, Kyushu University Graduate School of Medicine, 3-1-1 Maidashi, Higashi-ku, Fukuoka 812-8582, Japan. Tel: +81-092-642-6950; Fax: +81-092-642-6961; E-mail: hagihara{at}hsmp.med.kyushu-u.ac.jp and s-matsubara{at}umin.ac.jp
| Abstract |
|---|
|
|
|---|
Background. Although patient safety climate is an important factor in preventing adverse events in health care organizations, there is no usable Japanese scale.
Objective. To develop a Japanese scale to measure patient safety climate and to evaluate its psychometric properties.
Methods. Cross-sectional questionnaire survey was conducted with respect to 9 non-academic general hospitals in Japan. A total of 1878 health care professionals (nurses, therapists, technicians, pharmacists and physicians) were included in a study.
Results. The eight dimensions measuring worker attitudes (free communication flow, continuous improvement, reporting/rules compliance and patient/family involvement) and organizational factors (supervisors' safety leadership, allied professionals' safety leadership, patient safety committee leadership and rules/equipment availability) were extracted by factor analysis. The internal consistency (measured by Cronbach's alpha) and repeatability (measured by intraclass correlation) were more than 0.70 for all subscales. In addition, the mean score, the within-group interrater reliability statistic (rwg) and the correlation coefficients of the mean score were confirmed at the workplace level.
Conclusions. The scale showed acceptable dimensionality, reliability and validity. It also provided workplace team-evaluation and a tool for assessing the patient safety climate at the level of the workplace.
Keywords: health care organization, medical errors, patient safety, questionnaire, safety climate
| Introduction |
|---|
|
|
|---|
It is widely recognized that the safety climate (culture) is an essential factor influencing patient safety [1, 2]. The safety climate is conceptualized as workers' shared perceptions of how safety management is implemented in their workplace [3, 4]. Studies utilizing various safety climate scales have been reported in various industries [5]. In the health care field, the safety climate survey has become a common approach to monitoring and improving patient safety, and different types of questionnaire have been developed [6, 7].
After reviewing the existing questionnaires, we had several major concerns. First, most existing safety climate assessment tools lack a dimension indicating the major factor in health care, namely patients [6, 7], perhaps because many questionnaires have been derived from other industries, such as commercial aviation and the armed forces [8–15]. The Institute of Medicine recommends that patients be part of the care process [1], and a factor regarding patient involvement should be included in a patient safety climate questionnaire. Second, as Zohar described, if the safety climate is a property related to organization-level or group-level properties, individual-level variables such as beliefs or attributions should not be included in measurement instruments [4]. However, several existing questionnaires include items referring to personal beliefs (e.g. personal belief in the cause of adverse events) [14] and attributions (e.g. personal ability to provide high-quality patient care) [11, 15]. To improve patient safety, surveys that utilize a peer- or team-evaluation tool will provide more valuable information than a self-evaluation tool. Finally, most existing questionnaires have been developed in other countries and are in English, and it seems inappropriate to adapt them to the needs of Japanese society due to differences in culture and health care systems. Thus, we developed a new patient safety climate scale specifically for Japan.
Our objective was to develop a Japanese-language patient safety climate scale to be used by health care staff in the Japanese health care setting and to evaluate its reliability and validity.
| Methods |
|---|
|
|
|---|
Conceptual and theoretical framework
We adopted the recommendations of the Institute of Medicine [1] and the theory of organizational accidents by Reason [16]. The main framework of the scale is based on the following two assumptions: the incidence of medical errors is influenced by safe (or unsafe) attitudes of workers and organizational factors, and the leadership of frontline workers, organizational factors, and patients and their families are essential to a patient safety climate. Based on this framework, the components of patient safety climate were divided into worker attitudes and organizational factors. Worker attitudes were further divided into five categories: communication, medical-error reporting, compliance with rules and procedures, improving the work environment and including patients and their families in the process of care. Organizational factors were further divided into six categories: supervisors; allied professionals (especially focused on physicians); the patient safety committee; rules, procedures, and manuals; equipment; and hospital management (education, educational goals, policy and investment). Keeping these categories in mind, we developed questionnaire items in the next phase.
Questionnaire development
First, we conducted an open-ended survey to identify the patient safety concerns of health care workers. We distributed the anonymous self-completed questionnaire asking participants to indicate your concerns about the safety climate or medical accidents in the health care setting to 3750 health care workers (nurses, pharmacists and physicians) in 20 hospitals. In total, 259 (6.9%) completed questionnaires were returned. The list of answers was developed based on the above framework and used to develop and pool items.
Second, the items were retrieved and pooled. A Medline search was conducted to identify surveys used to measure the safety climate (culture) in health care organizations; articles were selected, references searched, and websites visited. We found six patient safety climate instruments included in the nine surveys reviewed by Colla [6, 9–15, 17, 18] and nine instruments included in the 12 studies reviewed by Flin [7, 13–15, 18–24]. Items from safety climate scales used in other industries were also reviewed and selected [25–28], and the items that we had previously developed were also pooled for the study [29].
Third, we translated the pooled items into Japanese and modified the wording of them, referring to the answers in the open-ended survey. The items were accompanied by a 5-point response scale (1 = disagree, 3 = neither agree nor disagree, 5 = agree). The items that had a negative connotation in terms of patient safety had their ratings reversed before calculating the score. We revised the statements so that the items evaluate how workers assess the safety conditions of their workplace, based upon the above framework. Content validity of the items was confirmed by experts in related fields (i.e. health care risk management specialists, psychologists, physicians and nurses who worked as patient safety managers at an academic hospital). A pilot study (n = 35) was performed to test the face validity of the items. Thirty-five health care professionals from three hospitals were asked to respond to the questionnaire inquiry and comment on the wording and relevance of the items.
The final questionnaire contained 50 items about patient safety climate and the following demographic data: age, gender, job type, years of practice in the current job, years of employment in the current hospital and job class (supervisory/non-supervisory).
Setting
We sent study protocols to 220 non-academic general hospitals with more than 100 beds in Fukuoka Prefecture, Japan. Forty-two hospitals initially agreed to participate in the survey. After detailed explanations about the study, nine of the 42 hospitals consented to participate in the study. Eight of these employed more than 200 workers, seven had more than 200 beds, six provided emergency care service, seven were accredited by the Japan Council for Quality Health Care, and six were postgraduate education hospitals.
Respondents
The subjects of the study were nurses (registered nurses, practical nurses, registered midwives and public health nurses), therapists (physical therapists, occupational therapists and speech therapists), technicians (radiological technicians, medical technologists and clinical engineering technologists), pharmacists and physicians. Part-time workers were excluded from the sample. Heads of each professional department were excluded from the survey because items asking about supervisor attitudes were included in the questionnaire.
After obtaining consent from all departmental representatives, managers at each workplace distributed the anonymous self-completed questionnaires to the study participants. Survey collection boxes were placed at workplaces, in wards (general inpatient wards and intensive care units), surgical units, ambulatory care units, rehabilitation units, laboratories, pharmacies and medical offices. The participants were asked to return the completed questionnaires within 2 weeks.
Eighty volunteers from five professions in two hospitals participated in the test–retest study to evaluate the repeatability of the questionnaire. Two weeks after initially completing the questionnaire, they received the same questionnaire to take again. They were asked to complete it within 2 weeks.
Period
The survey was conducted between October and December 2005.
Research ethics
A cover letter sent with the questionnaire explained the purpose of the study, emphasizing that it was voluntary and that respondents would remain anonymous. The letter also noted that by completing the survey, the volunteer had consented to participate in the study. In the test–retest study, a number known only to the respondent was used to match the first and second responses.
Statistical analysis
The construct validity of the scale was assessed using factor analysis, excluding items for which the skewness of distribution was greater than +1.0 or less than –1.0, or samples with one or more missing values. The Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett's test of sphericity were used to determine whether the data were appropriate for factor analysis. In the factor analysis, the unweighted least squares method was used. We used promax rotation to detect the underlying dimensions of worker attitudes because we hypothesized that the dimensions would be psychologically related. In contrast, we used varimax rotation to detect the underlying dimensions of organizational factors because we hypothesized that each factor would be psychologically independent. Items with a factor-loading score of less than 0.40 were eliminated. We also excluded items with loadings of more than 0.40 across multiple dimensions. Dimensions with only one or two items were eliminated. The dimensions extracted by the analysis were checked to see if they differed from our conceptual and theoretical framework.
Cronbach's alpha coefficient was used to test the internal consistency of each dimension. To evaluate the repeatability of the scale in the test–retest method, we calculated the intraclass correlation coefficient and 95% confidence intervals.
We examined the psychometric properties of the scale at the level of the workplace. Workplaces with a complete response rate of more than 60.0% were investigated according to the recommendations of Pronovost and Sexton [30]. We calculated the mean score and the within-group interrater reliability statistic (rwg) for each workplace, and calculated the mean and standard deviation of their values for each subscale. The subscale scores were calculated by summing the item scores without weighting. The within-group interrater reliability statistic expresses the degree of within-group agreement, namely aggregation [31]. The statistics were calculated using the observed variance of the items and a uniform distribution that reflected total absence of agreement. In addition, Pearson's correlation coefficients of the mean score were examined between extracted dimensions.
Analyses were performed using SPSS 11.0 for Windows (SPSS, Chicago, IL, USA).
| Results |
|---|
|
|
|---|
Sample characteristics
Questionnaires were distributed to 2453 participants (1619 nurses, 161 therapists, 186 technicians, 73 pharmacists and 414 physicians) in 132 workplaces (including 52 wards) at nine hospitals. Of these, 1878 (76.6%) were returned (Table 1). Workplace response rates ranged from 20 to 100%.
|
Repeatability was evaluated based on data from 80 respondents who had completed the questionnaire twice.
Development of the patient safety climate scale
One item for which the skewness of distribution was greater than +1.0 or less than –1.0 was excluded. Eighty-two samples (4.4% of responses) with one or more missing values were also excluded. A sample of 1796 (73.2%) questionnaires was used in the next analysis.
The Kaiser–Meyer–Olkin measure of sampling adequacy was more than 0.90, and Bartlett's test of sphericity was significant at P < 0.001, indicating that the data were appropriate for factor analysis.
After a factor analysis of 30 items concerning worker attitudes, four dimensions with 17 items were extracted: free communication flow (five items), continuous improvement (five items), reporting/rules compliance (four items) and patient/family involvement (three items). From the 20 items concerning organizational factors, a factor analysis extracted four dimensions with 16 items: supervisors' safety leadership (four items), allied professionals' safety leadership (four items), patient safety committee leadership (four items) and rules/equipment availability (four items). These dimensions explained 64.3% of the total variance. Finally, we developed a Japanese-language patient safety climate scale containing 33 items with eight subscales. The percentage of missing responses per item ranged from 0.1 to 0.4%. The ceiling and floor effects per item were 2.0–34.8% and 0.9–12.8%, respectively. Each subscale was intercorrelated at between 0.31 and 0.62. These results and the mean scores for each subscale are shown in Tables 2 and 3, respectively.
|
|
Both the Cronbach's alpha coefficients and the intraclass correlations of the test–retest survey were more than 0.70 for all subscales (Table 3).
Of 132 workplaces, 98 (74.2%) showed a complete response rate of more than 60.0%: 65 workplaces with nurses, 6 workplaces with therapists, 14 workplaces with technicians, 9 pharmacies and 4 medical offices. Mean scores, the within-group interrater reliability statistics and the correlation matrix of the mean scores at the workplace level are shown in Table 4. Regarding the within-group interrater reliability statistic, the means were greater than 0.70 for three subscales: free communication flow, continuous improvement and patient safety committee leadership; however, the means were less than 0.60 for two subscales: patient/family involvement and supervisors' safety leadership.
|
| Discussion |
|---|
|
|
|---|
We developed a scale that measures the patient safety climate with good psychometric properties. The potential items were pooled through literature reviews, a questionnaire on patient safety concerns, and expert opinion, which supported the content validity of the scale. In particular, the anonymous open-ended questionnaire provided us with honest opinions and straightforward critiques of workplaces and organizations. Regarding the wording and content of items, our scale seems to be suitable as a workplace team-evaluation tool, i.e. a peer-evaluation tool rather than a self-evaluation tool.
Factor analysis revealed eight dimensions reasonable for measuring safety climate in health care workplaces (Table 2). The scale contained almost all of the dimensions that we initially hypothesized. The scale contained the essential dimensions reported in previous studies [6, 7], partially supporting the construct validity of our tool. In addition, our scale included a factor concerning the involvement of patients and their families in the care process (i.e. patient/family involvement). This finding is in line with the theoretical expectation that a safety climate assessment tool should include a dimension focused on patients. However, it is still unclear whether the involvement of patients and their families in the care process is an indispensable factor for establishing a patient safety climate in health care settings. In addition, the items concerning hospital management were eliminated from the scale by factor analysis, possibly because they may have expressed patient safety management at the organizational rather than the workplace level (Appendix 2). Further study is necessary to determine a suitable dimension for measuring the patient safety climate.
The scale also contains the issue concerning the difference between quality of care and quality of safety. The nature of Q27 (Workers support patients and family participation in the care process) and Q25 (Patients and workers fully discuss their patient care) may express the generic quality of care rather than the quality of patient safety. However, it is reasonable to assume that the quality of safety in the health care field is positively and closely related to the quality of care. In other words, when patients and their families participate in the care process and in discussion of patient care, fewer medical errors will occur. Thus, we adopted these items. As a topic for future study, this assumption needs to be verified.
Each dimension showed acceptable internal consistency and repeatability (Table 3). Compared to other questionnaires, the present findings are satisfactory [6, 7, 32]. The high participation rate and low proportion of missing values are indicative of the validity of our scale. All items showed negligible ceiling or floor effects, demonstrating that the scale is an acceptable survey tool for determining the patient safety climate in Japan (Table 2).
Zohar asserted that the level of the safety climate refers to the extent to which patient safety is prioritized in individual units (i.e. workplaces), and the strength of the safety climate indicates the extent of agreement among employees regarding these priorities [33]. Zohar also suggested that the level and strength were expressed by the mean score and variance of a safety climate scale for each unit, respectively. Based on this theory, we examined the mean score and the within-group interrater reliability statistic (rwg) calculated from the variances for each workplace (Table 4). The within-group interrater reliability statistic is one of the indicators used to estimate agreement within a group [31]. In our study, the statistics were low (below 0.70) on some subscales, especially patient/family involvement. This result may suggest the difficulty of assessing the participation of patients and their families in the care process. Thus, researchers need to analyse the data taking into account differences in aggregation for each workplace when a workplace-level survey is conducted using this scale.
This study has several limitations. First, as reported in previous safety climate studies [11, 15, 34], the response rate of physicians was low in our study, possibly because it may be difficult for physicians, especially frontline physicians such as residents, to answer several questions aimed at evaluating their supervisors and organizations. Before the patient safety climate survey, researchers or administrators need to emphasize the importance of the survey to the participants of the study. Second, our data cannot be easily generalized, i.e. our results may not be applicable to other job types in the health care industry (e.g. independent practice clinics, teaching hospitals) or health care facilities in other countries. Future studies should survey more and various multidisciplinary health care organizations. Third, we did not examine the criterion-related validity of our scale by comparing it with existing questionnaires. This should be done in a future study.
In conclusion, we developed a patient safety climate assessment scale with acceptable dimensionality, reliability and validity. The scale is adequate for workplace-level surveys, and the questionnaire is thus applicable to health care organizations in Japan. We hope that our patient safety climate survey will be widely implemented in Japanese hospital settings as the first step of a comprehensive patient safety program.
| Appendix 1 |
|---|
|
|
|---|
Item wordings of the patient safety climate scale
|
The superscript for each item is the number of references. Items followed by NR are original.
All items are the reversal of the scores.
If the respondent is a nurse, therapist, technician or pharmacist, the subjects of the questions are Physicians. If the respondent is a physician, the subjects are Other professionals.
Items in Japanese are translated into English.
Appendix 2
Wordings of the items eliminated after factor analysis
|
The item is the reversal of the scores.Items in Japanese are translated into English.
| Acknowledgements |
|---|
|
|
|---|
The authors thank the participating hospitals and respondents of the survey. We also thank Junko Ayuzawa for the development of the questionnaire items. This research was supported by a research grant provided by the Health Care Science Institute, Japan. The authors declare that they have no competing interests.
| References |
|---|
|
|
|---|
- Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system, Institute of Medicine Report (2000) Washington, DC: National Academy Press.
- Department of Health. An organisation with a memory: report of an expert group on learning from adverse events in the NHS (2000) London: The Stationery Office.
- Zohar D. Safety climate in industrial organizations: Theoretical and applied implications. J Appl Psychol (1980) 65:96–102.[CrossRef][Web of Science][Medline]
- Zohar D. Safety climate: conceptual and measurement issues. In: In Handbook of Occupational Health Psychology (2003) Washington, DC: American Psychological Association.
- Flin R, Mearns K, O'Connor P, et al. Measuring safety climate: identifying the common features. Safety Science (2000) 34:177–92.[CrossRef][Web of Science]
- Colla JB, Bracken AC, Kinney LM, et al. Measuring patient safety climate: a review of surveys. Qual Saf Health Care (2005) 14:364–6.
[Abstract/Free Full Text] - Flin R, Burns C, Means K, et al. Measuring safety climate in health care. Qual Saf Health Care (2006) 15:109–15.
[Abstract/Free Full Text] - Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res (2006) 6:44. http://www.biomedcentral.com/1472-6963/6/44. [serial online].[CrossRef][Medline]
- Sexton JB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: cross sectional surveys. BMJ (2000) 320:745–9.
[Abstract/Free Full Text] - Flin R, Fletcher G, McGeorge P, et al. Anaesthetists' attitudes to teamwork and safety. Anaesthesia (2003) 58:233–42.[CrossRef][Web of Science][Medline]
- Gaba DM, Singer SJ, Sinaiko AD, et al. Differences in safety climate between hospital personnel and naval aviators. Human Factors (2003) 45:173–85.[Web of Science][Medline]
- Itoh K, Abe T, Andersen HB, et al. A questionnaire-based survey of safety culture in Japanese hospitals. (2003) Proceedings of the 15th Triennial congress of the International Ergonomics Association: Seoul, Korea.
- Itoh K, Abe T, Andersen HB. A survey of safety culture in hospitals including staff attitudes about incident reporting. (2002) Proceedings of the Workshop on the Investigation and Reporting of Incidents and Accidents: Glasgow, UK.
- Pronovost PJ, Weast B, Holzmueller CG, et al. Evaluation of the culture of safety: survey of clinicians and managers in an academic medical center. Qual Saf Health Care (2003) 12:405–10.
[Abstract/Free Full Text] - Singer SJ, Gaba DM, Geppert JJ, et al. The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care (2003) 12:112–18.
[Abstract/Free Full Text] - Reason J. Managing the risks of organizational accidents (1997) Aldershot: Ashgate Publishing Limited.
- Weingart SN, Farbstein K, Davis RB, et al. Using a multihospital survey to examine the safety culture. Jt Comm J Qual Saf (2004) 30:125–32.[Medline]
- Sorra JS, Nieva VF. Hospital survey on patient safety culture (2004) Rockville, MD: Agency for Health care Research and Quality Publication.
- Gershon RRM, Karkashian CD, Grosch JW, et al. Hospital safety climate and its relationship with safe work practices and workplace exposure incidents. Am J Infect Control (2000) 28:211–21.[CrossRef][Web of Science][Medline]
- Neal A, Griffin MA, Hart PM. The impact of organizational climate on safety climate and individual behavior. Safety Science (2000) 34:99–109.[CrossRef][Web of Science]
- McCoy KD, Beekmann SE, Ferguson KJ, et al. Monitoring adherence to standard precautions. Am J Infect Control (2001) 29:24–31.[CrossRef][Web of Science][Medline]
- DeJoy DM, Murphy LR, Gershon RM. The influence of employee, job/task, and organizational factors on adherence to universal precautions among nurses. Int J Ind Ergon (1995) 16:43–55.[CrossRef][Web of Science]
- Vredenburgh AG. Organizational safety: which management practices are most effective in reducing employee injury rates? J Saf Res (2002) 33:259–76.[CrossRef]
- Guastello SJ, Gershon RRM, Murphy LR. Catastrophe model for the exposure to blood-borne pathogens and other accidents in health care settings. Accident Anal Prevent (1999) 31:739–49.[CrossRef]
- Glendon AI, Litherland DK. Safety climate factors, group differences and safety behaviour in road construction. Safety Science (2001) 39:157–88.[CrossRef][Web of Science]
- Davies F, Spencer R, Dooley K. Summary guide to safety climate tools (2001) Sudbury: Health and Safety Executive books.
- Zohar D. A group-level model of safety climate: testing the effect of group climate on microaccidents in manufacturing jobs. J Appl Psychol (2000) 85:587–96.[CrossRef][Web of Science][Medline]
- Grote G, Künzler C. Diagnosis of safety culture in safety management audits. Safety Science (2000) 34:131–50.[CrossRef][Web of Science]
- Matsubara S, Ayuzawa J, Hagihara A. Development of safety climate scale for medical facilities: nurses' attributes related to their perceptions of and attitudes toward safety. Anzen Igaku (2004) 1:78–88. (in Japanese).
- Pronovost PJ, Sexton JB. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care (2005) 14:231–3.
[Free Full Text] - James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias. J Appl Psychol (1984) 69:85–98.[CrossRef][Web of Science]
- Kho ME, Carbone JM, Lucas J, et al. Safety climate survey: reliability of results from a multicenter ICU survey. Qual Saf Health Care (2005) 14:273–8.
[Abstract/Free Full Text] - Zohar D, Livne Y, Tenne-Gazit O, et al. Healthcare climate: a framework for measuring and improving patient safety. Crit Care Med (2007) 35:1312–17.[CrossRef][Web of Science][Medline]
- Singer S, Mark Meterko M, Baker L. Workforce perceptions of hospital safety culture: development and validation of the patient safety climate in healthcare organizations survey. Health Serv Res (2007) 42:1999–2021.[CrossRef][Web of Science][Medline]
![]()
CiteULike
Connotea
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||