OUP user menu

Lean interventions in healthcare—do they actually work? A systematic literature review

(CC)
John Moraros, Mark Lemstra, Chijioke Nwankwo
DOI: http://dx.doi.org/10.1093/intqhc/mzv123 mzv123 First published online: 24 January 2016

Abstract

Purpose Lean is a widely used quality improvement methodology initially developed and used in the automotive and manufacturing industries but recently expanded to the healthcare sector. This systematic literature review seeks to independently assess the effect of Lean or Lean interventions on worker and patient satisfaction, health and process outcomes, and financial costs.

Data sources We conducted a systematic literature review of Medline, PubMed, Cochrane Library, CINAHL, Web of Science, ABI/Inform, ERIC, EMBASE and SCOPUS.

Study selection Peer reviewed articles were included if they examined a Lean intervention and included quantitative data. Methodological quality was assessed using validated critical appraisal checklists. Publically available data collected by the Saskatchewan Health Quality Council and the Saskatchewan Union of Nurses were also analysed and reported separately.

Data extraction Data on design, methods, interventions and key outcomes were extracted and collated.

Results of data synthesis Our electronic search identified 22 articles that passed methodological quality review. Among the accepted studies, 4 were exclusively concerned with health outcomes, 3 included both health and process outcomes and 15 included process outcomes. Our study found that Lean interventions have: (i) no statistically significant association with patient satisfaction and health outcomes; (ii) a negative association with financial costs and worker satisfaction and (iii) potential, yet inconsistent, benefits on process outcomes like patient flow and safety.

Conclusion While some may strongly believe that Lean interventions lead to quality improvements in healthcare, the evidence to date simply does not support this claim. More rigorous, higher quality and better conducted scientific research is required to definitively ascertain the impact and effectiveness of Lean in healthcare settings.

  • Lean
  • Lean thinking
  • Lean interventions
  • quality improvement
  • healthcare

Introduction

Globally, healthcare systems are at a cross roads. Many political and healthcare leaders, and in fact the public itself is calling for, if not demanding, the redesign of healthcare delivery. The concern is fuelled by ever increasing costs and high expectations, while at the same time having surprisingly low rates of patient adherence to care and high rates of adverse events [1]. In response, many jurisdictions have attempted to introduce standardized protocols like Lean.

Lean is a widely used quality improvement methodology. Lean thinking was first developed in the automotive and manufacturing industries but it has recently expanded to the healthcare sector. Lean thinking begins with identifying and ‘removing waste’ in order to ‘add value’ to the customer or patient [2]. The Lean Enterprise Institute articulates five main principles of Lean: specify value from the standpoint of the customer, identify all the steps in the value stream and eliminate steps that do not create value, make the steps flow smoothly toward the customer, let customers pull value from the next upstream activity and begin the process again until a state of perfection is reached [3].

The introduction of these principles placed ‘customer value’ and ‘removing waste’ at the centre of Lean thinking. In this manner, the process is essentially driven by ‘what customers want’ and then organizational steps are taken to define which activities are considered to be ‘value-adding’ as opposed to ‘non-value adding’. ‘Value adding’ activities are encouraged because they directly contribute to creating a product or service a customer wants. On the other hand, ‘non-value adding’ activities are considered a waste and need to be removed or avoided [4].

To date, there have been a limited number of reviews of Lean or Lean interventions in healthcare. One of the reviews started with 207 articles under consideration. However, when the authors applied their inclusion criteria of only accepting papers that were published in peer review journals and studies that had quantifiable data available, it left them with merely 19 papers (9.2%) for critical appraisal [5].

Among the papers accepted, it was noted that the vast majority of studies had methodological limitations that undermined the validity of the results. These limitations included weak study designs, lack of statistical analysis, inappropriate statistical assumptions, inappropriate analysis, failure to rule out alternative hypotheses, no adjustment for confounding, selection bias and lack of control groups. The studies also did not review long-term organizational change, long-term impact or the independent effect of Lean while controlling for other organizational or staffing changes occurring at the same time [5]. Although this review was well-conducted, it was not a systematic literature review and it did not include a quality control checklist.

In North America, there are many examples of Lean healthcare interventions but the largest Lean transformation in the world was attempted in the province of Saskatchewan, Canada [6]. The Health Quality Council (HQC) of Saskatchewan concludes on its website that Lean increases patient safety by eliminating errors, increases patient satisfaction, reduces cost and improves patient health outcomes [7].

On the surface, Lean thinking seems to be an approach that generates positive results [8]. Yet, its application in healthcare has been controversial and its effectiveness questioned. As such, the purpose of this systematic literature review is to independently assess the effect of Lean thinking and Lean interventions on worker and patient satisfaction, health and process outcomes and financial costs.

Methods

We conducted an extensive systematic literature review on the following electronic databases: Medline, PubMed, Cochrane Library, CINAHL, Web of Science, ABI/Inform, ERIC, EMBASE and SCOPUS.

Searches were carried out using the following keywords: Lean Production System, Lean enterprise, Lean manufacturing, Virginia Mason Production System, Toyota Production System, Just in time production, Kaizen, HoshinKanri, Lean method, Lean thinking, Lean intervention, Lean healthcare, Lean principles, Lean process, Muda and Healthcare.

Peer-reviewed articles

Articles had to satisfy the following inclusion criteria to be considered: published in English, publicly available, peer reviewed, examined a Lean intervention and included quantitative data. These liberal criteria allowed the inclusion of a wide variety of relevant articles in our study. However, it also served as a means to exclude news reports, blog commentary, informational/promotional pieces and general ‘feel good’ success stories that lacked the necessary quantitative data to be able to critically judge the information presented.

The identification and approval of studies was carried out in three steps. First, the authors examined titles and abstracts to remove duplicates. Second, two of the authors (C.N. and M.L.) reviewed the full-text articles for relevance with regard to the field of healthcare and conformity to the inclusion criteria. Third, methodological quality was assessed by using validated critical appraisal checklists. The diffusion of innovations in health service checklists helped the authors assess the baseline comparability of the groups in each study, the research design, outcome measures and potential sources of bias. They were originally modelled after the Cochrane Effective Practice and Organization of Care Group for interventions in service delivery and organization [9]. Studies that scored >50% on the quality checklist were accepted (i.e. satisfied 6 or more out of 11 questions for before and after studies). Any disagreement between the two authors (C.N. and M.L.) was resolved by additional review and, if required, with a tie-breaking vote by the third author (J.M).

Grey literature

As mentioned, the largest Lean healthcare transformation in the world was attempted in the province of Saskatchewan, Canada [6]. The HQC has been surveying tens of thousands of patients over the years about their experiences in Saskatchewan hospitals. For the purposes of this systematic review, February 2012 was used as the cut-off point for the evaluation of pre- and post-Lean data as it coincided with the official date of the signed provincial contract with a Lean consultant firm [10]. A 26-month period was used to collect and analyse data on a monthly basis before Lean implementation (December 2009 to January 2012) and after Lean implementation (February 2012 to March 2014). This high quality data collected by certified Lean professionals have sample sizes ranging from 17 698 to 92 127 patients with a response rate of ∼51% and it is publicly available on a web site [11]. Additionally, the largest healthcare union or association in the province, the Saskatchewan Union of Nurses (SUN), contracted an external professional polling company to randomly survey 1500 nurses about their Lean experience in 2014 [12]. All 1500 nurses contacted, participated in the telephone survey.

Results

We identified a total of 1056 peer-reviewed articles of which 164 were removed as duplicates, 768 were removed due to lack of relevance to healthcare and 76 were removed because they did not meet the inclusion criteria. Among the 48 articles that were assessed for methodological quality, 22 articles passed [1334] and 26 articles failed the checklist review [3560] (Fig. 1 and Table 1). The original two reviewers (C.N. and M.L.) independently assessed and agreed on 43 studies with a tie breaking vote required by the third reviewer (J.M.) on five out of the 48 studies. Once finalized, the data from the included studies was pooled and summarized and confidence intervals for rate ratios were calculated with an established software application (SPSS 22.0).

View this table:
Table 1

Detailed list of eligible peer review articles from the literature search

Articles that passed methodology review
First author's last name, year of publication, country where study was doneStudy designNumber of participantsLocation of intervention (ex. Emergency department)InterventionIntervention goalType of outcomeQuality scoresOutcome rate ratio and 95% CI
Health outcome studies
Jha, 2012, USA [13]Retrospective cohort6 000 000HospitalPay for performanceReduce 30 day mortality rateHealth outcome9/11 Pass30 day mortality rate
0.08 (−0.30 to 0.46)
McCulloch, 2010, UK [14]Interrupted time series2083Emergency surgery wardPDCAReduced risk of care related harmHealth outcome6/11 PassAdverse events
0.91 (0.72–1.16)
Muder, 2008, USA [15]Pre-/post-test215ICU and a surgical unitHand hygiene, contact precautions, active surveillance (TPS)Reduce incidence of MRSAHealth outcome7/11 PassMRSA infections per 1000 patient days
2.47 (1.87–3.27)
Ellingson, 2011, USA [16]Pre-/post- test109Veteran affairs hospital surgical wardSystems and behaviour change to increase adherence to infection control precautionsReduce in MRSA incidence ratesHealth outcome7/11 PassMRSA incidence rate ratio
0.99 (0.98–1.01)
Process outcome studies
Murrell, 2011, USA [17]Pre-/post-test64 907Emergency departmentRapid triage and treatmentED length of stay and physician wait timeProcess outcome7/11 PassUnable to compute RR
Length of stay reduced from 4.2 (4.2–4.3) to 3.6 (3.6–3.7) hours
Physician start time reduced from 62.2 (61.5–63.0) to 41.9 (41.5–42.4) minutes
Kelly, 2007, Australia [18]Pre-/post-test63 085Emergency departmentStreaming of patients from triage, reallocation of medical and nursing staff (VSM)Reduce number of patients who leave without being seenProcess outcome8/11 PassLeft without being seen
0.99 (0.92–1.08)
Naik, 2012, USA [19]Pre-/post-test22,527Emergency departmentIdentify and eliminate areas of wasteEmergency wait timeProcess outcome6/11 PassUnable to compute RR
Wait time reduced from 4.6 (4.5–4.9) to 4.0 (3.7–4.1) hours
Simons F, 2014, Netherlands [20]Pre-/post-test8,009Operating room of University medical centreDMAIC using A3 interventionDoor movements in the operating roomProcess outcome6/11 PassUnable to compute RR
Door movements reduced by 78% from an average of between 15 and 20 times per hour during surgery to 4 times per hour
Burkitt, 2009, USA [21]Retrospective pre-/post2,550Veteran affairs surgical centerStaff training on hand hygiene, systematic culturing of all admissions, patient isolationIncrease appropriateness of perioperative antibiotics and reduction in length of stayProcess outcomes7/11 PassLength of stay
0.91 (0.76–1.08)
Weaver, 2013, USA [22]Pre-/post-test2444Mental health clinicIdentify and eliminate areas of waste (TPS)Improving number who attend first appointment, reduce wait for appointmentProcess outcome9/11 PassNumber who attended first appointment
1.0 (1.0–1.0)
Wait reduced from 11 days to 8 days
LaGanga, 2011, USA [23]Pre-/post-test1726Mental health centerRemove over bookingIncrease capacity to admit new patients and reduce no-showsProcess outcome7/11 PassNo shows
1.13 (1.03–1.23)
van Vliet, 2010, Netherlands [24]Pre-/post-test1207Eye hospitalIdentify and eliminate areas of wasteReduce patient visitsProcess outcome9/11 PassPatient visits
1.84 (1.33–2.56)
Martin, 2013, UK [25]Pre-/post-test500Radiology departmentValue stream analysis (VSM)Reduce patient journey timeProcess outcome6/11 PassUnable to compute.
No pre and post raw data—only percentage changes were given
White, 2014, Ireland [26]Cross-sectional study338HospitalImplementation of productive ward programImprove work engagementProcess outcome7/11 PassOverall work engagement score1.06 (0.96–1.18)
Ulhassan, 2014, Sweden [27]Pre-/post-test263Emergency department and two cardiology wardsIdentify and eliminate areas of waste (DMAIC)Improve teamworkProcess outcome8/11 PassOverall inclusion
1.02 (0.74–1.42)
Overall trust
1.04 (0.79–1.38)
Overall productivity
1.0 (1.0–1.0)
Collar, 2012, USA [28]Pre-/post-test234Otolaryngology operating roomIdentify and eliminate areas of waste (DMAIC)Improve efficiency and workflowProcess outcome7/11 PassUnable to compute due to data not being provided.
Turn-over time reduced from 38.4 min to 29 min
Blackmore, 2013, USA [29]Retrospective cohort200Breast clinicIdentify and eliminate areas of wasteImprove timeliness of diagnosis and reduce surgical consultsProcess outcome6/11 PassReduced surgical consults
4.60 (1.82–11.62)
Simons P, 2014, Netherlands [30]Pre-/post-test167Radiotherapy departmentImplementation of a standard operating procedureImprove compliance to patient safety tasksProcess outcome8/11 PassOverall compliance
0.96 (0.58–1.58)
Mazzocato, 2012, Sweden [31]Case study156Accident and Emergency departmentIdentify and eliminate areas of waste, system restructuringIncrease number of patients seen and discharged within four hoursProcess outcome10/13 PassDischarged within four hours
1.07 (0.92–1.26)
Health and process outcome studies
Vermeulen, 2014, Canada [32]Pre-/post-test
Only study with control group
6 845 185Emergency departmentTraining and system redesignLeft without being seen, discharged within 48 h, readmitted within 72 h, died within 7 days of dischargeProcess and health outcome8/11 PassIn comparison to control group:
Left without being seen
1.05 (0.77–1.43)
Discharged within 48 h
1.19 (0.72–1.98)
Readmitted within 72 h of discharge
1.0 (1.0–1.0)
Died within 7 days of discharge
1.03 (0.84–1.26)
Yousri, 2011, UK [33]Pre-/post-test608HospitalIdentify and eliminate areas of wasteOverall mortality, 30 day mortality, door to theatre time, admission to a trauma wardHealth and process outcome6/11 Pass30 day mortality rate
1.71 (0.70–4.17)
Door to theatre time within 24 h
1.17 (0.86–1.60)
Admission to trauma bed
1.03 (0.90–1.20)
Ford, 2012, USA [34]Pre-/post-test219Emergency departmentValue stream analysis (VSM)Reduce time dependant stroke care and stroke mimicProcess outcome and health outcome7/11 PassPercent of patients with DNT < 60 min
1.50 (1.21–1.86)
Stroke mimic
0.64 (0.26–1.58)
Articles that failed methodology review
First author's last name, year of publication, country where study was doneStudy designNumber of participantsLocation of intervention (ex. Emergency department)InterventionIntervention goalType of outcomeQuality scoresMajor methodological drawbacks
Health outcome studies
Ulhassan, 2013, Sweden [35]Pre-/post-test4399Cardiology departmentChanges to work structure and processImprove patient careHealth outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Outcomes were not blinded

Wang, 2014, China [36]Pre-/post-test622Nephrology departmentTraining, treatment of high risk patients, specialized outpatient clinicIncidence of peritonitisHealth outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Data did not cover most episodes of intervention at follow-up

Process outcome studies
Wong, 2012, USA [37]Pre-/post-test234 616Cytology laboratoryNew imaging system, workflow redesignTurnaround time, productivity and screening qualityProcess outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes measures were not blinded

Lodge, 2008, UK [38]Post-test9297Division of diagnostics and clinical supportIntranet based waiting list for radiology servicesReduce radiology wait timesProcess outcome3/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done

Willoughby, 2010, Canada [39]Pre-/post-test1728Emergency departmentVisual reminders, standard process worksheets (PDCA)Improve wait timesProcess outcome1/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Piggott, 2011, Canada [40]Pre-/post-test1666Emergency departmentIdentify and eliminate areas of waste (VSM)Time to ECG, time to see MD, time to aspirin administrationProcess outcome3/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

Mazzocato, 2014, Sweden [41]Pre-/post-test1046Emergency departmentIdentify and eliminate areas of waste (VSM)To reduce time to see MD, to increase number of patients leaving within 4 h, reduce number present at 4pm shiftProcess outcome5/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done

Richardson, 2014, USA [42]Pre-/post-test565Emergency departmentEducational trainingDecrease wasted nursing timeProcess outcome3/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

Wojtys, 2009, USA [43]Pre-/post-test454Sport medicine practiceIdentify and eliminate areas of waste (VSM)Improve patient schedulingProcess outcome1/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Niemeijer, 2012, Netherlands [44]Pre-/post-test445Traumatology departmentIdentify and eliminate areas of waste (DMAIC)Reduce length of stay and costProcess outcome1/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done

Hakim, 2014, USA [45]Pre-/post-test361Medical and surgical unitsIdentify and eliminate areas of waste (PDCA)Improve admission medication reconciliationProcess outcome3/11 Fail
  • Insufficient follow-up time

  • Primary outcome measures not reliable

  • Primary outcome measure was not valid

van Lent, 2009, Netherlands [46]Pre-/post-test255Chemotherapy day unitIdentify and eliminate areas of waste (PDCA)Data efficiency, patient satisfaction and staff satisfactionProcess outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Bhat, 2014, India [47]Case study224Outpatient health information departmentIdentify and eliminate areas of waste (DMAIC)Reduce registration timeProcess outcome2/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Al-Araidah, 2010, Jordan [48]Case study217Inpatient pharmacyIdentify and eliminate areas of waste (DMAIC)Lead time reductionProcess outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Hydes, 2012, UK [49]Pre-/post-test178HospitalValue stream analysis (VSM)Improve efficiency and patient satisfactionProcess outcome2/11 Fail
  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Smith, 2011, USA [50]Pre-/post-test171Cystic fibrosis clinicIdentify and eliminate areas of waste (DMAIC)Decrease non-value added timeProcess outcome3/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

Kullar, 2010, UK [51]Post-test141Cochlear implant unitValue stream analysis (VSM)Wait time for cochlear implantationProcess outcome1/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Siddique, 2012, UK [52]Post-test80 (or 129)General surgery departmentOne stop cholecystectomy clinicWaiting list time, number of hospital visits and pre op admissionsProcess outcome4/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

Lunardini, 2014, USA [53]Case series38Operating roomValue stream analysis (VSM)To optimize instrument utilizationProcess outcome4/13 Fail
  • Insufficient data points for statistical analysis, outcomes were not blinded, primary outcome measure was not reliable

Yeh, 2011, Taiwan [54]Pre-/post-test36Private hospitalIdentify and eliminate areas of waste (DMAIC)Improve door to balloon time (AMI revascularization), length of stayProcess outcome3/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Luther, 2014, UK [55]Pre-/post-test20Medical admission unit wardIdentify and eliminate areas of waste (PDCA)Improve patient handoverProcess outcome3/11 Fail
  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Shah, 2013, USA [56]Pre-/post-test17Breast imaging centreIdentify and eliminate areas of waste (VSM)Improve workflowProcess outcome2/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • Primary outcome measure was not reliable

Gijo, 2013, India [57]Case studyNot statedPathology departmentIdentify and eliminate areas of waste (DMAIC)Reduce wait timeProcess outcome2/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Belter, 2012, USA [58]Pre-/post-testNot statedOncology outpatientIdentify and eliminate areas of waste (DMAIC)Decrease patient wait times and improve communicationProcess outcome2/11 Fail
  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable

Snyder, 2009, USA [59]Pre-/post-testNot statedRural healthcare organizationTrainingDecrease supply time, patient wait time, documentation in EMR within 30 minutesProcess outcome0/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis done

Silva, 2012, USA [60]Pre-/post-testNot statedClinical engineering departmentIdentify and eliminate areas of waste (DMAIC)Improve medical equipment inventory controlProcess outcome0/11 Fail
  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

  • DMAIC: define, measure, analyse, improve, control; PDCA: plan do check act; TPS: Toyota production system; VSM: value stream mapping; DNT: door to needle time.

  • Rate ratio <1 is intervention resulted in negative outcome; rate ratio >1 is intervention resulted in positive outcome.

Figure 1

Prisma flow diagram of the included studies.

Among the 22 studies accepted, none used high quality experimental study designs (i.e. randomized controlled trials) or even lesser quality quasi-experimental study designs (i.e. prospective longitudinal cohorts). All study designs were of relatively low quality with almost all using before and after study designs without control groups. In fact, only one accepted study had a control group [26]. Among accepted studies, 4 were exclusively concerned with health outcomes, 3 included both health and process outcomes and 15 included process outcomes only (Fig. 2).

Figure 2

Diagrammatic mapping of included studies to specific outcomes.

Health outcomes

Among the four accepted studies with health outcomes, only one found a statistically significant impact of Lean. They found a reduced relative rate of MRSA infections (RR = 2.47, 95% CI 1.87–3.27), although absolute reductions were very small [15]. The largest study by far included six million patients. This study found no impact of Lean on 30-day mortality rate post-hospital discharge (RR = 0.08, 95% CI −0.30 to 0.46) [13]. The other two studies under this category found no statistically significant impact on adverse events (RR = 0.91, 95% CI 0.72–1.16) or on MRSA incidence (RR = 0.99, 95% CI 0.98–1.01) [14,16] (Table 1).

Process outcomes

Among the 15 accepted studies that examined a vast array of process outcomes (including wait times, patient flow and workplace engagement, inclusion and productivity), only 2 found a statistically significant positive effect of Lean. The benefits included reduced patient visits (RR = 1.84, 95% CI 1.33–2.56) and reduced surgical consults (RR = 4.60, 95% CI 1.82–11.62) [24,29]. In five studies, rate ratios and confidence intervals were not computed because the authors did not include raw data (only summary data). None of the accepted studies reviewed actual financial costs (Table 1).

Health and process outcomes

Of the three articles that evaluated both health and process outcomes, only one article reported a positive effect of Lean in that it improved time dependent stroke care (RR = 1.50, 95% CI 1.21–1.86) [34]. Conversely, in a large study of over 6.8 million patients, Lean had no statistically significant impact on patients leaving without being seen (RR = 1.05, 95% CI 0.77–1.43), patients discharged within 48 h of presentation (RR = 1.19, 95% CI 0.72–1.98) or number of patients readmitted to the hospital within 72-h of discharge (RR = 1.00, 95% CI 1.00–1.00) [32] (Table 1).

The largest Lean healthcare transformation in the world – results from Saskatchewan

The HQC of Saskatchewan surveyed tens of thousands of patients discharged from hospitals pre- and post-Lean [11]. In this systematic review, the most relevant 30 outcomes are reported under the umbrella of 5 broad groupings, which include: self-reported health, hospital experience, communication, respect and patient management. Among the 30 outcomes considered, Lean had no statistically significant impact in 27 of them (Table 2). For example, 30 574 patients were surveyed on self-reported health with no observed impact from Lean (RR = 1.00, 95% CI 0.98–1.04). When measuring direct outcomes for 90 000 patients on their experience with doctors (RR = 1.01, 95% CI 1.00–1.02) and nurses (RR = 1.00, 95% CI 0.99–1.01), no effect of Lean was observed. Only three outcomes showed statistically significant positive outcomes of Lean including: staff washing or disinfecting their hands (RR = 1.179 07, 95% CI 1.05–1.10), staff checking ID bands (RR = 1.08, 95% CI 1.06–1.10) and patients given safety brochures (RR = 1.56, 95% CI 1.49–1.63). The results are found in Table 2.

View this table:
Table 2

Data collected by the Saskatchewan health quality council

Saskatchewan health quality council—pre- and post-Lean data
SHQC variablesPre-Lean (December 2009–January 2012)Post-Lean (February 2012–March 2014)Total sample size (n)Rate ratio95% CI
Sample size (N)%LCL–UCLSample size (n)%LCL–UCL
Reported health
 High self-reported health16 63734.5226.78–37.9613 93734.7526.16–38.5830 5741.000.98–1.04
Hospital experience
 Patient experience—quality of care transitions42 43531.4828.45–35.4336 00032.8028.09–35.7878 4351.021.00–1.03
 Percentage of patients rating their hospital as 9 or 10/1016 52651.9547.42–59.3813 80352.9346.76–60.0530 3291.010.99–1.04
 Percentage of patients reporting they would definitely recommend the hospital to family and friends16 49858.852.78–64.6013 82857.3852.13–65.2530 3260.980.94–1.01
Communication
 Patient experience—quality of communication with nurses50 16268.3064.26–70.7141 96569.3163.91–71.0792 1271.011.00–1.02
 Patient experience—Quality of communication with doctors49 82673.7870.36–76.4741 59373.9370.01–76.8191 4191.000.99–1.01
 Percentage of patients reporting they always received good communication about medicines18 85250.1943.55–54.7816 50449.9443.08–55.2635 3560.990.97–1.02
 Percentage of patients responding nurses always listened to them carefully16 75063.6056.93–68.4614 04564.7656.30–69.0830 7951.021.00–1.04
 Percentage of patients responding nurses always explained things clearly16 69963.9557.53–69.0313 93764.9056.88–69.6830 6361.011.00–1.03
 Percentage of patients responding doctors always explained things clearly16 63767.0761.02–72.3013 88566.9860.39–72.9330 5221.000.99–1.01
 Percentage of patients responding doctors always listened to them carefully16 56270.9265.07–75.9913 83071.5264.46–76.6130 3921.000.99–1.02
 Treatment plan explained clearly15 75377.7973.25–83.3713 20178.5872.69–83.9328 9541.011.00–1.01
 Family encouraged to participate in care plan13 95580.6075.47–85.7811 80981.3174.92–86.3325 7641.000.99–1.02
 Percentage of patients reporting staff took their preferences into account discussing health needs12 88624.8819.16–30.9310 98026.2818.56–31.5223 8661.051.00–1.10
 Percentage of patients reporting staff always told them what their new medicine was for946864.1754.65–70.10829263.2954.00–70.6717 7600.990.97–1.01
 Percentage of patients reporting staff always talked to them about medication side effects941336.0928.32–43.58824536.5427.67–44.2217 6581.010.97–1.05
Respect
 Percentage of patients responding nurses always treated them with courtesy and respect16 80077.2871.41–81.5014 05678.2670.85–81.8730 8561.000.99–1.01
 Percentage of patients responding doctor always treated them with courtesy and respect16 66183.2778.51–87.4813 90683.2578.00–87.9930 5671.000.99–1.01
 Staff respect culture, beliefs, values15 75392.2389.18–95.6813 22192.4388.83–96.0328 9741.000.99–1.01
 Doctors treated patients as a partner in care15 73682.4778.04–87.3413 15983.377.52–87.8528 8951.011.00–1.02
 Staff treated patients as a partner in care15 55278.8573.68–83.8013 05480.0773.13–84.3428 6061.021.00–1.03
 Doctors respect culture, beliefs, values15 49393.8187.18–91.4512 94894.3991.13–97.4928 4411.001.00–1.00
Patient care management
 Percentage of patients responding their pain was always well managed22 18363.9057.35–67.3819 17461.5556.90–67.8241 3570.960.95–0.98
 Percentage of patients reporting they always received help they needed when they wanted it17 59960.5053.98–65.3915 73759.1253.57–65.6033 3360.980.96–1.01
 Unnecessarily long wait time for room16 60779.4574.62–84.2913 88979.1874.08–84.8330 4961.000.99–1.02
 Staff washed or disinfected their hands16 52943.4936.41–48.2713 83946.7135.76–48.9130 3681.071.05–1.10
 Discharge organization16 43227.7123.05–33.9113 75327.8822.45–34.5030 1851.000.97–1.10
 Suffered medical error15 9763.701.26–5.7513 3523.771.10–6.0029 3280.980.87–1.10
 Staff checked ID band before care14 08560.5250.31–63.1812 22465.4249.73–63.7626 3091.081.06–1.10
 Given patient safety brochure10 85430.6418.58–41.42898036.6317.85–42.1619 8341.561.49–1.63
  • Pre- and post-Lean periods were identical (26 months each).

In 2014, the SUN randomly surveyed 1500 nurses on their Lean experience [12]. Among nurses who had direct experience with Lean (729–173 nurses—depending on the variable), 15 outcomes were reviewed. All 15 outcomes reported a statistically significant negative effect of Lean on nurse engagement, usefulness, patient care, time for patient care, workplace issues, availability of supplies, workload, stress and patient safety (Table 3). For example, the following outcomes were reduced, nurse engagement (RR = 0.50, 95% CI 0.40–0.65), quality of patient care (RR = 0.23, 95% CI 0.17–0.31) and patient safety (RR = 0.44, 95% CI 0.37–0.53) while the nurses workload and stress levels increased (RR = 0.29, 95% CI 0.24–0.35) (Table 3).

View this table:
Table 3

Data collected by the Saskatchewan Union of Nurses

Saskatchewan Union of Nurses (SUN)—Lean Healthcare 2014 Survey
Strongly disagree (%)Strongly agree (%)nRate ratio95% CI
Experience with Leana
 Lean activities engage frontline registered nurses23.0010.007290.500.40–0.65
 Ideas put forward by registered nurses are taken seriously30.506.107290.270.20–0.37
 Registered nurse input is meaningfully incorporated into the Lean process35.706.007290.250.18–0.33
 Registered nurses feel safe and supported in voicing criticisms and concerns about Lean initiatives41.005.607290.210.16–0.30
 Lean is a useful support for the nursing process38.304.007290.170.11–0.24
 Lean leads to improvements in direct patient care38.205.807290.230.17–0.31
 Lean has resulted in policies and procedures that improve the workplace29.105.207290.230.17–0.33
DeclinedImprovednRate ratio95% CI
Did Lean decline, stay the same or improveb
 The quality of supplies42.209.9011730.370.31–0.44
 The availability of supplies50.5017.9011730.580.52–0.66
 The time available for direct patient care41.4010.4011730.380.32–0.47
 Workload and stress49.507.9011730.290.24–0.35
 Patient safety31.0010.6011730.440.37–0.53
 The ability to meet professional standards in the nursing process34.509.3011730.370.31–0.45
 Time and opportunity for clinical education and training35.007.5011730.330.27–0.41
 Staff morale and engagement58.207.8011730.300.25–0.36
  • Note: Rate ratio <1 = negative impact of intervention; rate ratio >1 = positive impact of intervention.

  • an, sample size—individuals who say they have been involved personally in a workplace Lean initiative. Likert scale was used (where 1 means ‘strongly disagree’ and 5 means ‘strongly agree’).

  • bn, sample size—individuals who say their workplace has gone through a Lean improvement process (denominator equals 1500).

Discussion

The purpose of this systematic literature review was to independently assess the effect of Lean thinking or Lean interventions on worker and patient satisfaction, health and process outcomes and financial costs.

For worker satisfaction, the largest study was carried out by the SUN. With every outcome reviewed, Lean had an overall negative effect on worker satisfaction [12]. Among other accepted studies from the electronic search of peer reviewed articles, Lean was shown to have no impact on workplace engagement, inclusion and productivity [26,27]. These outcomes are surprising in that worker engagement and input are essential for Lean principles to succeed [2].

For patient satisfaction, the largest dataset available has been collected by the Saskatchewan HQC [11]. When measuring direct outcomes for patient experience with doctors and nurses, no statistically significant positive or negative effect of Lean was observed. In the 22 studies accepted from the electronic search of peer reviewed articles, none directly evaluated patient satisfaction. That is also surprising because Lean reportedly begins with identifying and ‘removing waste’ in order to ‘add value’ to the customer or patient [2]. That said, it is unclear if other variables, like reduced number of medical consultations were used as proxy outcomes for patient satisfaction and what the patient's perception is (positive or negative) as a result of receiving less visits with their physician [24,29].

Among health outcomes like mortality, no study found a statistically significant impact of Lean. As mentioned previously, the largest study included six million patients and found no impact of Lean on 30-day mortality rate post-hospital discharge [13]. This is perhaps not surprising as Lean potentially only influences healthcare delivery. It obviously has no impact on complex health outcomes like patient adherence to care, let alone the behavioural or social determinants of health [1].

With regard to safety and errors, our systematic review shows that one study found no impact on adverse events while two studies had conflicting results on the impact of Lean on MRSA incidence [1416]. The suggested impact of Lean on variables like adverse events is interesting because hospitals everywhere have successfully implemented various safety interventions that have proved effective but are not directly related with Lean. For example, the Agency for Healthcare Research and Quality estimates that 1.3 million fewer patients were harmed in American hospitals from 2010 to 2013. These outcomes were mostly due to common sense efforts to reduce surgical site infections, adverse drug events and other preventable incidents. As such, it is unclear what, if any, was the independent effect of Lean in comparison to a multitude of other diverse initiatives to promote safety and reduce errors in healthcare [61].

Although reduced financial cost is a reported benefit of Lean, it is worthy to note that we were unable to identify a single study that had actual quantifiable data to that effect. The province of Saskatchewan appears to be the only jurisdiction with actual financial cost information. External consultant fees were originally estimated to be $40.5 million but were reduced to $35 million when the Lean contract was terminated early [62]. Additionally, $17 million per year was required for internal kaizen promotion offices or $51 million total over the first 3 years. In return, official estimates of cost savings from the Saskatchewan health regions totalled $56934.26 [63]. If the numbers reported are accurate and true, it will mean that $1511 was spent on Lean for every one dollar saved by the province.

Strengths and limitations

The key strengths of our study are that it was a systematic review of Lean interventions in healthcare, it used a quality control checklist, and included a separate examination of both peer-reviewed articles and grey literature. There are also several limitations to our study. First, there are many and quite differing definitions of Lean in healthcare. This study did not attempt to strictly define what Lean is but rather relied on the definitions used by the authors of the articles included in our systematic review. Second, the outcomes were too diverse to permit a meta-analysis. Third, the study designs under review did not incorporate the use of control groups and therefore, it is unclear if the results are actually valid or what the results would be in comparison with a control group. Finally, the pre Lean HQC data for the province of Saskatchewan includes three small pilot projects in three health regions. However, month-to-month comparisons pre- and post-Lean found no statistically significant difference from the small pilot projects.

Comparison of findings

The results of our systematic review on Lean thinking and Lean interventions in healthcare provide additional insight and support the findings of other recent systematic reviews [5,64]. For example, Vest et al. [5] concluded that Lean interventions mainly focused on process outcomes in healthcare. Similarly, a Lean review completed by Mason et al. [64] found that the studies demonstrated improved process outcomes.

However, both Vest et al. [5] and Mason et al. [64] acknowledged that when critically examined, only a few articles met the inclusion criteria for their respective reviews. While Lean was found to be successful in some process outcomes, there were several and serious concerns with the reported study findings. Specifically, they noted that the articles reviewed were fraught with systematic bias, imprecision and serious methodological limitations, which undermined the validity of the results and made measuring and interpreting the true and independent effect of Lean on process and healthcare outcomes unclear and difficult.

Conclusion

The findings of our systematic review suggest that Lean interventions have: (i) no statistically significant association with patient satisfaction and health outcomes, (ii) a negative association with financial costs and worker satisfaction and (iii) potential yet inconsistent benefits on process outcomes like patient flow (reduced patient visits, reduced surgical consults, improved time dependent care) and safety (washing hands, staff checking ID bands and giving patients safety brochures).

More rigorous, higher quality and better conducted scientific research is required to definitively ascertain the impact and effectiveness of Lean in healthcare settings.

While some may strongly believe that Lean interventions lead to quality improvements in healthcare, the evidence to date simply does not support this claim. It is far more likely that Lean is but one of many strategies that might or might not have an impact on healthcare delivery.

The reality is that there are a multitude of internal and external variables that impact complex healthcare and process outcomes and that the independent effect of a specific intervention such as Lean is potentially minimal. For now, the question remains whether continuing to heavily invest in Lean is bringing us closer to or taking us further away from a much needed, viable, long-term solution to an increasingly problematic and unsustainable healthcare delivery system.

Authors’ contributions

J.M. and M.L. contributed to the original conception and design of the study. C.N. and M.L. were responsible for the acquisition of data. M.L. was in charge of the data analysis. J.M., M.L. and C.N. contributed to the interpretation of the data and the drafting of and critical revisions to the manuscript. All authors read and approved the final manuscript.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflict of interest statement

None declared.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

References

View Abstract