OUP user menu

★ Examining the Evidence: . ★

Determinants of innovation within health care organizations
Literature review and Delphi study

Margot Fleuren, Karin Wiefferink, Theo Paulussen
DOI: http://dx.doi.org/10.1093/intqhc/mzh030 107-123 First published online: 29 March 2004


Purpose. When introducing innovations to health care, it is important to gain insight into determinants that may facilitate or impede the introduction, in order to design an appropriate strategy for introducing the innovation. To obtain an overview of determinants of innovations in health care organizations, we carried out a literature review and a Delphi study. The Delphi study was intended to achieve consensus among a group of implementation experts on determinants identified from the literature review.

Data sources. We searched 11 databases for articles published between 1990 and 2000. The keywords varied according to the specific database. We also searched for free text. Forty-four implementation experts (implementation researchers, programme managers, and implementation consultants/advisors) participated in the Delphi study.

Study selection. The following studies were selected: (i) studies describing innovation processes, and determinants thereof, in health care organizations; (ii) studies where the aim of the innovations was to change the behaviour of health professionals; (iii) studies where the health care organizations provided direct patient care; and (iv) studies where only empirical studies were included.

Data extraction. Two researchers independently selected the abstracts and analysed the articles. The determinants were divided into four categories: characteristics of the environment, characteristics of the organization, characteristics of the user (health professional), and characteristics of the innovation. When analysing the determinants, a distinction was made between systematically designed and non-systematically designed studies. In a systematic study, a determinant analysis was performed and the innovation strategy was adapted to these determinants. Furthermore, the determinants were associated with the degree of implementation, and both users and non-users of the innovation were asked about possible determinants. In the Delphi study, consensus was defined as agreement among 75% of the experts on both the influence of a determinant and the direction towards which that influence tended (i.e. facilitating, impeding, or neutral).

Results. From the initial 2239 abstracts, 57 studies were retrieved and 49 determinants were identified that affected (impeded or facilitated) the innovation process. The experts identified one other determinant. Seventeen studies had a more-or-less systematic design; the others did not. After three rounds, consensus was reached on the influence of 49 out of 50 determinants.

Conclusion. The results of the literature review matched those found in the Delphi study, and 50 potentially relevant determinants of innovation processes were identified. Many of the innovation studies had several methodological flaws, such as not adjusting innovation strategies to relevant determinants of the innovation process, or that data on determinants were gathered only from non-users. Furthermore, the degree of implementation was evaluated in several ways, which made comparison difficult.

  • Delphi-study
  • determinants
  • health care organizations
  • implementation
  • innovations
  • literature review

The introduction of innovations to health care is widely recognized as a complex process. By innovation, we mean an idea, practice, or object that is perceived as new by an individual or other unit of adoption [1]. Several factors affect, positively or negatively, the process, and sometimes changes do not occur because health professionals do not accept the innovation or insufficient financial sources are made available to implement the innovation [25]. Although the number of studies of innovation processes has increased greatly over the last 15 years [5], little is known about the conditions for, or determinants of, the successful implementation of innovations to health care organizations [2]. By determinants, we mean factors that facilitate or impede actual change [2]. It is essential to identify determinants of a particular innovation in order to design an appropriate and effective innovation strategy that is adapted to these determinants [6,7].

So far, most research on innovations in health care has focused on individual doctors working independently in small practices, such as general practitioners (GPs) working with guidelines [3,4]. Less is known about the determinants of innovations in larger health care organizations, which may be different from those of innovations for individual health care professionals. For example, in a study on the implementation of public health guidelines on hearing disorders among doctors and nurses in Dutch public health organizations, in many cases management, rather than individual doctors and nurses, decided whether the guidelines would be introduced [8]. Unlike GPs, for example, these doctors and nurses were unable to decide independently whether or not to accept the guidelines. Thus far there has been no systematic overview of determinants of innovation processes in health care organizations.

To gain a better understanding of determinants of innovation processes in health care organizations, we carried out a systematic literature analysis of implementation studies in health care organizations. Subsequently, a Delphi study was carried out with implementation experts. The research questions were: (i) which determinants of innovation processes are reported in the literature?; and (ii) are these determinants recognized as being relevant by implementation experts and why?

Theoretical framework

In order to analyse the studies, we developed a framework representing the main stages in innovation processes and related categories of determinants (Figure 1), based on several theories and models [1,612]. Each of the four main stages in innovation processes (dissemination, adoption, implementation, and continuation) can be seen as points at which, potentially, the desired change may not occur. The transition from one stage to the next can be affected by various determinants, which can be divided into [6,7]: (i) characteristics of the socio-political context, such as rules, legislation, and patient characteristics; (ii) characteristics of the organization, such as staff turnover or the decision-making process in the organization; (iii) characteristics of the person adopting the innovations (user of the innovation), such as knowledge, skills, and perceived support from colleagues; and (iv) characteristics of the innovation, such as complexity or relative advantage.

Figure 1

Framework representing the innovation process and related categories of determinants.

Although the user of the innovation (i.e. the health professional) and the characteristics of the innovation play a crucial role in the innovation process, the intended user does not work in isolation and is part of an organization, which in turn is part of a larger environment. For these reasons, the characteristics of the organization and the socio-political context in which the organization operates should also be taken into account.

Systematically designed strategies and the measurement of determinants

When designing a strategy for implementing an innovation, it is essential to identify determinants that can affect the successful implementation of the innovation and to accommodate these in the strategy. Many theories can provide a starting point for changing the determinants that have been shown to be relevant for successful implementation. We differentiate between theory-based methods and practical strategies [2]. Whereas a method is a theory-based technique to influence behaviour or environmental conditions, a strategy is a way of organizing and operationalizing the theory-based method [2]. For example, a person’s belief about his/her ability to accomplish a certain innovation-related task, so-called self-efficacy, may be an important impeding determinant [13]. Modelling is a theory-based method for influencing self-efficacy. A practical strategy to overcome low self-efficacy may be role playing or a videotape demonstrating the desired behaviour.

If a determinant analysis is not done and/or the applied innovation strategy is not adapted to relevant determinants, and/or the strategy is not based on a proper theory, the innovation process might fail for three reasons [2,6,7,14]. Firstly, the applied innovation strategy may focus on determinants that are irrelevant to the innovation process. For example, in the above-mentioned implementation study on public health guidelines on hearing disorders, time constraints were thought to be an important determinant of non-adherence. However, one major problem was the lack of sound-proofed areas in which hearing tests could be performed in schools [5]. Secondly, the chosen theory-based methods and strategies may not be appropriate for influencing the relevant determinants of the innovation process. In the case of the public health guidelines, group education as an innovation strategy would not have solved the problem caused by the lack of sound-proofed areas. Thirdly, data on the determinants may have been gathered solely among non-users of the innovation instead of among both users and non-users. This may lead to misjudgement of the importance of a particular determinant of the innovation process. For example, the non-users may say that time constraint is a problem in adhering to the innovation; however, the users—if they had also been asked this question—may have given the same answer. Therefore users appear to adhere to the innovation despite their perceptions of time constraint, which means that reasons other than time constraint should be decisive with respect to the innovation’s acceptability.

As outlined above, if a strategy is not systematically designed, change may fail to occur. However, it may also affect the determinants found in the literature review. Studies in which a proper determinant analysis is performed and in which the strategies are adapted to these determinants may identify different or even fewer determinants compared with studies in which this was not done properly. When reviewing the literature we distinguish between systematically and non-systematically designed innovation studies. We define a systematically designed study as a study in which: (i) a determinant analysis is performed and the innovation strategy is adapted to these determinants; and (ii) the determinants are associated with the degree of implementation, and data on the determinants are gathered among both users and non-users.


Literature review

We searched 11 databases, mainly medical ones, for articles that were published between 1990 and 2000 and were written in English or in Dutch. We chose this time period because the tradition of innovation studies in the field of health care is quite young and we assumed that the results of earlier relevant studies would have been incorporated into the studies published between 1990 and 2000. The databases were Medline, PsycLIT, Eric, Combined Health Information Database (CHID), Healthpromis, Healthstar, Sociological Abstracts, Heclinet, Pica (a Dutch database of all university libraries), GLIN (a Dutch database on literature in the Netherlands), and SWTL (a Dutch social scientific journal on literature). We used keywords related to the specific database. Furthermore, we searched for free text, and, finally, checked the references in the studies we found. Examples of keywords are: innovation, guidelines, clinical protocols, implementation, institutionalization, change, diffusion of innovation, and health plan implementation.

Inclusion criteria were: (i) studies in which innovation processes within health care organizations were described and in which determinants were reported; (ii) studies in which the innovations were aimed at changing the behaviour of health professionals (e.g. guidelines); (iii) studies in which the health care organizations should have provided direct patient care and at least 10 professionals should have been involved in the innovation; and (iv) empirical studies only.

The first two authors independently selected the abstracts based on these criteria and retrieved the original articles. These were independently analysed by the same authors and the analyses were discussed afterwards. For the purpose of analyses, we developed a special record form based on our theoretical framework. We recorded the design of each study, the type of innovation, the respondents, the intended users of the innovation, the type of organization, the innovation strategy applied, the reported determinants, and the instruments used for measuring them. A list of potential determinants was derived from the literature [1,12,1528], and new determinants drawn from the articles were added. The determinants that finally resulted from our literature review are listed in Table 1. Furthermore, we analysed whether a study was systematically designed according to the criteria described in the section entitled Theoretical framework. Systematically designed studies were analysed individually to find out whether they generated different determinants compared with the less systematically designed studies.

View this table:
Table 1

Description of the determinants1

Determinants related to the socio-political context
1. Willingness of the patient to cooperate with the innovation
2. Degree to which the patient is aware of the health benefits of the innovation
3. Patient doubts concerning the health professional’s expertise and competence with respect to the innovation
4. Financial burden of the innovation imposed on the patient (e.g. no insurance coverage)
5. Patient discomfort (physical or emotional) as a result of the innovation
6. The extent to which the innovation fits into existing rules, regulations, and legislation
Determinants related to the organization
7. Decision-making process and procedures in the organization: top-down or bottom-up/participatory
8. Hierarchical structure: extent to which decision-making process is formalized through hierarchical procedures
9. Formal reinforcement by management to integrate innovation into organizational policies
10. Organizational size (number of employees): large, medium size, small
11. Functional structure (task oriented) versus product structure (output oriented)
12. Relationship with other departments or organizations: introvert or outreaching
13. Nature of the collaboration between departments involved in the innovation
14. Staff turnover: high, average, low
15. Degree of staff capacity in the organization or department that implements the innovation
16. Available expertise, in relation to the innovation in the organization or department
17. Logistical procedures related to the innovation, e.g. logistical problems in scheduling patients
18. Number of potential users to be reached: many, few
Determinants related to the adopting person/user/health professional
19. Support from/of colleagues in implementing the innovation
20. Support from/of other health professionals in implementing the innovation
21. Support from/of their supervisors in the department/organization with respect to the implementation of the innovation
22. Support from/of higher management in the organization with respect to the implementation of the innovation
23. Extent to which colleagues implement the innovation (modelling)
24. Extent to which the health professional has the skills needed to implement the innovation
25. Extent to which the health professional has the knowledge needed to implement the innovation
26. Self-efficacy: confidence to perform the behaviour needed to implement the innovation
27. Extent to which ownership by the health professionals is perceived
28. Extent to which the innovation fits in the perceived task orientation of the health professional
29. Extent to which the health professional expects that the patient will cooperate in the innovation
30. Extent to which the health professional expects that the patient will be satisfied with the innovation
31. Extent to which the health professional suffers from work-related stress
32. Extent to which goals of health professionals with respect to the innovation are contradictory
33. Extent to which the health professional has ethical problems with the innovation
Determinants related to the innovation
34. Extent to which the procedures/guidelines of the innovation are clear
35. Compatibility: degree to which the innovation is perceived as consistent with existing work procedures
36. Trialability: extent to which the innovation can be subjected to trial
37. Relative advantage: extent to which the innovation is perceived as advantageous
38. Observability: degree to which the results of the innovations are observable to the health professional
39. Extent to which the innovation is appealing to use
40. Relevance of the innovation for the patient: extent to which the innovation has added value
41. Extent to which the innovation carries risks to the patient compared with the existing situation
42. Frequency of use of the innovation: high, low
Determinants related to facilities needed to implement the innovation
43. Financial resources made available for implementing the innovation
44. Reimbursement for health professionals/organizations to facilitate extra efforts in applying the innovation
45. Other resources made available for implementing the innovation (e.g. equipment, manuals)
46. Administrative support available to the users (health professionals) of the implementation
47. Time available to implement the innovation
48. Availability of staff responsible for coordinating implementation in the organization/department
49. Health professionals are involved in the development of the innovation
50. Opinion leader who influences opinions of others in the organization or department (not the coordinator)
  • 1 Some determinants, such as reimbursement (number 44), can also be classified in another category, e.g. as a characteristic of the organization.

Delphi study

Next, a Delphi study was conducted to facilitate consensus among experts about the determinants identified in the literature review. Sixty-two Dutch implementation experts from several settings were approached using the snowball method: 44 were willing to participate. The first two authors personally contacted all experts. The main inclusion criterion was whether the expert considered himself/herself an expert in the field of innovation. The group consisted of researchers, programme managers, and implementation consultants/advisors working in public health institutes, hospitals, research institutes, and universities.

The experts were asked to decide whether a determinant was ‘impeding’, ‘facilitating’, or ‘neutral’, to clarify their responses by means of an open-ended answer, and to indicate how influential the determinant was (‘hardly’ to ‘very’). The experts had to give their answers to both extremes of a determinant, respectively (the extremes are described in Table 4); for example, ‘How influential is much support from colleagues in applying the innovation?’ and ‘How influential is low support from colleagues?’. They were also asked if they thought the determinant was adequately described. Consensus was considered adequate if 75% of the experts (including the ‘do not knows’) agreed on the influence of a determinant and on the reason(s) why the determinant was facilitating, impeding, or neutral.

There were three rounds. Feedback from the previous round was given anonymously by presenting both the group answer per determinant (percentage ‘impeding’, ‘facilitating’, ‘neutral’, or ‘do not know’), and a summary of explanations given by respondents and by the particular respondent.

View this table:
Table 4

Influence of 50 determinants according to 44 implementation experts, and number of studies in the review in which this was confirmed

DeterminantsFacilitatingImpedingNo influenceTotal number of studies where the determinant was measured
Expert opinionsNumber of studies where the determinant was found to be facilitatingExpert opinionsNumber of studies where the determinant was found to be impedingExpert opinionsNumber of studies where the determinant was found to be of no influence
Socio-political context
1. Patient cooperationPositive5Negative1416
2. Patient awareness of benefits1Well informed323
3. Patient doubts user’s expertise12Doubts2No doubts3
4. Financial burden on patientExtra costs5No extra costs5
5. Patient discomfort2Discomfort7No discomfort8
6. Regulations and legislationInnovation fits in1Innovation does not fit in66
7. Decision-making processCentralized and decentralized11
8. Hierarchical structureHigh formalization2Low formalization44
9. Reinforcement managementFormal reinforcement3No formal reinforcement24
10. Organizational size2323Size2
11. Functional or product structureTask oriented2Output oriented12
12. Relationship to other organizations4?23?435
13. Collaboration between departmentsGood collaboration2Poor collaboration1011
14. Staff turnover1Medium turnover1High turnover910
15. Staff capacityComplete5Incomplete1214
16. Available expertiseMuch expertise3Little expertise34
17. Logistical proceduresWell arrangedBadly arranged33
18. Number of potential users1/health professionalsFew3
19. Support colleaguesMuch support3Little support57
20. Support other professionalsMuch support8Little support1217
21. Support supervisorsMuch support4Little support67
22. Support higher managementMuch support5Little support79
23. ModellingPositive modelling3Negative modelling1125
24. SkillsSufficient skills3Limited skills1515
25. KnowledgeSufficient knowledge2Limited knowledge152217
26. Self-efficacyHigh self-efficacy7Low self-efficacy67
27. OwnershipPositive modelling ownership10Negative282232
28. Task orientationInnovation fits in2Innovation does not fit in89
29. Expects patient cooperationHigh expectation2Low expectation6127
30. Expects patient satisfactionHigh expectation2Low expectation33
31. Work-related stress12Stress2No stress2
32. Contradictive goalsNot contradictive1Contradictive22
33. Ethical problemsNo problemsProblems44
34. Clear proceduresClear3Not clear1011
35. CompatibilityCompatible3Not compatible810
36. TrialabilityHigh trialability2Low trialability12
37. Relative advantageHigh advantage4Disadvantage4Neutral4
38. ObservabilityObservable1Not observable3125
39. AppealingAppealing5Not appealing1114
40. Relevance for patientAdded value6No added value1214
41. Risks for patientHigh risk6No risk6
42. Frequency in the use of facilitiesFrequent use1Limited use22
43. Financial resourcesMany resources5Few resources121214
44. ReimbursementReimbursement3No reimbursement88
45. Other resourcesMany5Few1314
46. Administrative supportMuch support1Little support5126
47. Time availableMuch time1Little time2828
48. CoordinatorAvailable5Not available4129
49. Users involved in developmentUsers involved3Users not involved24
50. Opinion leaderAvailable2Not available13
  • 1 No consensus among experts on one of the extremes: for explanation see Results.

  • 2,3 Differences between study in review and expert opinions: for explanation see Results.

  • 4 No consensus at all among experts.

Of the 44 experts who were initially willing to participate, 40 experts completed the first round, 37 the second round, and 34 the third round of consensus discussions. The main reason for non-response was lack of time. One respondent did not agree with the Delphi study method. In total, 33 experts responded to all three rounds and five experts responded to two rounds.


Studies with and without systematically developed innovation strategies

In total, 2239 abstracts were collected, from which 57 studies were selected. Most abstracts (n = 1963) were excluded because no determinants were reported, or because the innovation was not aimed at changing health professional behaviour. Other abstracts were excluded because they did not focus on health care organizations (n = 30) or did not report on empirical studies (n = 189). A determinant analysis had been carried out in six studies [2934]. Although in 25 studies one or more innovation strategies were reported, none of them were linked to the outcomes of a previously conducted determinant analysis, either theoretically or empirically. In one study the strategy was based on a review of the literature [35]. Thus none of the studies met both our criteria of a systematically designed study. In 17 studies the determinants were associated with the degree of implementation, and data on determinants were gathered among both users and non-users. Therefore these studies had a partial systematic design. The 57 studies included for further analyses are described in Tables 2 (partly systematically designed studies) and 3 (non-systematically designed studies).

View this table:
Table 2

Partly systematically designed studies: no determinant analysis or innovation strategy not linked to determinants (criterion a), but determinants are related to the degree of implementation and are measured by both users and non-users (criterion b) (see Theoretical framework) (n = 17)

AuthorSubject and settingDeterminant analysisStrategy usedRespondentsDeterminants reported1
Baskerville and LeTouzé, 1990 [36]P: health promotion; hospitalsNo732 chief executive officers,directors of nursing9, 10, 16, 19, 20, 22, 27, 28, 43,45, 46, 48
Weir et al., 1994 [37]Q: integrated electronic medical record system; hospitalsNo40 physicians, nurses, ward clerks managers, administrators7, 21, 24, 27, 36, 37, 39, 45, 46, 48, 49, s
Maly et al., 1996 [38]G: geriatric outpatients assessment; primary health careYes87 physicians1, 27, 37, 45, 50, h
Medder et al., 1997 [30]P: preventive services; family practiceYesYes408 family physicians19, 27, 34, 39, 43, 47
Soumerai et al., 1998 [32]G. acute myocardial infarction; hospitalsYesYes13 physicianss, h
Li et al., 1999 [39]G: prenatal screening; prenatal clinicsNo30 nurses/staff members1, 12, 14, 15, 17, 34, 39, 46, 47, s
Han et al., 1996 [40]G: teaching breast self-examination; hospitalsNo140 nurses24, 25, 26, 27, d
Gemson et al., 1996 [41]P: put prevention into practice; hospitalsYes89 physicians25, 26, 29, d
Wiefferink and Dukkers van Emden, 1996 [42]M: cooperation between GPs and medical specialistNo70 project coordinators14, 27, 37, 40, 43, 47, 48, 49, s
Freed et al., 1994 [43]G: hepatitis B immunization in infants; hospitals, family practiceYes478 family physicians, paediatricians1, 2, 4, 20, 27, 43, d
Reis et al., 1994 [44]G: oral rehydration therapy; hospitals, HMOs, private practicesNo104 paediatricians1, 5, 20, 24, 25, 39, 44, d, h
Grilli and Lomas, 1994 [45]G: review study; various organizationsNo23 articles34, 36, 38, d
Sluijs and Dekker, 1999 [46]Q: patient reports, peer review; guidelines; professionalsNo908 allied health professionals23, 27, 47, d
Farris and Schopflocher, 1999 [47]P: pharmaceutical care optimizing medication; pharmaciesNo182 pharmacists24, 26, 27, 40
Venkataraman et al., 1997 [48]P: patient-oriented pharmacy; pharmaciesNo162 pharmacists6, 20, 21, 26, 28, 29, 44, 47
Odedina et al., 1995 [49]P: pharmaceutical care in community practice; pharmaciesNo20 pharmacists1, 3, 11, 14, 15, 20, 22, 24, 26, 27, 29, 30, 45, 47, h
Odedina et al., 1996 [50]P: pharmaceutical care in community practice; pharmaciesNo617 pharmacists19, 21, 26, 27, 30, 40
  • P, protocol; G, guidelines; Q, quality system; M, muliple innovations; s, no good/good innovation strategy used; h, history patient; d, demographics user; HMO, health maintenance organization.

  • 1 See Table 1 for description of the determinants.

View this table:
Table 3

Non-systematically designed studies: no determinant analysis or innovation strategy not linked to determinants (criterion a) and determinants not related to the degree of implementation or measured by either the users or the non-users (criterion b) (see Theoretical framework) (n = 40)

AuthorSubject and settingDeterminant analysisStrategy usedRespondentsDeterminants reported1
McPhee and Bird, 1990 [51]G: cancer screening; primary careNo52 physicians1, 4, 5, 17, 27, 41, 46, 47, h
Corrigan et al., 1992 [52]P: behavioural therapy; psychiatric hospitalsNo40 psychiatrists/psychologists/social workers, 219 nurses, 47 administrators, 26 ancillary workers1, 2, 13, 14, 15, 19, 25, 27, 31, 33, 40, 43, 46
Harke and Richgels, 1992 [53]G: continence programme; nursing homesYes166 nurses15, 21, 27, 29, 34, 35, 47
Heins et al., 1992 [54]G: diabetes patient education; hospitalsNo±80 physicians, nurses, administrative staff8, 9, 14, 15, 19, 20, 22, 23, 27, 35, 40, 43, 45, 47, 48, 50
Inouye et al., 1993 [55]P: geriatric care programme preventing decline; hospitalsYes12 nurses13, 16, 21, 22, 43, 47
Hallstrom and Beck, 1993 [31]G: pre-operative skin shaving; hospitalsYesYes5 surgeons, 4 anesthesiologists, 19 nurses23, 27
Wender, 1993 [56]M: review-study cancer screening; primary careNo57 articles4, 5, 24, 25, 27, 29, 34, 38, 40, 44, 45, 47, h
Skovholt et al., 1994 [57]G: prenatal care; medical clinics, agencies providing prenatal careYes16 organizations: physicians, nurses, prenatal educators13, 27, 39, 44, 47, 49
Lipscomb and Ling, 1995 [33]G: laparoscopic sterilization; hospitalsYesYesphysicians, nurses; number unknown5, 24, 30, 35, 37, 47, 50
Steffensen et al., 1995 [58]G: stroke prevention; hospitals, general practiceYes161 GPs, 74 heads medical/neurological departments25, 40, 41
Rumore et al., 1995 [59]G: patient counselling new prescription; hospitals, pharmaciesNo194 pharmacists1, 2, 3, 5, 15, 20, 25, 34, 35, 39, 40, 41, 43, 44, 45, 47, h
O’Connor et al., 1996 [60]G: cystitis; primary care practices in HMOYes5 nurses19, 20, 34, 35, 39, s
Ramsey et al., 1996 [61]G: universal precaution procedures; hospitalsNo153 nurses15, 20, 22, 23, 33, 39, 40, 45, 47
Scutchfield et al., 1997 [62]G: future of public health; public health organizationsNo66 local health officials12, 27, 28, 35
Lekan et al., 1998 [63]G: urinary incontinence; long-term care facilitiesYes141 nurses13, 14, 15, 19, 21, 24, 25, 46, 47, s
Ely et al., 1999 [64]G: ventilator weaning; hospitalsYes89 physicians13, 20, 28, 35
Taylor et al., 1998 [65]P: community-based heart health promotion; public health departmentsNo262 staff members1, 5, 9, 11, 12, 13, 14, 15, 16, 22, 27, 28, 38, 40, 43, 45, 48, d
Lau et al., 1998 [34]G: guidance system for managing stroke; hospitals, family practiceYesYes11 physicians35, 45, 47, s
Brockopp et al., 1998 [66]G: pain management; hospitalsYes5 physicians, 7 nurses6, 8, 13, 20, 25, 28, 33, 41, 43, 45
University of Michigan, 1998 [67]G: breast and cervical cancer screening; health agencies providing screening facilitiesNo192 screening providers, programme staff6, 44, 47
Warren and Pohl, 1990 [68]G: cancer screening; primary careNo97 nurse practitioners24, 25, 43, 47, h
Sutherland et al., 1996 [69]M: cervical cancer screening; hospitalsNo44 physicians, 90 nurses, 103 patients1, 3, 5, 20
Lekkerkerk et al., 1998 [70]G: infection prevention urology; hospitalsNo171 hospital hygienists28, 47
Sluijs and De Bakker, 1995 [71]Q: quality systems; hospitals, nursing homes, homes for elderlyNo22 managers8, 13, 20, 22, 24, 25, 27, 38, 39, 40, 47, 49, s
Kaassenbrood and Van Tilburg, 1997 [72]G: discharge letter for psychiatric patients; psychiatric hospitalsYes10 physicians27, s
Van Rens et al., 1999 [73]G: prevention of sudden infant death; child and baby health clinicsNo15 nurses1, 4, 6, 20, 23, 24, 27, 41, h
Mur-Veeman et al., 1994 [74]M: home care arrangements; various organizations, e.g. hospitals, nursing homesNoManagers, physicians; number unknown6, 8, 12, 15, 20, 27, 28, 32, 43, 44, 48, d, s
Wolf, 1995 [75]M: psychiatric disorders; mental health institutionsNo258 managers, physicians, project leaders1, 6, 13, 15, 16, 21, 27, 28, 32, 34, 38, 40, 43, 45, 47, 48, d, s
Gökçay et al., 1997 [76]G: breast-feeding; hospitalsNo18 physicians, 45 nurses1, 5, 14, 15, 24, 25, 27, 29, 41, 45, 47
Hirth et al., 1996 [77]G: antibiotics against Helicobacter pylori; hospitals, family practiceNo950 internists, family practitioners, gastroenterologists42, d
Klazinga, 1994 [78]Q: record keeping, antibiotics, preoperative consultation, bedsores; hospitalsYes123 hospitals from 15 European countries15, 22, 27, 43, 48
Ettema, 1993 [79]G: bandaging practice; hospitalsYes22 physicians, nurses, managers, dermatologists24, 25, 27, 31, 34, 39, 42, 48, s
Ellrodt et al., 1995 [35]G: discharge patients with chest pain; hospitalsYesPhysicians; number unknown1, 15, 17, h
Matthews et al., 1994 [80]G: choice of radiographic projections; hospitalsYes29 radiologists, 116 radiographers20, 24, 27, h
Westrate et al., 1994 [29]G: 16 neurological guidelines; hospitalsYesYes20 physicians, staff members39
McNabb and Keller, 1991 [81]G: protective action against transmission of HIV; hospitalsNo249 nurses25, 27, 33, 39, 40, 45, 47, s, h
Kelen et al., 1990 [82]G: universal precaution; emergency departmentsYes75 physicians, nurses, paramedics25, 35, 39, 40, 47
Moriaty and Stephens, 1990 [83]P: diabetes teaching; hospitalsNo39 staff nurses1, 20, 25, 26, 27, 47, h
Lia-Hoagberg et al., 1999 [84]G: violence prevention and positive parenting; public health agenciesYes51 public health nurses13, 14, 22, 24, 25, 34, 35, 47, d
Goodson et al., 1999 [85]P: put prevention into practice; public health primary care sitesYes34 primary health care workers1, 4, 9, 10, 12, 13, 14, 27, 29, 34, 39, 44, 47, s, h
  • P, protocol; G, guidelines; Q, quality system; M, multiple innovations; s, no good/good innovation strategy used; h, history patient; d, demographics user.

  • 1 See Table 1 for description of the determinants.

Study designs

Most studies (63%, n = 36) had a cross-sectional design. The instruments used for measuring the innovation determinants were questionnaires (54%, n = 31) and interviews (44%, n = 25). There were four types of innovation: guidelines (63%, n = 36), programmes (e.g. health promotion programmes) (21%, n = 12), quality systems (7%, n = 4), or a combination of these (9%, n = 5). Most innovations focused on doctors (49%, n = 28), followed by nurses (40%, n = 23) and pharmacists (9%, n = 5). This is in line with the kind of organizations involved: hospitals (58%, n = 33), primary health care centres (16%, n = 9), and pharmacies (9%, n = 5). There was great variety in the way the degree of implementation was measured, ranging from asking management whether the innovation was used in the organization (yes/no) to daily recording per patient of the number of times each health professional had adhered to the guidelines.

Relative importance of determinants

Fifty different determinants were reported (Table 4). Except for the determinant ‘number of potential users to be reached’, all determinants were measured at least once; the average was 8.1 (range 1–32). Most determinants were characteristic of the person adopting the innovation (user), followed by characteristics of the organization, the innovation, and the socio-political context. The determinants were reported as impeding innovation 2.5 times more often (339 times) than they were reported as facilitating it (133 times) (Table 4). In only 10 out of 398 cases was a determinant judged to have a neutral effect (Table 4).

The analyses show that if a determinant was reported as facilitating the innovation (e.g. high self-efficacy), within the same study the opposite of that determinant (low self-efficacy) was nearly always reported as being impeding. In 48 out of 398 cases, a determinant was reported as being facilitating only. However, if a determinant was reported as impeding the innovation process (e.g. low self-efficacy), within the same study the opposite of that determinant (high self-efficacy) was only reported as facilitating it in one-sixth of all cases. In 256 of the 398 cases, a determinant was reported as only being impeding. These results hold true even after correction for the fact that some researchers only asked for impeding determinants, whereas others only asked for facilitating determinants.

Partly systematically versus non-systematically designed studies

Comparison of the more systematically designed studies (n = 17) with the non-systematically designed studies (n = 40) showed that fewer determinants were reported in the more systematic studies (a mean of 6.4 versus a mean of 7.3). This may be due to the fact that in 59% of the more systematic studies, the determinants were selected beforehand on theoretical or empirical grounds, whereas this occurred in only 10% of the non-systematic studies. Furthermore, the more systematically designed studies showed fewer determinants to have a neutral effect.

Delphi study

After three rounds there was consensus on nearly all 50 determinants (Table 4) and also on the reasoning behind why a determinant was impeding, facilitating, or neutral (available on request). There was no consensus on the magnitude of the effect of the determinant ‘relationship with other organizations’ (Table 1, number 12). There was also no consensus on one of the extremes of determinants 2, 14, and 18: ‘patient not aware of benefits’ (70% said this was impeding), ‘low staff turnover’ (58% said this was impeding) and ‘many people using the innovation’ (71% said this would only be impeding in case of active resistance).

Comparison review with Delphi study: relevance determinants

The experts considered nearly all determinants identified from the literature to be relevant to innovation processes (Table 4); however, there were three exceptions (Table 4, footnote 3). Firstly, the experts thought the determinant ‘organizational size’ (determinant 10) was of no influence in innovation processes because other related determinants, such as the hierarchical structure (determinant 8), were more important. Two studies reported that the size of an organization affected the innovation processes [36,85]. However, these results were contradictory: one study found large organizations as being facilitating and small organizations as being impeding [36], and the other found the opposite [85]. Secondly, in five studies a strong inter-organizational network was reported to be a relevant determinant of innovation processes (determinant 12), but the experts did not reach consensus on this determinant. Thirdly, the determinant ‘number of potential users to be reached’ (determinant 18) was not identified in the literature review, but was added by the experts as being a relevant determinant in innovation processes.

Comparison review with Delphi study: direction of influence determinants

When comparing the direction of influence (highly impeding versus highly facilitating or neutral), the results of the review generally matched the results of the Delphi study. A determinant identified as being a facilitating factor in the literature review was also judged so by the experts, with some exceptions (Table 4, footnote 2). In 10 studies a determinant was reported as being neutral, but other studies and the experts did not confirm this. For example, from the 32 studies that measured the influence of ownership (determinant 27), only two studies reported that this determinant was of no influence. The extremes ‘patient has no doubts about health professional’s expertise’ (determinant 3), ‘patient has no discomfort’ (determinant 5), and ‘the health professional does not suffer from work-related stress’ (determinant 31) were reported in a few studies to be of influence, whereas the experts thought they were not. Furthermore, patient lack of awareness of the potential benefit of an innovation (‘not aware of the benefits’) was considered an impediment in three studies, but not by the experts, only 70% of whom thought it was an impediment.


A first conclusion is that the innovation studies retrieved in our literature review did not have a systematic design. None of the 57 studies met the first criterion of having conducted a determinant analysis beforehand and of applying the results to the innovation strategy. Although, in many studies, one or more innovation strategies were applied, none were based on a theory (theoretical methods for change). This is surprising because such analyses are considered important and can help avoid the use of inappropriate, and thus ineffective, strategies, and hence save time and money. Furthermore, two-thirds of the studies did not meet the second criterion (associated the determinants to the degree of implementation and having gathered data on determinants among both users and non-users). The consequence of not systematically designing an innovation strategy is that the intended change might fail.

We can only speculate on the reason why we found so few well designed innovation studies. Implementation research in health care is still in its infancy and there are few innovation theories. Moreover, empirical studies mainly consist of case studies and there are few standardized procedures for measuring determinants as well as the stages of change (from dissemination, adoption, and implementation to continuation). Another possible reason is that the diffusion of good theories and studies is less widespread than thought. A consequence of finding only a few well designed studies is that we were unable to compare the determinants found in systematically designed studies with those found in non-systematically designed studies.

A second conclusion is that many of the innovation studies showed methodological flaws. Besides the already above-mentioned fact that determinants were not related to the degree of implementation, the degree of implementation was assessed in different ways, such as level of use (non-use, full use, adapted use), completeness of use (applied proportion of recommended activities), frequency of use (number of times used), intensity of use (number of people who use innovation), and duration of use. This means that the degree of implementation and the association with particular determinants depend on the operationalization of implementation. If, for example, we were to define ‘a smoker’ as someone who had smoked at least one cigarette during the past year (rather than, for example, as someone who had smoked seven cigarettes or more daily), we would not only find more ‘smokers’, but also different determinants of the smoking behaviour.

Despite the above-mentioned limitations of the studies reviewed, the determinants identified by the literature review and their effect (impeding, enhancing, or neutral) were consistent with the opinion of the experts. For example, when the literature review showed that ‘much support from colleagues’ was facilitating, the experts confirmed this. Fifty potential determinants were identified. We use the word ‘potential’ because there were discrepancies between the literature and Delphi studies that could be due to the non-systematic design of the included studies, and also because of the fact that the expert opinion was subjective and may not be empirically valid.

The literature review identified more determinants that impeded rather than facilitated innovation, even after correction for the fact that some studies investigated only facilitating or only impeding determinants. This could have been caused by the inclusion of more studies with negative or inconclusive outcomes of the innovation; however, because many studies did not report how successful the innovation was, we cannot determine whether this was the case. Furthermore, it can be stated that if a determinant is measured, the determinant shows up to be relevant for the innovation process. The more or less systematically designed studies yielded fewer determinants than the non-systematically designed studies, as we had anticipated (see the introduction to this study). Most identified determinants were related to the individual user. However, this does not necessarily mean that these were the most important determinants, because we also found that if a determinant is measured it will show up to be relevant for the innovation process. The determinant ‘self-efficacy’, for example, was measured mainly in studies on health promotion. This suggests that the outcome of a particular study on innovation determinants is liable to selection bias on the part of the researchers.

Our study had some limitations. Although our database search was extensive, we may have overlooked one or more relevant studies, in particular those published in internal or governmental reports. Another limitation is that the Delphi group consisted of experts (academic and practitioner) from three different professional disciplines. Their familiarity, or not, with current opinion, as expressed in the published literature, could have influenced the agreement between the determinants identified from the literature and the expert opinion, producing more apparent agreement than there was in reality. Moreover, we do not know whether the experts who dropped out of the Delphi study agreed or not with the other experts, thus potentially influencing the degree of agreement established. Furthermore, it is not possible to rank the 50 determinants in order of importance because many determinants may have been related to the type of innovation studied and to the context in which the innovation was introduced. For example, ‘observability’ may be a greater impediment in a public health setting than in an emergency department setting. Finally, the interrelation of the 50 determinants is unclear: given the presence of determinant x, determinant y may lose its importance.

Despite these criticisms, we feel encouraged by the participants of the Delphi study who said the 50 potential determinants provide a good starting point for developing a measurement instrument or can function as a checklist, if reduced to ±10 main categories, for daily innovation practice.

On the basis of these conclusions and critical reflections, we suggest that the relative impact of the 50 determinants of innovation processes be evaluated in an empirical study. We would like to invite implementation researchers and programme managers to explore this list of determinants further, and to report their results. In the future, the quality of innovation studies should be improved by systematically designing strategies that are tailored to an empirically based selection of innovation determinants, and by asking both users and non-users why they accepted or rejected the innovation. Moreover, we recommend that researchers look more closely at the procedures used to measure the degree of implementation of an innovation. We believe the first step would be to describe systematically why a certain way of assessing the degree of implementation was chosen and what the implications are for the reported results.


View Abstract