OUP user menu

★ Examining the Evidence: . ★

Determinants of innovation within health care organizations
Literature review and Delphi study

Margot Fleuren , Karin Wiefferink , Theo Paulussen
DOI: http://dx.doi.org/10.1093/intqhc/mzh030 107-123 First published online: 29 March 2004

Abstract

Purpose. When introducing innovations to health care, it is important to gain insight into determinants that may facilitate or impede the introduction, in order to design an appropriate strategy for introducing the innovation. To obtain an overview of determinants of innovations in health care organizations, we carried out a literature review and a Delphi study. The Delphi study was intended to achieve consensus among a group of implementation experts on determinants identified from the literature review.

Data sources. We searched 11 databases for articles published between 1990 and 2000. The keywords varied according to the specific database. We also searched for free text. Forty-four implementation experts (implementation researchers, programme managers, and implementation consultants/advisors) participated in the Delphi study.

Study selection. The following studies were selected: (i) studies describing innovation processes, and determinants thereof, in health care organizations; (ii) studies where the aim of the innovations was to change the behaviour of health professionals; (iii) studies where the health care organizations provided direct patient care; and (iv) studies where only empirical studies were included.

Data extraction. Two researchers independently selected the abstracts and analysed the articles. The determinants were divided into four categories: characteristics of the environment, characteristics of the organization, characteristics of the user (health professional), and characteristics of the innovation. When analysing the determinants, a distinction was made between systematically designed and non-systematically designed studies. In a systematic study, a determinant analysis was performed and the innovation strategy was adapted to these determinants. Furthermore, the determinants were associated with the degree of implementation, and both users and non-users of the innovation were asked about possible determinants. In the Delphi study, consensus was defined as agreement among 75% of the experts on both the influence of a determinant and the direction towards which that influence tended (i.e. facilitating, impeding, or neutral).

Results. From the initial 2239 abstracts, 57 studies were retrieved and 49 determinants were identified that affected (impeded or facilitated) the innovation process. The experts identified one other determinant. Seventeen studies had a more-or-less systematic design; the others did not. After three rounds, consensus was reached on the influence of 49 out of 50 determinants.

Conclusion. The results of the literature review matched those found in the Delphi study, and 50 potentially relevant determinants of innovation processes were identified. Many of the innovation studies had several methodological flaws, such as not adjusting innovation strategies to relevant determinants of the innovation process, or that data on determinants were gathered only from non-users. Furthermore, the degree of implementation was evaluated in several ways, which made comparison difficult.

  • Delphi-study
  • determinants
  • health care organizations
  • implementation
  • innovations
  • literature review

The introduction of innovations to health care is widely recognized as a complex process. By innovation, we mean an idea, practice, or object that is perceived as new by an individual or other unit of adoption [1]. Several factors affect, positively or negatively, the process, and sometimes changes do not occur because health professionals do not accept the innovation or insufficient financial sources are made available to implement the innovation [25]. Although the number of studies of innovation processes has increased greatly over the last 15 years [5], little is known about the conditions for, or determinants of, the successful implementation of innovations to health care organizations [2]. By determinants, we mean factors that facilitate or impede actual change [2]. It is essential to identify determinants of a particular innovation in order to design an appropriate and effective innovation strategy that is adapted to these determinants [6,7].

So far, most research on innovations in health care has focused on individual doctors working independently in small practices, such as general practitioners (GPs) working with guidelines [3,4]. Less is known about the determinants of innovations in larger health care organizations, which may be different from those of innovations for individual health care professionals. For example, in a study on the implementation of public health guidelines on hearing disorders among doctors and nurses in Dutch public health organizations, in many cases management, rather than individual doctors and nurses, decided whether the guidelines would be introduced [8]. Unlike GPs, for example, these doctors and nurses were unable to decide independently whether or not to accept the guidelines. Thus far there has been no systematic overview of determinants of innovation processes in health care organizations.

To gain a better understanding of determinants of innovation processes in health care organizations, we carried out a systematic literature analysis of implementation studies in health care organizations. Subsequently, a Delphi study was carried out with implementation experts. The research questions were: (i) which determinants of innovation processes are reported in the literature?; and (ii) are these determinants recognized as being relevant by implementation experts and why?

Theoretical framework

In order to analyse the studies, we developed a framework representing the main stages in innovation processes and related categories of determinants (Figure 1), based on several theories and models [1,612]. Each of the four main stages in innovation processes (dissemination, adoption, implementation, and continuation) can be seen as points at which, potentially, the desired change may not occur. The transition from one stage to the next can be affected by various determinants, which can be divided into [6,7]: (i) characteristics of the socio-political context, such as rules, legislation, and patient characteristics; (ii) characteristics of the organization, such as staff turnover or the decision-making process in the organization; (iii) characteristics of the person adopting the innovations (user of the innovation), such as knowledge, skills, and perceived support from colleagues; and (iv) characteristics of the innovation, such as complexity or relative advantage.

Figure 1

Framework representing the innovation process and related categories of determinants.

Although the user of the innovation (i.e. the health professional) and the characteristics of the innovation play a crucial role in the innovation process, the intended user does not work in isolation and is part of an organization, which in turn is part of a larger environment. For these reasons, the characteristics of the organization and the socio-political context in which the organization operates should also be taken into account.

Systematically designed strategies and the measurement of determinants

When designing a strategy for implementing an innovation, it is essential to identify determinants that can affect the successful implementation of the innovation and to accommodate these in the strategy. Many theories can provide a starting point for changing the determinants that have been shown to be relevant for successful implementation. We differentiate between theory-based methods and practical strategies [2]. Whereas a method is a theory-based technique to influence behaviour or environmental conditions, a strategy is a way of organizing and operationalizing the theory-based method [2]. For example, a person’s belief about his/her ability to accomplish a certain innovation-related task, so-called self-efficacy, may be an important impeding determinant [13]. Modelling is a theory-based method for influencing self-efficacy. A practical strategy to overcome low self-efficacy may be role playing or a videotape demonstrating the desired behaviour.

If a determinant analysis is not done and/or the applied innovation strategy is not adapted to relevant determinants, and/or the strategy is not based on a proper theory, the innovation process might fail for three reasons [2,6,7,14]. Firstly, the applied innovation strategy may focus on determinants that are irrelevant to the innovation process. For example, in the above-mentioned implementation study on public health guidelines on hearing disorders, time constraints were thought to be an important determinant of non-adherence. However, one major problem was the lack of sound-proofed areas in which hearing tests could be performed in schools [5]. Secondly, the chosen theory-based methods and strategies may not be appropriate for influencing the relevant determinants of the innovation process. In the case of the public health guidelines, group education as an innovation strategy would not have solved the problem caused by the lack of sound-proofed areas. Thirdly, data on the determinants may have been gathered solely among non-users of the innovation instead of among both users and non-users. This may lead to misjudgement of the importance of a particular determinant of the innovation process. For example, the non-users may say that time constraint is a problem in adhering to the innovation; however, the users—if they had also been asked this question—may have given the same answer. Therefore users appear to adhere to the innovation despite their perceptions of time constraint, which means that reasons other than time constraint should be decisive with respect to the innovation’s acceptability.

As outlined above, if a strategy is not systematically designed, change may fail to occur. However, it may also affect the determinants found in the literature review. Studies in which a proper determinant analysis is performed and in which the strategies are adapted to these determinants may identify different or even fewer determinants compared with studies in which this was not done properly. When reviewing the literature we distinguish between systematically and non-systematically designed innovation studies. We define a systematically designed study as a study in which: (i) a determinant analysis is performed and the innovation strategy is adapted to these determinants; and (ii) the determinants are associated with the degree of implementation, and data on the determinants are gathered among both users and non-users.

Methods

Literature review

We searched 11 databases, mainly medical ones, for articles that were published between 1990 and 2000 and were written in English or in Dutch. We chose this time period because the tradition of innovation studies in the field of health care is quite young and we assumed that the results of earlier relevant studies would have been incorporated into the studies published between 1990 and 2000. The databases were Medline, PsycLIT, Eric, Combined Health Information Database (CHID), Healthpromis, Healthstar, Sociological Abstracts, Heclinet, Pica (a Dutch database of all university libraries), GLIN (a Dutch database on literature in the Netherlands), and SWTL (a Dutch social scientific journal on literature). We used keywords related to the specific database. Furthermore, we searched for free text, and, finally, checked the references in the studies we found. Examples of keywords are: innovation, guidelines, clinical protocols, implementation, institutionalization, change, diffusion of innovation, and health plan implementation.

Inclusion criteria were: (i) studies in which innovation processes within health care organizations were described and in which determinants were reported; (ii) studies in which the innovations were aimed at changing the behaviour of health professionals (e.g. guidelines); (iii) studies in which the health care organizations should have provided direct patient care and at least 10 professionals should have been involved in the innovation; and (iv) empirical studies only.

The first two authors independently selected the abstracts based on these criteria and retrieved the original articles. These were independently analysed by the same authors and the analyses were discussed afterwards. For the purpose of analyses, we developed a special record form based on our theoretical framework. We recorded the design of each study, the type of innovation, the respondents, the intended users of the innovation, the type of organization, the innovation strategy applied, the reported determinants, and the instruments used for measuring them. A list of potential determinants was derived from the literature [1,12,1528], and new determinants drawn from the articles were added. The determinants that finally resulted from our literature review are listed in Table 1. Furthermore, we analysed whether a study was systematically designed according to the criteria described in the section entitled Theoretical framework. Systematically designed studies were analysed individually to find out whether they generated different determinants compared with the less systematically designed studies.

View this table:
Table 1

Description of the determinants1

Delphi study

Next, a Delphi study was conducted to facilitate consensus among experts about the determinants identified in the literature review. Sixty-two Dutch implementation experts from several settings were approached using the snowball method: 44 were willing to participate. The first two authors personally contacted all experts. The main inclusion criterion was whether the expert considered himself/herself an expert in the field of innovation. The group consisted of researchers, programme managers, and implementation consultants/advisors working in public health institutes, hospitals, research institutes, and universities.

The experts were asked to decide whether a determinant was ‘impeding’, ‘facilitating’, or ‘neutral’, to clarify their responses by means of an open-ended answer, and to indicate how influential the determinant was (‘hardly’ to ‘very’). The experts had to give their answers to both extremes of a determinant, respectively (the extremes are described in Table 4); for example, ‘How influential is much support from colleagues in applying the innovation?’ and ‘How influential is low support from colleagues?’. They were also asked if they thought the determinant was adequately described. Consensus was considered adequate if 75% of the experts (including the ‘do not knows’) agreed on the influence of a determinant and on the reason(s) why the determinant was facilitating, impeding, or neutral.

There were three rounds. Feedback from the previous round was given anonymously by presenting both the group answer per determinant (percentage ‘impeding’, ‘facilitating’, ‘neutral’, or ‘do not know’), and a summary of explanations given by respondents and by the particular respondent.

View this table:
Table 4

Influence of 50 determinants according to 44 implementation experts, and number of studies in the review in which this was confirmed

Of the 44 experts who were initially willing to participate, 40 experts completed the first round, 37 the second round, and 34 the third round of consensus discussions. The main reason for non-response was lack of time. One respondent did not agree with the Delphi study method. In total, 33 experts responded to all three rounds and five experts responded to two rounds.

Results

Studies with and without systematically developed innovation strategies

In total, 2239 abstracts were collected, from which 57 studies were selected. Most abstracts (n = 1963) were excluded because no determinants were reported, or because the innovation was not aimed at changing health professional behaviour. Other abstracts were excluded because they did not focus on health care organizations (n = 30) or did not report on empirical studies (n = 189). A determinant analysis had been carried out in six studies [2934]. Although in 25 studies one or more innovation strategies were reported, none of them were linked to the outcomes of a previously conducted determinant analysis, either theoretically or empirically. In one study the strategy was based on a review of the literature [35]. Thus none of the studies met both our criteria of a systematically designed study. In 17 studies the determinants were associated with the degree of implementation, and data on determinants were gathered among both users and non-users. Therefore these studies had a partial systematic design. The 57 studies included for further analyses are described in Tables 2 (partly systematically designed studies) and 3 (non-systematically designed studies).

View this table:
Table 2

Partly systematically designed studies: no determinant analysis or innovation strategy not linked to determinants (criterion a), but determinants are related to the degree of implementation and are measured by both users and non-users (criterion b) (see Theoretical framework) (n = 17)

View this table:
Table 3

Non-systematically designed studies: no determinant analysis or innovation strategy not linked to determinants (criterion a) and determinants not related to the degree of implementation or measured by either the users or the non-users (criterion b) (see Theoretical framework) (n = 40)

Study designs

Most studies (63%, n = 36) had a cross-sectional design. The instruments used for measuring the innovation determinants were questionnaires (54%, n = 31) and interviews (44%, n = 25). There were four types of innovation: guidelines (63%, n = 36), programmes (e.g. health promotion programmes) (21%, n = 12), quality systems (7%, n = 4), or a combination of these (9%, n = 5). Most innovations focused on doctors (49%, n = 28), followed by nurses (40%, n = 23) and pharmacists (9%, n = 5). This is in line with the kind of organizations involved: hospitals (58%, n = 33), primary health care centres (16%, n = 9), and pharmacies (9%, n = 5). There was great variety in the way the degree of implementation was measured, ranging from asking management whether the innovation was used in the organization (yes/no) to daily recording per patient of the number of times each health professional had adhered to the guidelines.

Relative importance of determinants

Fifty different determinants were reported (Table 4). Except for the determinant ‘number of potential users to be reached’, all determinants were measured at least once; the average was 8.1 (range 1–32). Most determinants were characteristic of the person adopting the innovation (user), followed by characteristics of the organization, the innovation, and the socio-political context. The determinants were reported as impeding innovation 2.5 times more often (339 times) than they were reported as facilitating it (133 times) (Table 4). In only 10 out of 398 cases was a determinant judged to have a neutral effect (Table 4).

The analyses show that if a determinant was reported as facilitating the innovation (e.g. high self-efficacy), within the same study the opposite of that determinant (low self-efficacy) was nearly always reported as being impeding. In 48 out of 398 cases, a determinant was reported as being facilitating only. However, if a determinant was reported as impeding the innovation process (e.g. low self-efficacy), within the same study the opposite of that determinant (high self-efficacy) was only reported as facilitating it in one-sixth of all cases. In 256 of the 398 cases, a determinant was reported as only being impeding. These results hold true even after correction for the fact that some researchers only asked for impeding determinants, whereas others only asked for facilitating determinants.

Partly systematically versus non-systematically designed studies

Comparison of the more systematically designed studies (n = 17) with the non-systematically designed studies (n = 40) showed that fewer determinants were reported in the more systematic studies (a mean of 6.4 versus a mean of 7.3). This may be due to the fact that in 59% of the more systematic studies, the determinants were selected beforehand on theoretical or empirical grounds, whereas this occurred in only 10% of the non-systematic studies. Furthermore, the more systematically designed studies showed fewer determinants to have a neutral effect.

Delphi study

After three rounds there was consensus on nearly all 50 determinants (Table 4) and also on the reasoning behind why a determinant was impeding, facilitating, or neutral (available on request). There was no consensus on the magnitude of the effect of the determinant ‘relationship with other organizations’ (Table 1, number 12). There was also no consensus on one of the extremes of determinants 2, 14, and 18: ‘patient not aware of benefits’ (70% said this was impeding), ‘low staff turnover’ (58% said this was impeding) and ‘many people using the innovation’ (71% said this would only be impeding in case of active resistance).

Comparison review with Delphi study: relevance determinants

The experts considered nearly all determinants identified from the literature to be relevant to innovation processes (Table 4); however, there were three exceptions (Table 4, footnote 3). Firstly, the experts thought the determinant ‘organizational size’ (determinant 10) was of no influence in innovation processes because other related determinants, such as the hierarchical structure (determinant 8), were more important. Two studies reported that the size of an organization affected the innovation processes [36,85]. However, these results were contradictory: one study found large organizations as being facilitating and small organizations as being impeding [36], and the other found the opposite [85]. Secondly, in five studies a strong inter-organizational network was reported to be a relevant determinant of innovation processes (determinant 12), but the experts did not reach consensus on this determinant. Thirdly, the determinant ‘number of potential users to be reached’ (determinant 18) was not identified in the literature review, but was added by the experts as being a relevant determinant in innovation processes.

Comparison review with Delphi study: direction of influence determinants

When comparing the direction of influence (highly impeding versus highly facilitating or neutral), the results of the review generally matched the results of the Delphi study. A determinant identified as being a facilitating factor in the literature review was also judged so by the experts, with some exceptions (Table 4, footnote 2). In 10 studies a determinant was reported as being neutral, but other studies and the experts did not confirm this. For example, from the 32 studies that measured the influence of ownership (determinant 27), only two studies reported that this determinant was of no influence. The extremes ‘patient has no doubts about health professional’s expertise’ (determinant 3), ‘patient has no discomfort’ (determinant 5), and ‘the health professional does not suffer from work-related stress’ (determinant 31) were reported in a few studies to be of influence, whereas the experts thought they were not. Furthermore, patient lack of awareness of the potential benefit of an innovation (‘not aware of the benefits’) was considered an impediment in three studies, but not by the experts, only 70% of whom thought it was an impediment.

Discussion

A first conclusion is that the innovation studies retrieved in our literature review did not have a systematic design. None of the 57 studies met the first criterion of having conducted a determinant analysis beforehand and of applying the results to the innovation strategy. Although, in many studies, one or more innovation strategies were applied, none were based on a theory (theoretical methods for change). This is surprising because such analyses are considered important and can help avoid the use of inappropriate, and thus ineffective, strategies, and hence save time and money. Furthermore, two-thirds of the studies did not meet the second criterion (associated the determinants to the degree of implementation and having gathered data on determinants among both users and non-users). The consequence of not systematically designing an innovation strategy is that the intended change might fail.

We can only speculate on the reason why we found so few well designed innovation studies. Implementation research in health care is still in its infancy and there are few innovation theories. Moreover, empirical studies mainly consist of case studies and there are few standardized procedures for measuring determinants as well as the stages of change (from dissemination, adoption, and implementation to continuation). Another possible reason is that the diffusion of good theories and studies is less widespread than thought. A consequence of finding only a few well designed studies is that we were unable to compare the determinants found in systematically designed studies with those found in non-systematically designed studies.

A second conclusion is that many of the innovation studies showed methodological flaws. Besides the already above-mentioned fact that determinants were not related to the degree of implementation, the degree of implementation was assessed in different ways, such as level of use (non-use, full use, adapted use), completeness of use (applied proportion of recommended activities), frequency of use (number of times used), intensity of use (number of people who use innovation), and duration of use. This means that the degree of implementation and the association with particular determinants depend on the operationalization of implementation. If, for example, we were to define ‘a smoker’ as someone who had smoked at least one cigarette during the past year (rather than, for example, as someone who had smoked seven cigarettes or more daily), we would not only find more ‘smokers’, but also different determinants of the smoking behaviour.

Despite the above-mentioned limitations of the studies reviewed, the determinants identified by the literature review and their effect (impeding, enhancing, or neutral) were consistent with the opinion of the experts. For example, when the literature review showed that ‘much support from colleagues’ was facilitating, the experts confirmed this. Fifty potential determinants were identified. We use the word ‘potential’ because there were discrepancies between the literature and Delphi studies that could be due to the non-systematic design of the included studies, and also because of the fact that the expert opinion was subjective and may not be empirically valid.

The literature review identified more determinants that impeded rather than facilitated innovation, even after correction for the fact that some studies investigated only facilitating or only impeding determinants. This could have been caused by the inclusion of more studies with negative or inconclusive outcomes of the innovation; however, because many studies did not report how successful the innovation was, we cannot determine whether this was the case. Furthermore, it can be stated that if a determinant is measured, the determinant shows up to be relevant for the innovation process. The more or less systematically designed studies yielded fewer determinants than the non-systematically designed studies, as we had anticipated (see the introduction to this study). Most identified determinants were related to the individual user. However, this does not necessarily mean that these were the most important determinants, because we also found that if a determinant is measured it will show up to be relevant for the innovation process. The determinant ‘self-efficacy’, for example, was measured mainly in studies on health promotion. This suggests that the outcome of a particular study on innovation determinants is liable to selection bias on the part of the researchers.

Our study had some limitations. Although our database search was extensive, we may have overlooked one or more relevant studies, in particular those published in internal or governmental reports. Another limitation is that the Delphi group consisted of experts (academic and practitioner) from three different professional disciplines. Their familiarity, or not, with current opinion, as expressed in the published literature, could have influenced the agreement between the determinants identified from the literature and the expert opinion, producing more apparent agreement than there was in reality. Moreover, we do not know whether the experts who dropped out of the Delphi study agreed or not with the other experts, thus potentially influencing the degree of agreement established. Furthermore, it is not possible to rank the 50 determinants in order of importance because many determinants may have been related to the type of innovation studied and to the context in which the innovation was introduced. For example, ‘observability’ may be a greater impediment in a public health setting than in an emergency department setting. Finally, the interrelation of the 50 determinants is unclear: given the presence of determinant x, determinant y may lose its importance.

Despite these criticisms, we feel encouraged by the participants of the Delphi study who said the 50 potential determinants provide a good starting point for developing a measurement instrument or can function as a checklist, if reduced to ±10 main categories, for daily innovation practice.

On the basis of these conclusions and critical reflections, we suggest that the relative impact of the 50 determinants of innovation processes be evaluated in an empirical study. We would like to invite implementation researchers and programme managers to explore this list of determinants further, and to report their results. In the future, the quality of innovation studies should be improved by systematically designing strategies that are tailored to an empirically based selection of innovation determinants, and by asking both users and non-users why they accepted or rejected the innovation. Moreover, we recommend that researchers look more closely at the procedures used to measure the degree of implementation of an innovation. We believe the first step would be to describe systematically why a certain way of assessing the degree of implementation was chosen and what the implications are for the reported results.

References

View Abstract