Skip Navigation


International Journal for Quality in Health Care Advance Access originally published online on March 23, 2005
International Journal for Quality in Health Care 2005 17(3):203-208; doi:10.1093/intqhc/mzi031
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
17/3/203    most recent
mzi031v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (3)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Roeg, D.
Right arrow Articles by Garretsen, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Roeg, D.
Right arrow Articles by Garretsen, H.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

International Journal for Quality in Health Care vol. 17 no. 3 © The Author 2005. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved

Towards quality indicators for assertive outreach programmes for severely impaired substance abusers: concept mapping with Dutch experts

Diana Roeg1, Ien van de Goor1 and Henk Garretsen1,2

1 Tranzo, Tilburg University, The Netherlands and 2 Addiction Research Institute Rotterdam, Erasmus University Rotterdam, The Netherlands

Objective. We investigated the concept of ‘quality of assertive outreach programmes for severely impaired substance abusers’ with the aim of developing a conceptual framework as the basis for an assessment instrument.

Design. We held a concept-mapping session with 13 experts in 2003. Fifty measurable elements of quality were mentioned and rated in terms of relative importance on a Likert-type response scale. Subsequently, the experts grouped the statements that were similar in content. The resulting concept map and additional interpretation made up the final quality framework.

Setting/study participants. Theoretical sampling was used to select Dutch managers, team leaders, and service providers from different assertive outreach delivery systems for substance abusers. Variation in both perspective and region was reflected in the sample.

Results. Nine aspects of quality were formulated: preconditions for care, preconditions for service providers’ work, relationship to regular care, service providers’ activities and goals, service providers’ skills, the role of repression, optimal care for the client, goals of assertive outreach, and nuisance reduction to society. Each aspect was presented using a selection of measurable elements.

Conclusions. According to the experts, optimal assertive outreach depends on a broad range of aspects that were later classified in three regions: structure, process, and outcomes. Saturation of the elements has not been proved so far. Nevertheless, it is promising that the framework’s regions are supported by theory and that it is largely in accordance with clients’ perspectives on assertive community treatment.

Keywords: assertive community treatment, assertive outreach, concept mapping, dual diagnosis, quality, substance abusers

Address reprint requests to Diana Roeg, Department Tranzo, Tilburg University, Warandelaan 2, 5037 AB Tilburg, The Netherlands. E-mail: d.p.k.roeg{at}uvt.nl

Accepted for publication February 3, 2005.

At the so-called Leidschendam conferences, which have been held regularly since 1990, providers, clients, and insurers make agreements regarding the quality assurance of health care in the Netherlands. At the 2000 conference, it was stated that more transparency was necessary to improve the external accountability towards third parties concerning the quality of care, and to improve the internal management of health care processes. Harteloh [1] points out that quality does not exist as such: ‘It is constructed in an interaction between people’ (p. 259). It requires the selection of relevant aspects of care and indicators must be defined to make it measurable. In general health care, evidence-based knowledge or protocols are often used for this purpose [2,3]. For new and evolving types of health care, however, this is impossible as standards are usually lacking. An alternative method of defining quality indicators is necessary in such cases.

Assertive outreach is a form of care developed internationally since the socialization of psychiatric patients became an important issue of debate [4–6]. This type of care was originally aimed at seriously mentally ill persons with psychotic disorders, but owing to the increasing numbers of chaotic substance abusers, it is also made available to this group. A number of elements are shared by different assertive outreach programmes: the need felt by service providers and other stakeholders to reach people that are not receiving treatment; community-based care (people receive treatment in their own environment); an assertive approach; and a broad package of interdisciplinary care with an emphasis on practical help [4,5,7–9]. The only available standard for this type of care (the programme for assertive community treatment) is the subject of international debate. The programme has proved effective in the USA, but has not had similar effects in Europe [10,11]. Controversy exists about whether this reflects the content of the programme (some authors argue that it lacks theoretical base) or the context in which it operates [12–14]. Because of this discussion, no uniform model is used for assertive outreach programmes in Europe. Dutch programmes, for instance, differ in target group, network partners, strategy, institutional integration, and methodologies used. As a result, they may also differ in performance [14–17]. The quality of these programmes must be evaluated to determine whether assertive outreach is successful. It is also necessary to improve our understanding of the relationship between specific programme features and effectiveness. As part of a larger study into assertive outreach for marginalized substance abusers, two relevant concepts were explored using the method of concept mapping developed by Trochim [18]. The first concept-mapping session focused on the quality of assertive outreach and the second, in which a different expert panel was used, on the key organizational features of assertive outreach. We examined the first concept map with the aim of developing a conceptual framework as the basis for an instrument for quality assessment.


   Methods
Top
 Methods
Results
 Discussion
 Conclusions
 References
 
Concept mapping
Concept mapping is a standardized tool for the conceptualization of a specific subject. It is often used when a problem area is in an exploratory stage and when there is a lack of existing theory or predetermined categories [18–20]. An inductive approach is used in concept mapping: experts are invited to share their implicit knowledge on the subject in a structured group process. It is recommended that between 8 and 15 respondents participate in the group process [21]. In our study, theoretical sampling was used to select respondents (n = 13). Theoretical sampling is a much-used sampling strategy in qualitative research [22,23]. It is aimed at the structured selection of respondents based on characteristics that are expected to influence the type of statements made on the topic. Except in quantitative research, the sampling aims at saturation of all the concept dimensions rather than population representativeness.

As quality was the primary concept and different perspectives can influence the range of statements, managers, team leaders, service providers, clients, and client representatives were invited to participate in the concept-mapping session. As there is much interregional variance in assertive outreach programmes, it is possible that people from different regions have different points of view concerning aspects of quality. Therefore, a conscious effort was made to include programmes from different regions (Table 1). Unfortunately, no clients or client representatives could be present at the actual session. The concept map was completed in a 5-hour session in September 2003. The session took place in Utrecht, as it is centrally located in the Netherlands.


View this table:
[in this window]
[in a new window]
  		Table 1 Participants (n = 13)1

 

Brainstorming and structuring
The aim and background of the meeting, as well as the focus of the concept-mapping process, were explained during the introduction. In addition, the working definition of assertive outreach for substance abusers was presented, including the shared elements of assertive outreach mentioned above. The participants were asked to complete the following assertion in a brainstorming session: ‘Optimal assertive outreach for people with addiction problems is provided when . . .’. In this part of the session, each participant was free to come up with whatever he or she considered relevant. The statements needed to be understood by everyone, but no criticism was allowed [20]. The group came up with 50 statements in 45 minutes.

All statements were entered directly into the computer, using the Ariadne program for Concept Mapping [21]. The statements were printed on paper, and the participants were asked to structure them. Structuring involves two distinct tasks: rating and sorting [18]. For the rating task, each participant was asked to judge the statements on a 5-point Likert-type response scale in terms of how important the statement was to his or her idea of the quality of assertive outreach (1 = least important and 5 = most important). The statements needed to be more or less equally distributed to prevent the respondents from classifying all statements as equally important (a commonly observed tendency with this method) [18]. For the sorting task, each participant was instructed to group the statements ‘in a way that made sense to him or her’. Any number of groups (between 2 and 50) was permitted, except for a group with leftover items, as this can influence the final visualization. Leftover items were placed in individual groups [18].

Analyses
The individual rating and sorting data were entered into the computer and combined. The data were analysed in two steps. First, the sorting results for each person were placed in an N x N matrix that had as many rows and columns as there were statements. A ‘1’ was placed in a cell when two statements were grouped together by a person; a ‘0’ when they were not. All the individual matrices were added together to obtain a group matrix that was used as input for principal component analyses. Using these analyses, so-called imaginary distances between statements were calculated, which made it possible to plot the statements as points on the two-dimensional concept map. In the concept-mapping method, it is usual to limit the solution to two dimensions for reasons of usability, as argued by Trochim [20]. The more often statements were grouped together by individual participants, the smaller the imaginary distance between the statements on the map. Second, the positioned statements were partioned into clusters (Figure 1). This was done using the X–Y coordinate values for each point as input for cluster analysis [18], starting with 50 clusters and categorizing until the clusters no longer made sense conceptually [19]. We decided to use the nine-cluster solution. In spite of the quantitative analyses, no large sample was required because of the inductive (observational) nature of the analyses. They were used for conceptual exploration and the results, therefore, were just as valuable with a small sample [22–24]. Finally, the mean ratings of both the statements and the clusters were calculated (Table 2). The concept map was shown to the participants and explained. The participants discussed the logic of the clustering and formulated labels that would cover the content of the final clusters as well as possible [21].



View larger version (18K):
[in this window]
[in a new window]
  		Figure 1 Final concept map of quality of assertive outreach. Statements are not reproduced as the figure is illustrative. The original concept map can be obtained from the author. The clusters are represented by the rectangles.

 

View this table:
[in this window]
[in a new window]
  		Table 2 Framework of aspects of quality and measurable elements

 


   Results
Top
 Methods
 Results
Discussion
 Conclusions
 References
 
After the group process was finished, the researchers interpreted the map in more detail. The use of two dimensions made it possible to identify opposing issues. The vertical axis was found to represent, from top to bottom, a scale from ‘client’ to ‘service provider’, and the horizontal axis, from left to right, ‘providing’ and ‘organizational preconditions’.

The statement ‘Quality of life is maintained’ was plotted in the left-most quadrant and related to desired outcomes for the client during or after the care-provision process. The statement ‘A safety net is created in a network of institutions’ was plotted across both quadrants on the right and concerned a structural element that had to do with both the client and the service provider. With this knowledge, we aimed to identify interpretable groups of clusters (called ‘regions’ by Trochim [20]). This led to the following classification: structure, process, and outcomes. For every cluster (or aspect of quality) the statements with an importance rate of 2.50 and higher were selected as measurable elements of that aspect. The mean ratings revealed the clusters ‘service providers’ activities’ and ‘optimal care for the client’ to be the most important aspects of care in relation to quality. The formation of an interinstitutional safety net was found to be the most important aspect for the structure of assertive outreach; making contact, fulfilling the necessities of life, and maintaining the active and persistent approach were the main elements of the process; and improving the quality of life was considered to be the ultimate outcome.


   Discussion
Top
 Methods
 Results
 Discussion
Conclusions
 References
 
Despite the broad range of statements, caution is required in generalizing the results. Although theoretical sampling was used, the results nevertheless represent the opinions of a small group of individuals, and clients were not represented. It is possible that other experts would come up with other aspects or elements of quality. Furthermore, this exploration uncovered diversity in opinion as some elements seemed to contradict or complement each other. It is important to get a complete overview of all relevant dimensions of the concept of quality (saturation [22,23]) before defining indicators for this type of care. To expand the preliminary conceptual framework, additional interviews and comparisons with available theory on assertive community treatment and other models [5,25] were performed in a follow-up study.

In order to determine whether specific issues were overlooked owing to the absence of clients during the concept-mapping session, the results were compared with those of a study of clients’ perspectives on helpful ingredients of assertive community treatment [26]. Although the comparison was not exactly parallel because of possible content or context differences, it was helpful in revealing possible gaps in our framework. Most things mentioned by the clients were also present in our framework (e.g. availability of staff, problem-solving support). Clients were more specific about the content of treatment than were the experts in our session. While the elements in our framework were somewhat general (e.g. ‘Service provider is open to all kinds of demands’, ‘Service provider is willing to tailor care to client’s needs’), the clients mentioned specific activities, such as money management and improving general life skills. Two important ingredients mentioned by clients were absent in our framework: the personal bond between service provider and client, and the reduction of the number of days spent in hospital. The latter is most probably context dependent. In the USA, the reduction of the number of hospitalization days is the most important aim of assertive outreach programmes, while in the Netherlands, the most important goal is reaching drug abusers who are not receiving treatment [15].

Note that the traditional quality assessment triad of structure, process, and outcomes is reflected in the three regions of the framework. Donabedian [27] argues that these are the three major approaches in assessing the quality of care. The most direct route is examination of the care itself. He defines the process of care as ‘the primary object of assessment’, which consists in his opinion of a set of actions that go on between practitioners and patients. Two other, less direct approaches to assessment are the measurement of structure and the measurement of outcomes. Donabedian [27] defines structure as the relatively stable characteristics of the providers of care, of the tools and resources they have at their disposal, and of the physical and organizational settings in which they work. Structure increases or decreases the probability of good performance (quality). Outcomes are defined as the direct consequences of the care for the health and welfare of individuals and of society: the most immediately discernible attributes of that care. The apparent fit with such a traditional and leading theoretical approach strengthens our framework’s content validity [28].

There is a fundamental and chronological relationship between structure, process, and outcomes. Mant [3] argues that process measures can only be used as direct measures for quality of care provided a link has been demonstrated between a given process and its outcome. In other words, process measures can only be used when the process itself has been shown to be effective. The advantage of using outcome indicators is that these are the only measures that reflect all aspects of care, including technical expertise and operational skills, which are difficult to measure in other ways [3]. As stated above, assertive outreach programmes are still evolving. The exact relationship between their structural elements, performance, and outcomes is still unclear. Therefore, outcome indicators appear to be the most appropriate measurements for quality assessment at present.


   Conclusions
Top
 Methods
 Results
 Discussion
 Conclusions
References
 
This concept-mapping session was part of a systematic, inductive exploration of the concept of quality of assertive outreach for marginalized substance abusers. Our main goal was to make the concept concrete and to contribute to making it operational for assessment. The software used enabled the structuring of the data to be carried out according to the respondents’ logics. The result was the selection by experts of nine aspects of and subordinate elements of quality, organized into a preliminary conceptual framework. According to the participants, optimal assertive outreach depends on a broad range of aspects; these were later classified in three regions: structure, process, and outcomes. Firstly, some structural preconditions, such as good cooperation between institutes and availability of necessary facilities, must be met. As multiple disciplines are involved, all stakeholders have responsibilities. Secondly, relational and functional aspects of the process were mentioned as indicative of the quality of care. The service provider should be competent to provide good care. Willingness to tailor care to the client’s needs and activities, and willingness to integrate different services seem to be more important than technical skills alone. Thirdly, the client should benefit from the care, achieving a situation where life is as normal as possible. This should also result in the reduction of nuisance to society, and relieve family and friends of some of their burden.

This framework is preliminary, as saturation cannot yet be proved and the measurable elements are still the subject of discussion. Nevertheless, it is promising that the findings of the study are largely in accordance with clients’ perspectives on assertive community treatment. In order to further elaborate the framework, clients and providers (from parts of the country and assertive outreach programmes other than those included in this concept-mapping session) were interviewed in a follow-up study. As all regions following from general quality of care theory seem to be present in the framework, the primary focus in future research should be to find additional aspects and elements to fill in these regions.

According to the theory discussed above, structure and process indicators can only be used when their relationship to the outcome of care is proved. Since assertive outreach is an evolving practice, in which the link between specific process characteristics and outcomes is still the subject of international debate, outcome measures seem to be the most appropriate indicators for the quality of care at present.


   References
Top
 Methods
 Results
 Discussion
 Conclusions
 References
 

  1. Harteloh PPM. The meaning of quality in health care: A conceptual analysis. Health Care Anal 2003; 11: 259–267.[CrossRef][Web of Science][Medline]

  2. Wensing M, van der Weijden R, Grol R. Implementing guidelines and innovations in general practice: Which interventions are effective? Brit J Gen Pract 1998; 48: 991–997.

  3. Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001; 13: 475–480.[Abstract/Free Full Text]

  4. Lachance KR, Santos AB. Modifying the PACT model—preserving critical elements. Psychiatr Serv 1995; 46: 601–604.[Abstract/Free Full Text]

  5. Kroon H. Growing Care: The Development of Case Management in the Health Care for Chronically Psychiatric Patients (in Dutch). Utrecht: NcGv, 1996.

  6. Wingerson D, Ries RK. Assertive community treatment for patients with chronic and severe mental illness who abuse drugs. J Psychoactive Drugs 1999; 31: 13–18.[Web of Science][Medline]

  7. Henselmans HWJ. Assertive Outreach: Uncalled-for Help for Psychotic Patients (in Dutch). Utrecht: Delft, 1993.

  8. Deci PA. Dissemination of assertive community treatment programs. Psychiatr Serv 1995; 46: 676–678.[Abstract/Free Full Text]

  9. Test MA, Stein LI. Practical guidelines for the community treatment of markedly impaired patients. Community Ment Health J 2000; 36: 47–60.[CrossRef]

  10. Phillips SD, Burns BJ, Edgar ER et al. Moving assertive community treatment into standard practice. Psychiatr Serv 2001; 52: 771–779.[Abstract/Free Full Text]

  11. McHugo GJ, Drake RE, Teague GB. Fidelity to assertive community treatment and consumer outcomes in the New Hampshire dual disorders study. Psychiatr Serv 1999; 50: 818–824.[Abstract/Free Full Text]

  12. Burns BJ, Catty J. Assertive community treatment in the UK. Psychiatr Serv 2002; 53: 630–631.[Free Full Text]

  13. Burns T. Case management and assertive community treatment in Europe. Psychiatr Serv 2001; 52: 631–636.[Abstract/Free Full Text]

  14. Vanderplasschen W, Rapp RC, Wolf JR, Broekaert E. The development and implementation of case management for substance use disorders in North America and Europe. Psychiatr Serv 2004; 55: 913–922.[Abstract/Free Full Text]

  15. Roeg DPK, Goor LAM, Garretsen HFL. When a push is not a shove: assertive care, Dutch-style. Drugs and Alcohol Today 2004; 4: 26–32.

  16. Bransen E, Hulsbosch L, Wolf J. Cooperation Projects in Public Mental Health Care for Socially Vulnerable Individuals (in Dutch). Utrecht: Trimbos-instituut, 2002.

  17. Wolf J, Planije M, Thuijls M. Case management for chronic substance abusers with multiple problems. In: Rooijen Sv, Gaag Mvd, Kroon H, Veldhuizen Rv, eds, We Will be There! Home-based Care for People with Severe Mental Problems (in Dutch). Amsterdam: SWP, 2003.

  18. Trochim WMK. An introduction to concept mapping for planning and evaluation. Eval Program Plann 1989; 12: 1–16.

  19. Johnsen JA, Biegel DE, Shafran R. Concept mapping in mental health: Uses and adaptations. Eval Program Plann 1999; 23: 67–75.[CrossRef]

  20. Trochim WMK, Cook JA, Setze RJ. Using concept mapping to develop a conceptual framework of staff’s views of a supported employment program for individuals with severe mental illness. J Consult Clin Psychol 1994; 62: 766–775.[CrossRef][Web of Science][Medline]

  21. NcGv/Talcott. Handbook of Concept Mapping with Ariadne (in Dutch). Utrecht: NcGv/Talcott, 1995.

  22. Bowling A. Research Methods in Health: Investigating Health and Health Services. Philidelphia: Open University Press, 2000.

  23. Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, London, New Delhi: Sage Publications, 1990.

  24. Loehlin JC. Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis. 4th edn. Mahwah, N.J., Lawrence Erlbaum Associates, 2004.

  25. Teague GB, Bond GR, Drake RE. Program fidelity in assertive community treatment: Development and use of a measure. Am J Orthopsychiatry 1998; 68: 216–232.[Web of Science][Medline]

  26. McGrew JH. Client perspectives on helpful ingredients of assertive community treatment. Psychiatr Rehabil J 1996; 19: 13–22.

  27. Donabedian A. The Definition of Quality and Approaches to its Assessment. Michigan: Health Administration Press, 1980.

  28. Zeller RA, Carmines EG. Measurement in the Social Sciences: The Link between Theory and Data. New York: Cambridge University Press, 1980.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Int J Qual Health CareHome page
R. W. Gibberd
Performance measurement: is it now more scientific?
Int. J. Qual. Health Care, June 1, 2005; 17(3): 185 - 186.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
17/3/203    most recent
mzi031v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (3)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Roeg, D.
Right arrow Articles by Garretsen, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Roeg, D.
Right arrow Articles by Garretsen, H.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?