International Journal for Quality in Health Care Advance Access originally published online on September 7, 2006
International Journal for Quality in Health Care 2006 18(6):397-402; doi:10.1093/intqhc/mzl044
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Effects of visitation among allied health professionals
1 Netherlands Institute for Health Services Research, Utrecht and 2 Department of Pediatric Oncology-Hematology, University Medical Centre Nijmegen, Nijmegen, The Netherlands
Objective. Visitation is a method for external peer review. The goal is to improve the quality of patient care by giving feedback on quality of competence and performance of a professional during a practice visit. Feedback is offered as recommendations for improvement. This study aims to evaluate the effects of visitation and to determine which factors are related to the effectiveness of visitation.
Participants. Members of seven allied health professions in the Netherlands: dieticians, exercise therapists, physiotherapists, dental hygienists, occupational therapists, podiatrists, and radiology assistants.
Design. Evaluation questionnaires were sent to 151 allied health professionals who had participated in visitation. The questions included all practice management aspects that had been assessed during the practice visit. The effects of visitation were studied at three levels: change in awareness of weak and strong aspects of competence and performance, intention to carry out recommendations, and actual improvements.
Results. Results showed effects of visitation on all three levels. Respondents intended to carry out two-thirds of the recommendations. Visitation led to a better awareness of weak points on 36% of the aspects and better awareness of strong points on 53% of the aspects of practice management. Young respondents reported more changes in awareness than older respondents. Actual improvements were carried out on 33% of the aspects.
Conclusions. Visitation is an effective method to stimulate quality improvement in allied health professionals. Although changes in awareness more often occurred in younger respondents, actual improvements were made by all respondents.
Keywords: peer review, quality assessment, quality improvement, visitation
Address reprint requests to Hannelore Hofhuis, Netherlands Institute for Health Services Research, Utrecht, The Netherlands. E-mail: h.hofhuis{at}nivel.nl
Accepted for publication August 14, 2006.
Visitation is a method for external peer review. The aim is to improve the quality of patient care by measuring the quality of competence and performance of the professional participating in visitation. The professional can use the information about the quality of competence and performance to implement changes to improve patient care. Quality assessment is carried out by a group of peers in the practice facilities of the professional during a practice visit. The assessment is based on objective criteria such as standards of good quality of care and professional guidelines. The recommendations of the visiting team are confidential and communicated to the professional in writing [1]. The method of visitation is often used in combination with other measures to improve the quality of patient care such as medical education, peer review groups, re-licensing, audit, and accreditation. Although visitation shares common ground with accreditation, certificates of achievement are not awarded.
Visitation is used as a means of improving quality of care in general practice and other medical professions in countries like New Zealand and the Netherlands. Visitation-based quality improvement systems have also spread to the UK, Sweden, and Finland; other countries have shown an interest in the method [2,3].
In the Netherlands, practice visits have also been implemented in the allied health sector [4]. About 35000 allied health professionals work in primary care and in institutional care. Physical therapy is the largest profession with more than 18000 members. Other allied health professions that have implemented practice visits are dieticians, dental hygienists, exercise therapists, occupational therapists, podiatrists, and radiology assistants. In comparison, the group of general practitioners in the Netherlands consists of 8000 members.
Evidence of the effectiveness of visitation is scarce. The effectiveness has been studied for general practitioners and medical specialists [5,6]. Visitation proved to be successful in inducing improvements in general practitioner equipment, delegation, organization of information, and record keeping. In another study, the effect of supporting medical specialists with the implementation of recommendations was evaluated. This study showed a positive effect on the development of management of medical specialist care [7]. There are no other reports on the effectiveness of visitation. In addition, there is no information on determinants of the effectiveness of this method. The care allied health professionals deliver differs from the general practitioners and specialists care. Information about the effectiveness in one profession can therefore not be applied directly to other professions. More research in other professions such as the allied health professions is therefore essential.
The aim of this study was to assess the effectiveness of visitation among allied health professionals in the Netherlands and to determine which factors are related to the effectiveness of visitation. The Dutch National Institute for Quality Improvement (CBO) was responsible for the development and implementation of the visitation method. For each profession, the visitation was adapted to the specific work settings and care. The practice or department visits were primarily focused at management aspects. Visits concerned individual professionals. Visits were carried out between 1997 and 2001.
Visitation for allied health professionals in the Netherlands is characterized by voluntary participation. The implementation of the recommendations of the visiting team is not compulsory. The first goal is increased insight in performance, and the second goal is the implementation of actual improvements. The realization of both the goals will be measured in this study analogous to the process of behaviour change: change of awareness, intention to change, and actual behaviour change [8,9]. The evaluation of the effectiveness of visitation was carried out by an independent research institute (NIVEL). Self-reported effects were assessed by mail questionnaires.
 |
Method |
|---|
 Top Method ResultsDiscussionReferences |
|---|
Approximately 6 months after the practice visit, an evaluation questionnaire was sent to every allied health professional who participated in visitation during the study period. A total of 185 professionals participated; 151 questionnaires could be sent within the time schedule of the evaluation study. The period of 6 months was adopted to be able to observe effects of visitation. The evaluation questionnaire covered the following respondent characteristics: sex, age, work setting, and reason to participate.
The questions about visitation covered three topics:
- Awareness/increased insight. This concept was operationalized as self-reported change of awareness of weak and strong aspects of practice management. In the evaluation questionnaires, a list of aspects of practice management was presented corresponding to the aspects assessed during the practice visit. These lists vary because some aspects are not applicable to all professions. Examples of aspects are organization of practice management, quality policy activities, continuing education, accessibility of the practice facility, collaboration with colleagues and general practitioners, record keeping, care process, evaluation of care, patient education, and hygiene. The number of aspects in the evaluation questionnaires varies between 11 for dieticians and 14 for physiotherapists. For each aspect, the respondent was asked to indicate whether visitation had led to a better awareness of weak points of that aspect and a better awareness of strong points. The questions were answered on a three-point scale (yes, somewhat, and no). Because the number of aspects differs between groups, the dependent variable is expressed as percentage: (number of aspects with increased insight)/(total number of aspects) x 100.
- Intention to change. The number of recommendations for improvement, opinion of the allied health professional about this number, the number of months between the practice visit and the reception of the findings of the visiting team, the number of recommendations that the professional intended to execute, and the number of recommendations that the professional intended to reject. No questions were asked about the content of the recommendations to respect the confidentiality of the practice visit.
- Behaviour change. For each aspect of practice management, the respondent was asked whether actual improvements had been made. The questions were answered on a five-point scale (improvements already made, engaged in improving, will be improving shortly/within 6 months, will be improving within 2 years, and no plans for improvement). The dependent variable is also expressed as percentage: (number of improved aspects)/(total number of aspects) x 100.
Recommendations for improvement are often elaborate explanations in which several aspects of practice management are interrelated. In addition, recommendations are personal and highly confidential. It was estimated that asking respondents for information about the content of the recommendations is not appropriate and would not yield reliable information. Therefore, only superficial questions were asked about the recommendations. To measure effects of visitation, we asked questions about all aspects of practice management, which are examined during the practice visit.
The results of the seven allied health professions were calculated and analysed separately to show differences between groups and in combination to assess the overall effects of visitation. The analysis of the determinants of visitation focused on the effect of profession, age, sex, the number of months between the practice visit, and the reception of feedback and the number of recommendations. Number of months was dichotomized to within 3 versus 4 months or more.
The data were analysed using the SPSS statistical package. Descriptive statistics were used to describe frequencies and percentages. Stepwise regression analysis was used to determine which factors are related to the effectiveness of visitation.
|
Results |
|---|
|
TopMethodResultsDiscussionReferences |
|---|
Evaluation questionnaires were sent to 151 allied health professionals. The response after one reminder was 73% (n = 110): 15 dieticians, 26 exercise therapists, 10 physiotherapists, 17 dental hygienists, five occupational therapists, 26 podiatrists, and 11 radiology assistants.
Respondents were asked to indicate reasons for participating in visitation; more than one reason could be given (five answering categories). Respondent characteristics and reasons to participate are summarized in Table 1.
|
About three-quarters of the participating allied health professionals were female. Among podiatrists and physiotherapists, there were relatively more male professionals than among the other professions. These figures reflect the actual situation in the Netherlands, where a large part of the allied health professions consist of females. The mean age of the respondents varied between 35 and 46 years.
The reason to participate, mentioned most often, was that the respondent wanted to learn to improve practice management aspects (79%). Fifty-five per cent of the respondents participated to get confirmation about their standard of practice management. Reasons that were mentioned less often were the improvement of the image of the profession (32%), and the opportunity to show external parties such as general practitioners, colleagues, and insurance companies their quality improvement efforts.
In Table 2, the results on change of awareness are summarized. Respondents rated all aspects of practice management answering the questions ‘Have you become more aware of weak/strong aspects of your practice policy as a result of the visitation, concerning one or more of the following practice management topics?’ (yes, somewhat, and no). The figures in the table represent percentages: for each respondent, the number of answers in an answering category (categories yes and somewhat are taken together) was added and divided by the total number of aspects. Subsequently, the mean of the proportions of all respondents in a profession was calculated and multiplied by 100 to obtain percentage scores.
|
Overall, respondents experienced an increased awareness of weakness on 36% of the aspects and an increased awareness of strength on 53% of the aspects. The increase of awareness varied between 27% for dieticians and 57% for occupational therapists. Of all respondents, only 7% were not aware of any weak points. Both stepwise regression analyses showed a significant effect for age. Younger respondents report increased awareness of weak and strong aspects more often than older respondents (P = 0.003 and 0.002). Profession, sex, number of months between the practice visit and the written report, and number of recommendations did not show a significant effect on the dependent variables.
Seventy-five per cent of all respondents received written feedback within 3 months; 25% received feedback within 4–7 months (all respondents who participated in this evaluation study returned the questionnaire after they had received feedback). Feedback concerned the findings of the visiting team on all aspects of practice management, conclusions, and recommendations for improvement. Respondents were asked whether they intended to execute or reject the recommendations (intention to change) (Table 3).
|
The mean number of recommendations varied between 7.3 for exercise therapists and 11.4 for dental hygienists (the lowest reported number was 1 and the highest number 19, both from podiatrists). The respondents intended to execute most of the recommendations. Analysis of variance was used to test differences between groups. Respondents intended to execute two-thirds of the recommendations (5.7/8.6); the proportions did not differ significantly between the groups. One-fifth of the recommendations was rejected; these proportions did not differ between the groups either.
In Table 4, the results on self-reported actual improvements are summarized. The figures in the table represent percentages: for each respondent, the number of answers in an answering category was added and divided by the total number of aspects. Subsequently, the mean of the proportions of all respondents in a profession was calculated and multiplied by 100 to obtain percentage scores.
|
Respondents already realized improvements concerning 11% of practice management aspects and were engaged in improving 21% of the aspects; together, this was one-third of all aspects of practice management. Plans for improvement were made for 14% of the aspects; no plans were made for 52% of practice management aspects. For the regression analysis, two new dependent variables were constructed: the categories ‘already realized’ and ‘engaged’ were taken together for the dependent variable ‘started improvements’. The categories ‘start within 6 months’ and ‘start within 2 years’ were taken together for the dependent variable ‘plans for improvement’.
The regression analyses for ‘started improvements’ and for ‘plans for improvements’ did not show any significant effects of profession, age, sex, number of months between the practice visit and the written report, and number of recommendations. The regression analysis for ‘no plans for improvement’ showed a significant effect for number of months between the practice visit and the written report (P = 0.04).
Respondents who received the report at least 4 months after the practice visit reported having no plans for improvement more often than respondents who received the report within 3 months after the practice visit. There were no significant effects of profession, age, sex, and number of recommendations.
As an example of the kind of improvements that were made or planned, the improvements made by dental hygienists are summarized in Table 5. For each practice management aspect, it is indicated how many dental hygienists have already made or are engaged in improvements, how many dental hygienists have plans for improvements, and how many do not have plans for improvement.
|
For dental hygienists, the list of practice managements consists of 13 aspects. Improvements are already made or are in the process of being made for all aspects. Record keeping, organizational aspects, and evaluation of care most often show rapid improvements. When dental hygienists intend to change, they do so shortly after the practice visit. Table 5 summarizes that there are hardly any plans for improvement within 6 months or within 2 years. Most dental hygienists do not have plans for improving patient education, undertaking safety measures, or improving the accessibility of the practice facilities for disabled patients.
|
Discussion |
|---|
|
TopMethodResultsDiscussionReferences |
|---|
In this study, the self-reported effects of visitation were studied among a group of allied health professionals. Most of these professionals participated to learn how to improve practice management and to get confirmation about their management standard.
An increase in awareness of weak and strong aspects of practice management is the first goal of visitation. This is an important effect that may lead to actual improvements. Increased awareness of weaknesses occurred in one-third of the aspects. An increased awareness of aspects that were already strongly developed occurred in more than half of all the aspects. This effect may be particularly relevant for respondents who participated to receive confirmation about their standard of management.
All professionals received written feedback after the practice visit with recommendations for improvement. The mean number of recommendations was 8.6; respondents intended to implement two-thirds of these suggestions to improve practice management. This finding can be interpreted as a potential effect of visitation. This effect was not outweighed by the rejected recommendations, which amounted to one-fifth of the total number of recommendations. The reasons to reject recommendations were not investigated in this study. In the pilot phase, some of these reasons were communicated orally. Some recommendations were not (financially) feasible; others did not stroke with the opinion of good practice management of the respondent.
The second goal of visitation is actual improvement. At the time of this study, respondents have already realized improvements or were engaged in improving one-third of the aspects. This means that visitation proved to be an effective method to induce potential and actual changes in allied health professionals.
These results are consistent with two other studies that documented effects of visitation for general practitioners and medical specialists [6,7].
The effectiveness of visitation was related to age: younger respondents tended to report an increase in awareness of weak and strong aspects more often than older respondents. Surprisingly, no relationship between age and actual improvements was found. The effectiveness of visitation was not related to profession. This may be the consequence of the small number of respondents in the separate professions. A more likely explanation, however, is that the effects of visitation are more or less the same in all groups. This thought is supported by the finding that the most important reason to participate was to get the necessary input to improve practice management.
The effects found in this study may be partly the result of positive self-presentation, because respondents know one of the aims of visitation is the inducement of change [10]. The confidentiality that was promised to the respondents may have decreased the social desirability effect.
The interval of 6 months between the actual practice visit and this evaluation study may have been too short in retrospect. A quarter of all respondents received feedback between 4 and 7 months after the practice visit. Consequently, these respondents had less time to implement improvements than others. The finding that respondents who received the feedback after 4 months more often reported having no plans for improvement than respondents who received feedback within 3 months supports this finding. A longer interval between the practice visit and this evaluation study may have resulted in larger effects.
In this study, it was not possible to show in detail which improvements were made by the allied health professionals. Between professions, differences exist in the practice management aspects that are studied. For instance, hygiene is not a practice management aspect that is assessed with dieticians. With dental hygienists, however, it is one of the most important practice management aspects. Studying only one profession would not have met with this drawback. Within one profession, improvements on one aspect of practice management such as hygiene may also vary. One dental hygienist can purchase up-to-date sterilization appliances, another can improve the facilities for storing instruments, and a third can start using protective gloves while treating patients. Because of these differences, effects of visitation had to be studied on a higher, more abstract level.
Changes in the use of methods for external quality assurance can be seen in the Netherlands and in other European countries [11]. For Dutch allied health professionals, the scope of the quality assessment is being extended to include clinical aspects of patient care. Also, visitation models are used to develop and implement accreditation schemes. A recent development is the use of web-based information gathering and benchmarking [12]. The results of this study support the continuation and further development of visitation in the quality improvement systems of allied health professions.
|
References |
|---|
|
TopMethodResultsDiscussionReferences |
|---|
- Van Weert C. Developments in professional quality assurance towards quality improvement: some examples of peer review in The Netherlands and the United Kingdom. Int J Qual Health Care 2000; 12: 239–242.
[Abstract/Free Full Text] - Heaton C. External peer review in Europe: an overview from the ExPeRT Project. Int J Qual Health Care 2000; 12: 177–182.
[Abstract/Free Full Text] - Baker R, Booth B, Grol R, Roberts R. Systems for quality improvement in general practice. A survey of 26 countries. Eur J Gen Pract 1997; 3: 65–68.
- Sluijs EM, Dekker J. Diffusion of a quality improvement programme among allied health professionals. Int J Qual Health Care 1999; 11: 337–344.
[Abstract/Free Full Text] - Van den Hombergh P, Grol R, van den Hoogen HJM, van den Bosch WJHM. Assessment of management in general practice: validation of a practice visit method. Br J Gen Pract 1998; 48: 1743–1750.[Web of Science][Medline]
- Van den Hombergh P, Grol R, van den Hoogen HJ, van den Bosch WJ. Practice visits as a tool in quality improvement: mutual visits and feedback by peers compared with visits and feedback by non-physician observers. Qual Health Care 1999; 8: 161–166.[Abstract]
- Lombarts MJHM, Klazinga N. Supporting Dutch medical specialists with the implementation of visitation recommendations: a descriptive evaluation of a 2-year project. Int J Qual Health Care 2003; 15: 119–129.
[Abstract/Free Full Text] - Rogers EM. Diffusion of Innovations. New York: Free Press, 1983.
- Grol RTPM, van Everdingen JJE, Casparie AF. Implementation of Guidelines and Change (in Dutch). Utrecht: De Tijdstroom, 1994.
- Bradburn NM. Response effects. In Rossi PH, Wright JD, Anderson AB, eds. Handbook of Survey Research. New York: Academic Press, 1983.
- Klazinga N. Re-engineering trust: the adoption and adaption of four models for external quality assurance of health care services in Western European health care systems. Int J Qual Health Care 2000; 12: 183–189.
[Abstract/Free Full Text] - Dutch Institute for Healthcare Improvement: http://www.cbo.nl/english/folder20031512164118.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||