International Journal for Quality in Health Care 16:285-291 (2004)
International Journal for Quality in Health Care vol. 16 no. 4 © International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved
Therapy outcome measures for allied health practitioners in Australia: the AusTOMs
1 School of Human Communication Sciences, 2 School of Physiotherapy, 3 School of Occupational Therapy and 4 School of Public Health, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia
Objective. The aim of this study was to develop a valid and reliable measure of therapy outcome for three allied health professions in Australia: speech pathology, occupational therapy, and physiotherapy. The Australian Therapy Outcome Measures (AusTOMs) enable measurement of the differences in client profiles and patterns of services provision across health care settings. In this paper we describe phase 1 of the study: the development and preliminary validation of the AusTOMs.
Method. The UK TOMs, developed by Enderby, were scrutinized by the research team. A pilot core scale was developed, based on the structure of the TOM. Focus groups of expert clinicians for each profession, across the state of Victoria in Australia, analysed and refined the scales further. A mail-out survey was then sent to therapists across Australia to assess both face and content validity of the AusTOMs.
Main results. A new tool, the AusTOM, was developed and tailored to the needs of each profession, with input from specialist clinicians and allied health researchers. The face and content validity of the new scales were assessed, and good consensus was obtained for the wording and content validity of the scales. The discriminative validity, concurrent validity, and reliability of the tool are now being evaluated.
Conclusion. We have produced an outcome measure in the Australian context for speech pathology, physiotherapy, and occupational therapy. There are six speech pathology scales, nine physiotherapy scales, and 11 occupational therapy scales in the AusTOMs. A clinician chooses the relevant scale(s) for the client (based on the goals of therapy) and makes a rating across all domains for each scale. Further papers will report on the reliability, validity, and clinical usefulness of the AusTOMs.
Keywords: AusTOMs, measurement, occupational therapy, physiotherapy, speech pathology, therapy outcome
Address reprint requests to Alison Perry, School of Human Communication Sciences, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia. E-mail: a.perry{at}latrobe.edu.au
Accepted for publication April 5, 2004.
In order to adopt an evidence-based approach to clinical practice, therapists need to routinely evaluate patient outcomes for a range of health care domains. As well as measuring impairments, recently there has been a shift towards quantifying the effects of therapy on activity limitations and restrictions to participation in societal roles. The well-being of patients and caregivers is another health domain that is now being monitored as a key indicator of therapy outcome. One such tool that is used to measure outcomes across these domains is the Therapy Outcome Measure (TOM), developed by Enderby [1] in the UK.
Using the TOM [2,3] involves ordinal ratings to assess a client across four domains, based on the World Health Organization’s (WHO) International Classification of Impairment, Disability, and Handicap (ICIDH) [4]. One domain describes impairment of body structure/body function (dysfunction), one describes disability (limitations in the performance of functional activities), and the remaining two address psychosocial status: handicap (social disadvantage to the individual) and well-being/distress (emotional level). These domains capture the targets (goals) of therapy [1]. Using TOM further demands consideration of the person’s disorder and its effects on function, as well as social and psychological factors such as societal participation and well-being. Several reliability trials involving speech and language therapists, occupational therapists, and physiotherapists have been conducted in the UK [2,3,5], which have demonstrated that the TOM has robust measurement properties, with good face and content validity reported [2].
The aim of this project was to devise and validate a therapy outcome tool for use by Australian allied health practitioners, based on the TOM approach. The relevance of the TOM to the professional culture and terminology used within Australia were first assessed; then substantial modifications were made to contextualize the final tool to Australian clinical practice. It is our view that an outcome measure that is specific to Australian health care will enhance efforts at quality assurance by enabling clinicians to quantify changes in patient functioning, using a tool that provides a common language across professionals.
 |
Materials and methods |
|---|
 Top Materials and methods ResultsDiscussionReferences |
|---|
Task 1: development of one Australian core scale
The research team’s first task was to investigate whether the descriptors used in the TOM were appropriate for use in Australia and, where necessary, adapt definitions in accordance with local requirements. Since the original development of the TOM, the WHO has revised the ICIDH classification (which underpins TOM) and a new classification has been published: the International Classification of Functioning, Disability, and Health (ICF) [6]. The ICF differs in several ways from ICIDH, including with respect to the terminology and focus. The ICF has a focus on the descriptors of ‘health’, rather than the consequences of ‘disease’. A more positive focus in terminology is noticeable in this new classification. After discussion, it was agreed that the descriptors used for the domains to be assessed in the Australian context needed to reflect these changes. The research team therefore first developed one Australian ‘core’ scale from which to expand more specific scales, relevant to each of the three professions. This mirrored the process used by Enderby and colleagues when developing the TOM [2]. An overview of the study process is shown in Figure 1.
|
Task 2: operational definitions for AusTOMs scales
Focus groups of expert Australian clinicians were asked to examine the TOM scales relevant to their profession. Although they all favoured using outcome measurement, criticism from those therapists in Australia who had used the TOM included dissatisfaction with the headings and with many of the written descriptors used in the UK scales.
Using the core scale developed in task 1 as a starting point, focus groups from each profession set out to develop further scales that would not only be profession-specific, but that would be able to measure outcomes of therapy for patients treated in Australia. The focus groups were made up of invited ‘expert’ clinicians (i.e. those with 
2 years experience in the relevant clinical area). This was a convenience sample, and included staff from centres where data would be collected later in the project.
The first step for each focus group was to look at the proposed scale headings for each profession, to determine if these were an accurate reflection of therapy areas covered by that profession. The groups then drafted text, defining each of the 0–5 points across the four domains on the scale. Clinicians provided terminology pertinent to these scales; for example, how they would describe a ‘severe’ speech impairment. This was done by debating and reaching consensus about, firstly, the extreme ends of each scale’s domain (e.g. defining points 0 and 5 under the ‘impairment’ domain), and then refining the mid-points.
Each focus group lasted 2–3 hours. Key discussion at each focus group was recorded onto audiotape and by writing consensus statements onto large sheets of paper. These were later typed as summaries by a member of the project team. Table 1 shows the focus group format for each of the three professions.
|
Task 3: face and content validity of the AusTOMs (modified Delphi Survey)
The profession-specific scales that had been assessed and adapted by the focus groups were then sent out as part of an Australia-wide survey of clinicians, using a modified Delphi technique [7] (i.e. two progressive mail-outs to gain consensus) to assure face and content validity of the scales.
A list of 1050 allied health managers from Australia’s National Allied Health Casemix Committee (NAHCC) was obtained. A mail-out to a convenience sample of 600 managers of allied health services (200 speech pathologists, 200 physiotherapists, and 200 occupational therapists) across Australia took place. The list of managers was stratified, first by state and then to achieve representation of both rural and metropolitan areas across as wide a representation of each profession as possible. This sample was used to assure the face validity of AusTOMs, via national consensus on the scale headings, and then agreement about the 0–5 point level domain descriptors, which were developed by the focus groups.
Each manager was sent a copy of all their own profession-specific scales, and asked to distribute the scales to ‘expert’ clinicians in each of the specialist clinical areas indicated by the particular scale heading. This was the only efficient way to disseminate the scales across Australia.
Speech pathology and occupational therapy
Clinicians who were given the survey were asked: (i) to scrutinize the scale descriptors line-by-line; (ii) to rate the overall accuracy of each line on a four-point scale (where 1 = an extremely inaccurate description and 4 = an extremely accurate description); and (iii) to add or remove words or phrases from the scale as necessary to achieve an accurate portrayal of the relevant clinical area and scalar points. Clinicians were also asked to comment on the face validity of the scale, including whether they felt it would be useful, whether the scale provided logical descriptors across domains, and whether they felt ‘comfortable’ with the concepts described.
Physiotherapy
At this stage, the physiotherapy scales did not have detailed descriptors for each of the 0–5 points in each domain. Hence, the Delphi survey used for physiotherapy differed considerably from the other professions. For the first mail-out, physiotherapy clinicians were asked to do two things; firstly, 5-point scales were used to rate their strength of agreement (1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, and 5 = strongly disagree) with the following statements for face validity:
- ‘This scale would be useful to measure outcomes for clients with this disorder’;
- ‘It would be easy to decide where a client would ‘fit’ on this scale’;
- ‘I feel comfortable with the descriptions given on this scale’;
- ‘The concepts of impairment, activity limitation, participation restriction, and well-being are relevant to the therapy I do and the outcomes I wish to measure’; and
- ‘I feel comfortable about using this scale with a range of clients with this disorder’.
Furthermore, physiotherapists were asked to comment (open-ended) on the suitability of the scale headings and the overall domains.
Task 4: consumer focus group
While the AusTOMs aims to provide a measure of outcomes to be used by the therapist, it was felt to be important to include consumer (client) views of health in the development of the tool. To this end, a focus group of consumers was convened, which six clients and consumer advocates attended. An additional seven clients/advocates were invited, but were unable to attend. The group examined all of the AusTOMs, as developed in phases 2 and 3 of the study.
|
Results |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
Task 1: development of an Australian core scale
The initial milestone in this project was a revision, on both a linguistic and cultural level, of the core outcome measurement scale. A core scale was derived for all professions to use with their focus groups. This scale in its final form is shown as Table 2. The scale represented a departure from the TOM in the terminology used for each domain, although the core ‘structure’ of the TOM was retained.
|
Task 2: operational definitions for AusTOMs
Speech pathology
The participants in the focus groups in speech pathology expressed the view that the speech pathology TOM presented a mixture of terminology, and a suggestion was made to use the terminology used by Speech Pathology Australia in their Competency Based Occupational Standards (CBOS) [8]. These are terms with which all Australian speech pathologists are familiar. After further discussion, it was felt that there may be a need for one additional scale to recognise ‘cognitive communication’ as an area of speech therapy practice that would not fit into any of the CBOS areas. Table 3 shows the final headings for the speech pathology scales.
|
These terms have two advantages: firstly, they are all descriptions of functional ability in areas where speech pathologists work with clients; and secondly, they all are descriptions in a positive direction, rather than of disorder. The latter is conceptually more in keeping with the new ICF philosophy (see above), describing what clients can do, rather than what they cannot [6].
Speech pathology focus groups then developed terminology for each of these six scales.
Occupational therapy
Using the ICF [6] framework, occupational therapists work mostly at the levels of activity and participation. The initial two focus groups of academic occupational therapists suggested a structure for AusTOMs according to the ICF [6] classifications of ‘activity/activity limitation’ and ‘participation/participation restriction.’ Scale headings were developed to reflect client activities, such as self-care, functional walking, domestic life, and employment. A chosen scale would then align with the dominant goals of therapy interventions. Clinicians participating in the seven focus groups then confirmed these headings and developed the descriptors for each of the four AusTOMs domains under each scale heading. The set of occupational therapy scales developed is shown in Table 4.
|
Physiotherapy
Based on the ICF [6], the participants of the physiotherapy focus groups felt that basing scale headings on impairments of body structure and function would provide a strong construct for the physiotherapy scales for AusTOMs. Using the body function headings from the ICF, 11 scales were originally proposed for further debate and refining. Later modifications (see below) resulted in nine scales, as shown in Table 5. Overall descriptors for each domain were developed.
|
Task 3: face and content validity of the AusTOMs (Delphi Survey)
The modified Delphi [7] survey across Australia resulted in 240 scales returned by speech pathologists, 228 by physiotherapists, and 228 by occupational therapists in round 1. Returns represented all Australian states and territories. These same respondents were sent a second mailing with the modified scales. There was a return rate ranging from moderately good for speech pathology (42%) and occupational therapy (52%), to poor for physiotherapy (21%), for the second round of the Delphi.
Speech pathology
The results from the first mail-out showed that speech pathologists mostly agreed that the scales had good face validity and were comfortable with the concepts on which the scales were developed. For the impairment domain of the voice, language, speech, cognition, and swallowing scales, the percentage of descriptors rated by speech pathologists as ‘extremely accurate’ (i.e. with a rating of 4, see Methods) ranged from 55% to 93%, with the exception of the fluency scale, where descriptors were rated as extremely accurate only 8% of the time. The results for fluency impairment were poor as clinicians wished to have a descriptor added: ‘percentage of syllables stuttered’ across each level of the impairment domain. This was changed before sending out the second round for Delphi consensus. With this addition, the second round consensus resulted in 92% of impairment descriptors of the fluency scale being rated by participants as extremely accurate. For the other scales, clinicians’ agreement with the descriptors was consistently >75% for the second mail-out, after minor changes in wording were made to the scales, as recommended by clinicians.
At the first mail-out, participating speech pathologists rated between 73% and 85% of the descriptors in the activity limitation domain as extremely accurate, and 73–89% of the participation restriction domain descriptions. The results for well-being were similar, with participating speech pathologists rating 77–92% of descriptors as extremely accurate. After minor modifications were made to the second mail-out, final results indicated that clinicians believed that well over 75% of the domains in every scale had accurate descriptors.
Occupational therapy
From the results for the first mail-out, participating occupational therapists rated the descriptors of the impairment domain across all 12 scales as extremely accurate >80% of the time (range 83–100%). The results were even better for the activity limitation and participation restriction domains, with therapists indicating that 92–100% of these descriptors were extremely accurate. For well-being, excellent results were obtained, with 94–100% of descriptors being judged as extremely accurate.
The second round of the process resulted in >90% concordance regarding the scales and scale headings to be used, and for the text descriptors across the 0–5 points of rating.
Physiotherapy
Across the 11 initial physiotherapy scales, clinicians were either in moderate agreement with statements, or they were neutral/undecided. No scale showed either full agreement or disagreement/strong disagreement with any of the statements (see Methods).
There was a moderate degree of concordance with the scale headings and the overall descriptors on this initial version of the physiotherapy scales. As a number of clinicians disagreed with the wording of some of the descriptors, further revision was necessary. Thus, before the second mail-out, the physiotherapy AusTOMs team ran further in-depth interviews with a small panel of physiotherapists to guide refinement and revision of the scales in areas where disagreement was greatest. More detailed descriptors were developed and revised scales were re-sent to the clinicians who had participated in the first modified Delphi. In the second mail-out, physiotherapy clinicians were given the following instructions: (i) cross out any parts of the scale or the ‘Instructions for use’ document that you think are unsuitable or need to be re-worded; and (ii) change the words and add any comments or extra information that you feel are necessary to the scale(s) or the ‘instructions’ document, on the attached papers.
The result of this second round of returns was overwhelmingly positive, with very few changes to wording suggested for headings and scalar descriptors. Many physiotherapists expressed satisfaction with the scales and noted that their earlier concerns about the scale descriptors had been addressed. All the suggested wording changes were therefore incorporated into the final versions. There was consensus from the mail-out responses and from the expert clinicians that ‘mental functions’ and ‘reproductive system functions’ were not scales that were appropriate or useful to Australian physiotherapy practice. These were therefore omitted from the second phase of development, resulting in nine final scales.
Task 4: consumer focus group
The consumer focus group made no changes to the wording of the impairment or activity limitation domains for any of the scales. It was generally felt that these domains were carefully thought through, with close attention to detail. The group was positive about these domains, with participants advising that these sections of the tool appeared to be suitable for enabling allied health practitioners to measure outcomes of therapy. Participants pointed out that some of the wording of the participation restriction domain was ambiguous, and would require careful explanation in training and/or guidelines for use of the tool. Minor wording changes were suggested, such as replacing the phrase ‘reassurance and support needed’ with ‘emotional support or encouragement needed’, as the word ‘reassurance’ was not felt to reflect a person’s level of well-being. As a result of this focus group, several minor changes in wording were made to the ‘participation restriction’ and ‘well-being’ domains of the scales.
|
Discussion |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
This paper describes the process by which clinical consensus was used to develop new therapy outcome measures, the AusTOMs, for use across speech pathology, occupational therapy, and physiotherapy practices in Australia, thereby providing evidence for the face and content validity of the measure. During the course of developing the tool, clinicians reiterated the importance of using consistent constructs across scales used to measure therapy outcomes. They were also more comfortable in using terminology that reflected practices in their own country (in this case Australia), rather than directly adopting terms that may be used overseas.
The three professions that participated in this study have different professional mores and philosophies. Occupational therapy has grown out of the need to understand the relationship between what a person does—their occupation—and their health and well-being. The ‘core business’ of occupational therapists is to habilitate and rehabilitate people who have difficulties managing their daily activities and participating in society following physical, emotional, psychological, and/or social disorders. Physiotherapy aims to enable people to move more easily so that they can perform functional activities at their optimal level. Although traditionally physiotherapy practice focused on the treatment of impairment, more recently there has been a paradigm shift and practice is beginning to take into account disability and societal participation. Speech pathology has a mixed emphasis. Addressing communication and/or swallowing disabilities involves tackling the changes at the impairment level (e.g. structural changes that occur after surgery or from neurological deficit), and then the psycho-social, activity, and well-being domains are addressed, as communication impacts upon all three. The AusTOMs provide a common vocabulary for assessing outcomes, thereby crossing philosophical boundaries of the three disciplines.
One of the strengths of this study lies in the fact that both therapist and client viewpoints were sought in its development. Clinician involvement was essential, as clinicians will ultimately be those who need to accept and use this tool in clinical practice. The need for a ‘common, patient-understood language of health outcomes’ was emphasized by Paul Ellwood ([9], p. 1551), one of the key contributors to outcome measurement in the United States. The AusTOMs were developed primarily from the clinicians’ perspective, but have also been scrutinized and modified by clients, resulting in a tool that may be comprehensible and useful to both groups.
A limitation of the AusTOMs is that they have been deliberately contextualized to Australian allied health practice. The tool reflects the ways in which speech pathology, occupational therapy, and physiotherapy are provided in Australia, which may vary in other parts of the world. Further testing of the validity, reliability, sensitivity to change, and clinical usefulness have been undertaken, and will be reported in a companion paper.
The measurement of therapy outcomes is an essential, yet sometimes under-emphasized, component of clinical practice. In allied health professions such as physiotherapy, occupational therapy, and speech pathology, there are few valid therapy outcome tools that are well matched to contemporary practice, as well as being quick and easy to administer. The AusTOMs aim to address these needs and provide clinicians with scales that measure impairment, activity limitation, participation restriction, and well-being, using terminology that reflect the different cultures of the allied health disciplines. The next paper will report the reliability, validity, and sensitivity of this new therapy outcome measure, as well as its usefulness across clinical settings.
La Trobe University acknowledges that this publication draws heavily upon the ‘AusTOMs; Australian Therapy Outcome Measures project’, commissioned by the Commonwealth Department of Health and Ageing, Canberra, Australia. We thank Ms Sue Cotton, Biostatistician from the University of Melbourne, who assisted us with advice on data management and data analyses for this project. Ms D. Benetti provided administrative assistance throughout. Professor Pam Enderby assisted the research team at La Trobe University in the application to the Commonwealth to support the project, and both professor Enderby and Dr Alexandra John from Sheffield University, UK, are associate researchers to this project, providing the research team with advice, discussion, and support in this development of the AusTOMs. We are grateful for their guidance and encouragement.
|
References |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
- Enderby P. Outcome measures in speech therapy: impairment, disability, handicap, and distress. Health Trends 1992; 20: 61–64.
[Abstract/Free Full Text] - Enderby P, John A. Therapy Outcome Measures: Speech-Language Pathology Technical Manual. London: Singular, 1997.
- Enderby P, John A, Petherham B. Therapy Outcome Measures manual: physiotherapy, occupational therapy, rehabilitation nursing. San Diego, CA: Singular, 1998.
- WHO. International Classification of Impairments, Disabilities and Handicaps (ICID-H). Geneva: WHO, 1980.
- John A, Enderby P. Reliability of speech and language therapists using Therapy Outcome Measures. Int J Lang Commun Disord 2000; 9: 287–302.
- WHO. International Classification of Functioning, Disability and Health (ICF). Geneva: WHO, 2001.
- Sim J, Wright C. Research in Health Care: Concepts, Designs and Methods. Cheltenham: Stanley Thornes, 2000.
- The Speech Pathology Association of Australia (SPAA). Competency Based Occupational Standards (CBOS) for Speech Pathologists. Melbourne: SPAA Limited, 1994.
- Ellwood PM. The Shattuck Lecture—outcomes management: a technology of patient experience. N Engl J Med 1988; 318: 1549–1556.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  E. Ward, J. Crombie, M. Trickey, A. Hill, D. Theodoros, and T. Russell Assessment of communication and swallowing post-laryngectomy: a telerehabilitation trial J Telemed Telecare, July 1, 2009; 15(5): 232 - 237. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Starrost, S. Geyh, A. Trautwein, J. Grunow, A. Ceballos-Baumann, M. Prosiegel, G. Stucki, and A. Cieza Interrater Reliability of the Extended ICF Core Set for Stroke Applied by Physical Therapists Physical Therapy, July 1, 2008; 88(7): 841 - 851. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||