International Journal for Quality in Health Care Advance Access originally published online on April 14, 2005
International Journal for Quality in Health Care 2005 17(4):287-292; doi:10.1093/intqhc/mzi040
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Measuring the quality of hospital tuberculosis services: a prospective study in four Zimbabwe hospitals
1 Aurum Health Research, Economics and Health Systems, Johannesburg, Gauteng, South Africa, 2 London School of Hygiene and Tropical Medicine, Public Health and Policy, London, UK.
Objective. To show how the use of a prospective approach to measuring the quality of services for a specific diagnosis can generate useful information for improving the quality of services in environments with limited information technology and data.
Design. Tracer approach focusing on intensive treatment of tuberculosis in hospital. The study was conducted in Zimbabwe in 1999. Local tuberculosis management guidelines were first translated into explicit quality assessment criteria and a panel of public health experts assisted in weighting different factors (structural and process) of the criteria. Factor weightings were based on both local knowledge and experience, and potential contribution of a factor to the likelihood of a positive outcome. A total of 138 patients was recruited into the study cohort at admission and followed up to discharge. An assessment of what was done to and for the patient was made for the entire hospitalization episode using explicit criteria. Comparisons were made between actual and maximum performance scores.
Setting. The study was conducted at four regional referral hospitals. The hospitals serve at least six secondary hospitals, and several public and private primary care facilities. The hospitals have a dual role as they also provide secondary care to their immediate catchment population.
Results. Notable quality gaps are observed between actual and maximum quality levels in all four hospitals although the size of the gap differed significantly. Variation in the quality of services between the hospitals is explained by distinguishable differences in structural and process aspects of tuberculosis management.
Conclusions. It is feasible to conduct prospective quality assessment in developing countries with minimal disruption of routine activities. The study also showed that prospective exploration of health care quality for a specific diagnosis can provide insights into hospital-level quality issues. Such information is useful for monitoring and improving the quality of hospital services in general.
Keywords: quality of services, tuberculosis, Zimbabwe
Address reprint requests to Charles Hongoro, E-mail: charles.hongoro{at}lshtm.ac.uk,hongoro{at}hotmail.com
Accepted for publication March 6, 2005.
Health systems around the world are being reformed. Pressure for change derives from the need to improve access, efficiency, effectiveness and quality of health services. However, it remains unclear what the impact of these reforms has been on quality of services [1]. To understand this requires facing the challenge of assessing the quality of health services in developing countries where information technology and systems for data collection are still in their embryonic stage.
The difficulties of measuring quality of health services correspond to those of defining it. The measurement method used depends on how quality is defined and what is feasible given the prevailing circumstances. This study sought to define and measure quality in a manner that was both sensitive to the context, and capable of exposing areas responsive to improvement.
Many have grappled with the conceptualization and operationalization of the quality of care concept [2–6]. Attempts to be comprehensive have often led to quality definitions that are difficult to measure in practice. The study adopted the Institute of Medicine [7] definition that: ‘Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired outcomes and are consistent with current professional knowledge’ because this definition is focused and could be translated in the study context.
Donabedian’s [3] structure–process–outcome paradigm is commonly used in quality assessment studies. In developing countries, quality assessment studies have tended to focus on structural and process aspects of quality [8,9] because of the problems of data availability and reliability. Data tend to be collected for mostly administrative purposes. Furthermore, these studies have relied on retrospective data the quality and completeness of which is often questionable, and patient satisfaction surveys. Little attention has been given to assessing the quality of in-patient services, and even less to using prospective approaches. In this study an attempt was made to correct for data deficiencies in estimating quality of in-patient services by using prospective patient-specific methods.
Use of specific tracer diseases to explore quality issues in health care is not new [10]. Use of tracer diseases in combination with explicit management criteria might provide an understanding for improving hospital services quality. Approaches available for setting explicit quality criteria include review of literature, panel of experts [11], and use of consensus building methods like the Delphi technique [12,13]. When using a panel of experts, it is important to ensure that the panel is constituted from practitioners working in the system.
There are two broad ways of measuring quality performance: (i) categorization of care into qualitative divisions such as ‘excellent’, ‘good’, ‘fair’, or ‘poor’; and (ii) use of quantitative weighted scores for specified aspects of care. The weight should reflect the relative importance of each factor to the total quality score. A panel of experts or literature review is used to assign the scores and weights. Providers are then ranked according to the total scores, and also against an expected yardstick quality score [14]. A provider performance index (PPI) is an example of such a total score [12,13].
However, the use of numerical scores faces two major challenges: (i) medical care sometimes has an all or nothing aspect which the component numerical scores might not reflect; and (ii) the manner in which different treatment components are weighted to arrive at a total score remains subjective. In this study, a local panel of experts assisted in designing the weighting system.
 |
Methods |
|---|
 Top Methods ResultsConclusionsReferences |
|---|
Choice of tracers and case definitions
The study was carried out in four tertiary hospitals in Zimbabwe in 1999. The hospitals are regional referral centres with 150–235 beds. Tuberculosis was selected for the study because it is a major cause of hospital admissions and deaths. In 1997, 3965 tuberculosis cases were hospitalized (7% of total admissions), and 438 patients died (20% of all hospital deaths) in one province [15]. The study was supported by the Ministry of Health and Child Welfare who provided national tuberculosis management guidelines.
The case definition for tuberculosis was described according to site and sputum status or history. Using the former, sputum-positive tuberculosis refers to a patient with two consecutive positive sputum smear examinations or with one positive smear examination and a chest X-ray suggestive of tuberculosis. Historically, a new case was defined as a patient who had never received a full course of tuberculosis treatment or who had received treatment for less than 1 month [16]. The treatment protocol for such patients was Category 1 high priority: 2 months of intensive treatment followed by a continuation phase of 4 months. The drug regimens based on WHO guidelines are given in the Essential Drug List and Standard Treatment Guidelines for Zimbabwe [17]. For practical purposes, only the principal diagnosis was considered in recruiting patients. Ninety per cent of all admitted tuberculosis cases were estimated to be HIV positive [15], which meant that cases were likely to be homogeneous in that respect. Co-morbidities tend to increase hospital stay, but the presence of home-based care programmes for chronic illnesses at these hospitals might have reduced the effect on length of stay.
Setting quality criteria and case recruitment
Explicit normative quality criteria for tuberculosis management were developed from the national management guidelines with the help of a selected panel of regional experts: three public health physicians and two senior nursing officers. National guidelines formed the core to which other clinically relevant but non-clinical process and structural factors were added. The criteria included diagnostic, therapeutic management, patient and environmental hygiene, documentation, and nursing variables. Each quality component was assigned a score weighted according to its relative importance in increasing the likelihood of the desired positive outcome. Process aspects of quality were entirely patient specific. Organizational and policy-related process factors were excluded because of physical constraints.
For each study site, two research assistants (health accountants) with nursing experience were recruited for patient recruitment and follow-up. They were trained in the application of the quality criteria, followed by trial runs on at least two in-patient cases each. Training was essential for standardizing data abstraction and ratings. Patients with symptoms of tuberculosis were conveniently recruited from hospital outpatient and casualty departments. Once recruited, they were visited every day, and as and when necessitated by critical moments in their treatment process. The assistant recorded what was done to and for the patient. Data were collected from patient charts and interviews (where possible), and ward staff (on non-sensitive aspects or verification of patient records).
A total of 138 cases was recruited from four hospitals. Suspected tuberculosis patients that were admitted for presumptive treatment and/or investigative purposes, and were later on either referred to a specialized infectious disease hospital (if positive) or discharged (if negative), were excluded from the analysis. Informed consent was obtained from the patients or their guardians before recruitment in the study. Patient codes were used to ensure confidentiality and protection of people’s medical data. The study obtained ethical approval from the Medical Research Council of Zimbabwe as part of a larger study on hospital performance.
Data were coded by hospital and patient. Data were computerized and analysed using SPSS. A total of 39 quality variables (including background characteristics) was created, and each variable entry represented a weighted score (Tables 1 and 2). A total weighted score was computed for each case/patient: 
, where Xij is the weighted score for each quality variable i for each patient j and n is the number of applicable variables per patient. The number of applicable variables depended on whether the overall score was being calculated or whether the score by quality cluster, process or structural, was being calculated. These weighted scores were used to calculate the average weighted score per patient/case in each hospital:
 |
|
|
where N is the total number of cases/patients in a particular hospital. The actual scores were compared against the maximum scores, calculated using maximum scores derived from the explicit criteria for tuberculosis management. By calculating subcategory or cluster scores it was possible to examine which factors influenced the overall score or quality performance. A PPI was also calculated by dividing the observed scores (AS) by the maximum scores (MS), and measures the extent to which hospital care providers performed according to explicit criteria, and takes values from 0 to 1. A value of 1 means total conformity with the quality criteria and zero reflects total non-conformity. Statistical comparisons of quality scores across hospitals were made using chi-squared and analysis of variance (ANOVA).
|
Results |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
The general characteristics of the study cohort are shown in Table 1. The mean age of patients ranged from 34 to 37 years, and no significant differences were found in sex mix. No marked differences in the proportion of referred and unreferred cases were observed across the study hospitals suggesting possible similarities in case-severity. However, high levels of self-referral (30–51%) reflect defective referral systems. The hospitals had significantly different average lengths of stay. Case fatality rates were relatively high probably because of high levels of HIV infection.
Table 2 shows a translation of the explicit quality criteria for tuberculosis into a weighted schema as advised by the panel of experts. The weights reflect the importance of a factor to both local quality expectations and prognosis, and were used to establish maximum overall and cluster scores. The PPI measures the extent to which hospital care providers performed in accordance with explicit criteria.
The overall quality scores for all the hospitals are shown in Table 3. None of the hospitals fully met the desired standard of service. Hospital 2 had the highest score whilst Hospital 3 had the lowest. Actual scores exceeded 64% of the maximum overall score. However, a shortfall of up to 36% for a fairly manageable disease such as tuberculosis raises serious questions about the quality of service. The overall scores differed significantly between the hospitals (P < 0.001) signifying dissimilarities in tuberculosis protocol compliance levels between hospitals. The overall difference between maximum and observed overall scores might be explained by the relatively large discrepancies that exist between optimum and actual structural quality scores (rather than between actual and optimum process scores). To enhance interpretation of the overall scores, it is necessary to look at the component parts separately.
|
Attempts to interpret structural quality scores should recognize the dearth of knowledge on the relationship between structure and process. All but Hospital 4 fell within the upper quarter of the maximum score even though statistically significant differences were observed between them (P < 0.001). This means that the basic infrastructure to potentially provide services of comparable quality was available. The low score for Hospital 4 is explained by poor ratings in the following factors: availability of drugs, presence of floor beds (overcrowding), inadequate microscopes, and poor building condition. For Hospital 2, the low score is explained by inadequate hand-washing facilities in the wards and functional equipment, and the presence of floor beds. The high score for Hospital 1 (>70% of the maximum score, 25) was due to better drug availability, working equipment, and the absence of floor beds. Hospital 3 had adequate space, and water and sanitation facilities.
Significant differences in process quality scores were observed (P < 0.001). Hospital 4 had the highest process score (25/31) whereas Hospital 2 had superior ratings for the majority of process aspects except for supervision of sputum collection, direct observation of treatment, and patient privacy compared with the remaining two hospitals. This means that Hospitals 2 and 4 provided relatively better diagnostic, clinical, and nursing services for in-patients. What is striking is that Hospital 4 had the lowest structural quality score but the highest process score. Hospital 1 had a relatively low process quality rating even though its PPI surpassed 69%. Low process quality ratings could be linked to hospital clinical practices (relatively low protocol compliance—frequency and observation of treatment), patient and environmental hygiene. Despite the observed statistical differences in the quality scores, at the aggregate level, the PPIs indicate relatively high levels of treatment compliance across all four hospitals
|
Conclusions |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
Measuring the quality of hospital services is difficult, and more so in contexts where information technology is poor. Even where data exist, they are of poor quality because the rationale for collection is usually administrative and not linked, for instance, to improving patient management or quality assurance activities. The prospective approach used in this study might have directly or indirectly introduced a Hawthorne effect. We sought to minimize this by avoiding direct observation of treatment, for instance, the administration of an injection, and instead checked (through records and patient interviews) at timely intervals whether an injection was given or not. The use of two research assistants at each study site for data collection allowed for cross-checking of data quality, and it enhanced patient recruitment. No comparisons were made to assess consistency. Such comparisons could have improved the quality of the results. Overall, the study showed that prospective assessment of the quality of tuberculosis services in hospitals was feasible and allows for an examination of both patient-specific and non-patient-specific aspects of hospital services.
At the patient level, it was clear that the management of tuberculosis services in all hospitals did not meet the local and international standards because of structural and process deficiencies. Hospital 4 had two wards (male and female) with low but similar structural scores. Although Hospital 4 had a poor structure for delivering tuberculosis services it had relatively better process scores attributable to better nursing services, environmental hygiene and diagnostic response. The hospital had the lowest case fatality rate, of 7%. Critical factors like patient counselling, direct observation of treatment, record-keeping, and patient hygiene were often ignored by hospital staff in hospitals which had better process quality scores.
Explanations of quality differences between hospitals that account for the differences in scores allow for consideration of the question of whether differences judged statistically significant have clinical significance. The relatively small absolute size of the differences in scores might suggest not, but the presence of drug availability and facilities for basic hygiene among those factors causing differences suggests that differences may in fact be important for outcomes.
The absence of adequate facilities and human resources to deliver quality hospital services, particularly for a high-burden disease such as tuberculosis in such a context, might not reflect successes in non-facility-based directly observed treatment (DOTs) programmes but poor funding and coordination of resources. This situation has been worsened by the high HIV/AIDS prevalence amongst adults (24%). Intensive phase treatment is a key component of the national tuberculosis programme and forms the basis for the DOTs programme. Therefore, the quality of initial treatment for tuberculosis has implications for the success of the national programme.
At the hospital level, prospective analysis of the quality of tuberculosis services allowed for an in-depth exploration of key hospital services essential for other pathologies such as radiography and laboratory (diagnostic) services, treatment, nursing services, and environmental hygiene. Such information is crucial in identifying areas of weakness and constraints in the hospital system—a prerequisite for any quality improvement programme. Tracers are used in this study as diagnostic tools for quality of hospital services, and it is clear from the results that interventions may generate notable quality improvements by focusing on structure and process aspects. The underlying assumption is that appropriate structures and processes increase the likelihood of positive outcomes. The foregoing is true for both developed and developing countries [18,19]. Nevertheless, the need is greater in developing countries where resources and information infrastructure cannot generate outcome measures of quality.
|
References |
|---|
|
TopMethodsResultsConclusionsReferences |
|---|
- Heiby JR, A crisis in quality. Int J Qual Health Care 2002; 14 (suppl.): 5.
- Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966; 44 (suppl.): 166–206.
- Donabedian A. The Definition of Quality and Approaches to its Assessment. Ann Arbor, MI: Health Administration Press, 1980.
- Shortbridge MH. Quality of medical care in an outpatient setting. Med Care 1974; 12: 283–300.[Web of Science][Medline]
- Black N. Quality assurance of medical care. J Public Health Med 1990; 12: 97–104.
[Free Full Text] - Bitran R. Efficiency and quality in the public and private sectors in Senegal. Health Policy Plann 1995; 10: 271–283
[Abstract/Free Full Text] - Lohr KN, Institute of Medicine. Medicare: A Strategy for Quality Assurance. Vol. I. Washington, DC: National Academies Press, 1990.
- De Geyndt W. Managing the Quality of Health Care in Developing Countries. World Bank Technical Paper No. 258. Washington, DC: World Bank, 1995.
- Gilson L, Magomi M, Mkangaa E. The structural quality of Tanzanian primary health facilities. Bull World Health Organ 1995; 73: 105–114.[Web of Science][Medline]
- Kessner DM, Snow C, Singer J. Assessing health quality—the case of tracers. New Engl J Med 1973; 288: 189–194.[Web of Science][Medline]
- Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003; 15: 523–530.
[Abstract/Free Full Text] - Payne BC, Lyons TF, Neuhaus E et al. Method of evaluating and improving ambulatory medical care. Health Serv Res 1984; 19: 219–245.[Web of Science][Medline]
- Mates S, Sidel VW. Quality assessment by process and outcome methods: evaluation of emergency room care of asthmatic adults. Am J Public Health 1981; 71: 687–693.
[Abstract/Free Full Text] - Montoya-Aguilar C. Measuring the performance of hospitals and health centres. World Health Organization, District Health Systems, Division of Strengthening of Health Services, WHO/SHS/DHS/94.2, 1994.
- Government of the Republic of Zimbabwe, Ministry of Health and Child Welfare. Benchmark report, 1998. MOHCW, 1998.
- Government of Republic of Zimbabwe, Ministry of Health and Child Welfare. Zimbabwe Tuberculosis Control Programme Manual. MOHCW, 1994.
- Government of Republic of Zimbabwe, Ministry of Health and Child Welfare. Essential Drug List of Zimbabwe and Standard Treatment Guidelines for Zimbabwe. MOHCW, 1994.
- Hostein A, Plaschke H, Schleiker et al. Structural and process quality in the management of diabetic emergencies in Germany. Int J Qual Health Care 2002; 14: 33–38.
[Abstract/Free Full Text] - Urassa DP, Carlstedt A, Nystrom LG. et al. Quality assessment of the antenatal program for anaemia in rural Tanzania. Int J Qual Health Care 2002; 14: 441–448.
[Abstract/Free Full Text] - Government of Republic of Zimbabwe, Ministry of Health and Child Welfare. Revised Standard Treatment Guidelines on Tuberculosis. MOHCW, 1998.
- Heidermann EG. The contemporary use of standards in health care. Geneva: World Health Organization, 1993.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  T. V. Perneger and P. M. Hudelson How international is the International Journal for Quality in Health Care? Int. J. Qual. Health Care, December 1, 2007; 19(6): 329 - 333. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||