OUP user menu

HIVQUAL-T: monitoring and improving HIV clinical care in Thailand, 2002–08

Sombat Thanprasertsuk , Somsak Supawitkul , Rangsima Lolekha , Peeramon Ningsanond , Bruce D. Agins , Michelle S. Mcconnell , Kimberley K. Fox , Saowanee Srisongsom , Suchin Chunwimaleung , Robert Gass , Nicole Simmons , Achara Chaovavanich , Supunnee Jirajariyavej , Tasana Leusaree , Somsak Akksilp , Philip A. Mock , Sanchai Chasombat , Cheewanan Lertpiriyasuwat , Jordan W. Tappero , William C. Levine
DOI: http://dx.doi.org/10.1093/intqhc/mzs008 338-347 First published online: 4 June 2012


Objective We report experience of HIVQUAL-T implementation in Thailand.

Design Program evaluation.

Setting Twelve government hospital clinics.

Participants People living with HIV/AIDS (PLHAs) aged ≥15 years with two or more visits to the hospitals during 2002–08.

Intervention HIVQUAL-T is a process for HIV care performance measurement (PM) and quality improvement (QI). The program includes PM using a sample of eligible cases and establishment of a locally led QI infrastructure and process. PM indicators are based on Thai national HIV care guidelines. QI projects address needs identified through PM; regional workshops facilitate peer learning. Annual benchmarking with repeat measurement is used to monitor progress.

Main Outcome Measure Percentages of eligible cases receiving various HIV services.

Results Across 12 participating hospitals, HIV care caseloads were 4855 in 2002 and 13 887 in 2008. On average, 10–15% of cases were included in the PM sample. Percentages of eligible cases receiving CD4 testing in 2002 and 2008, respectively, were 24 and 99% (P< 0.001); for ARV treatment, 100 and 90% (P= 0.74); for Pneumocystis jiroveci pneumonia prophylaxis, 94 and 93% (P= 0.95); for Papanicolau smear, 0 and 67% (P< 0.001); for syphilis screening, 0 and 94% (P< 0.001); and for tuberculosis screening, 24 and 99% (P< 0.01). PM results contributed to local QI projects and national policy changes.

Conclusions Hospitals participating in HIVQUAL-T significantly increased their performance in several fundamental areas of HIV care linked to health outcomes for PLHA. This model of PM-QI has improved clinical care and implementation of HIV guidelines in hospital-based clinics in Thailand.

  • Thailand
  • quality improvement
  • performance measurement


Since the onset of the Thailand HIV epidemic in the mid-1980s, 1.1 million cumulative HIV infections are estimated to have occurred, and ∼500 000 persons have died, leaving an estimated 610 000 persons living with HIV/AIDS (PLHA) [1]. Over the past decade, public hospitals have developed services for care of PLHA, and all now provide highly active antiretroviral therapy (ART) for persons who meet criteria for therapy. The most rapid expansion period for ART services in Thailand was in 2003–04 [2]. By 2008, an estimated 153 000 of 250 000 eligible patients were receiving ART [1].

As HIV care is being provided on a large scale, a major concern has been the development of mechanisms for monitoring the quality of care, and for improving care when it is not provided in accordance with recommended guidelines [36]. Considering the enormous investment in HIV care and treatment, these systems are critically important to maximize the benefits of ART and other HIV care, to minimize adverse events, including opportunistic infections (OIs), and to limit development of resistant HIV strains through appropriate use of medications [7]. Although hospital staff are familiar with basic quality improvement (QI) concepts from the hospital-wide patient care system led by the Thailand Health Care Accreditation Institute (HCAI) since 2001 [8], there was no QI program specific for HIV care in Thailand. Systematic implementation of QI methods would potentially improve systems through which HIV care is delivered and optimize specific components of HIV care [9].

In 2003, the Thailand Ministry of Public Health (MOPH) began working with the US Centers for Disease Control and Prevention (CDC) and the US Health Resources Services Administration (HRSA) to pilot a system for monitoring the quality of HIV care. We adapted HIVQUAL, an approach developed by the New York State Department of Health (NYSDOH) AIDS Institute in 1992 which has since been adopted statewide and expanded across the USA to many other HIV clinics supported by HRSA [10]. Key components of this approach include (i) systematic reporting of HIV care performance measurement (PM) data by clinics; (ii) development of specific activities to improve the quality of care for PLHA; and (iii) fostering a sustainable QI program structure that supports ongoing improvement in HIV care.

We applied this model to periodically monitor the quality of care provided to adult PLHA at Thai hospital clinics based on MOPH national guidelines for HIV care [1113]. Information generated from this monitoring was intended for use by hospital staff and public health authorities to initiate QI projects and monitor their results. It would also assist MOPH personnel, provincial health officials and government hospital staff to monitor the extent to which HIV care adults adheres to the MOPH HIV care guidelines. Following HIVQUAL-T implementation in an initial 12 pilot hospitals in 2002, the model was rapidly expanded to more than a hundred hospitals across Thailand in 2006, with technical support from Thailand MOPH and Thailand MOPH-U.S. CDC Collaboration. In 2008, the royal Thai government provided financial support to expand HIVQUAL-T to all public hospitals in Thailand [14]. We report here our experience with initial development and implementation of this system with 12 hospitals of varying size, selected from several regions of the country, during and after the period of rapid scale-up of ART in Thailand. We also compare performance scores among the 12 hospitals participating in HIVQUAL-T in 2002 and those initiating HIVQUAL-T in 2004 and 2006.


The PM and QI program for HIV care in Thai outpatient clinics (HIVQUAL-T) project was initiated through workshops held in 2002 with collaborators from the Thailand MOPH, Regional Offices of Disease Prevention and Control in Chiang Rai and Ubon Ratchathani provinces; the Bangkok Metropolitan Administration (BMA); Provincial Health Offices (PHO) from Chiang Rai, Chiang Mai, Phayao and Ubon Ratchathani provinces; and 12 major hospitals in these jurisdictions. The 12 participating hospitals were selected on the basis of HIV prevalence, existing collaborations with CDC and the hospitals' interest.

Performance indicators were developed based on Thai national HIV care guidelines to reflect the proportion of patients at each hospital that received recommended services [1113]. These guidelines recommended triple-drug ART for all HIV-infected persons with CD4 < 200 cells/μl, symptomatic persons with CD4≤250 cells/μl, and those with an AIDS-defining condition. Trimethoprim–sulfamethoxazole prophylaxis for Pneumocystis jiroveci pneumonia (PCP) is recommended for persons with CD4 < 200 cells/μl, and fluconazole and itraconazole prophylaxis for infection with Cryptococcus neoformans and Penicillium marneffei, respectively, is recommended for those with CD4 < 100 cells/μl. National guidelines also recommend an annual clinical assessment for tuberculosis (TB). Although recommendations for annual Papanicolau testing, serologic screening for syphilis and laboratory screening for Neisseria gonorrhoeae and Chlamydia trachomatis infection were not included in Thai national guidelines, care providers were interested in assessing uptake and screening rates in accordance with internationally recommended practices [15] and chose to include these indicators.

The NYSDOH AIDS Institute HIVQUAL software, a Microsoft Access-based program, was translated into Thai and modified to measure the indicators listed above. Staff at participating hospitals received 2-day trainings on the HIVQUAL-T model, including software used for self-reporting of performance data, use data for planning QI activities and the basic QI process using the plan-do-study-act (PDSA) cycle. The PDSA cycle is an iterative four-step problem-solving process. This process helps ensure that all changes are planned and tested and that feedback is incorporated before widespread implementation [16]. Examples of the QI process for HIVQUAL-T are reported elsewhere [17].

Each hospital made a list of eligible HIV-infected patients under review, identifying them by hospital number, sex and date of birth. Eligible criteria were PLHA aged 15 years and over with two or more visits to the hospital during the calendar year. The HIVQUAL-T software then generated a random sample of the records for review. When medical record review found that a patient was ineligible or when a record could not be located, the case was replaced by selecting the next patient of the same sex from the case list, such that the final case list approximated a random sample. The sample size was initially calculated using the method employed by NYSDOH incorporating Bayesian estimation methodology to reduce sample size [18]. In 2007, the sample size calculations were changed to include a standard formula for a binomial proportion (50%) with a 90% confidence interval width of ±15%, including a finite population correction [19]. To accommodate the requirements of female-specific indicators, the software sampled a higher proportion of females.

Data on the selected indicators were abstracted from patient records of visits made from 2002 through 2008, recorded on standardized forms and entered into the HIVQUAL-T software. The software applied algorithms to determine the number and percentage of patients who received appropriate care according to the selected indicators. Following data entry, results of analyses were immediately available to each hospital through the report generation function in HIVQUAL-T software. Each hospital provided this information in a report to the HIV care team, the PHO and the Thailand MOPH. The HIV care team used the results to identify areas for improvement, select topics for QI projects and develop QI work plans. While formal PM occurred annually, PDSA cycles were repeated at sites as needed, but usually every 1–3 months with simple review of data and QI processes. Annual benchmarking reports were also produced to compare performance among clinics at provincial and national levels. The PHO and MOPH used the HIVQUAL reports to provide supervision and monitoring to hospitals.

The unit data for analysis was the hospital indicator value; the median indicator values for each year were compared. The statistical significance of trends over time was assessed using the Wilcoxon non-parametric test for trend [20].

The median performance scores of various indicators in 2006 of the hospitals that started implementing HIVQUAL-T in 2002, 2004 and 2006 were compared. The Wilcoxon rank-sum test was used to test differences of median proportion between the hospitals implementing HIVQUAL in different time series. STATA software (STATA Corp., 2005, STATA Statistical software release 9, College Station, TX, USA) was used for analysis.

In 2006, following at least one QI cycle at each facility, which included data entry, report generation, development of a QI plan and follow-up PM, the HIVQUAL coordinator from each participating clinic was informally asked to identify to the QI committee the benefits of the project to the hospitals and challenges in implementation. Coordinators were also asked about time and other resource requirements for HIVQUAL-T implementation.

Implementation of the HIVQUAL-T protocol for data collection was approved by the Thailand MOPH and CDC as a program evaluation activity that did not require an IRB review.


The 12 participating hospitals that initiated HIVQUAL-T in 2002 were located in five provinces and Bangkok (Fig. 1), with a range of size, staffing and HIV case load (Table 1). Overall, the adult HIV case load at the 12 hospitals increased from 4855 patients in 2002 to 13 887 patients in 2008. The total number of patients included in the PM sample increased from 546 patients in 2002 to 1369 patients in 2008. Results for the HIV care indicators for each year are shown in Table 2.

View this table:
Table 1

Characteristics of 12 pilot hospitals using HIVQUAL-T in Thailand, 2005a

View this table:
Table 2

Quality of HIV care at 12 hospitals in Thailand: performance on key indicators, 2002–08

Figure 1

Pilot sites for HIVQUAL-T implementation, Thailand, 2002–08.

CD4 testing and ART

The median percentage of sampled patients that had ever received a CD4 test increased from 24% in 2002 to 99% in 2008 (P< 0.001). The percentage of the subpopulation of patients on ART who received CD4 testing every 6 months increased from 14% in 2003 to 77% in 2008 (P< 0.001). The median percentage of patients with CD4 < 200 cells/μl on triple-drug therapy was 100% (0–100%) in 2002 and 90% (52–100%) in 2008 (P= 0.741).

Prophylaxis for OIs

The median percentage of eligible sampled patients receiving cryptococcal prophylaxis increased from 65% in 2002 to 94% in 2008 (P= 0.049). Because P. marneffei infection is prevalent only in northern Thailand, the penicilliosis prophylaxis indicator was applied to clinics in the northern provinces of Chiang Mai, Phayao and Chiang Rai; at those eight clinics, the median percentage of eligible patients receiving prophylaxis increased from 0% in 2002 to 32% in 2008 (P= 0.011).

Screening for infections and cancer

The median percentage of patients screened for TB increased from 24% in 2003 to 99% in 2008 (P< 0.001). The median percentage screened for syphilis increased from 0% in 2002 to 94% in 2008 (P< 0.001). A small percentage of women had a Papanicolau test for cervical cancer screening during 2002; however, this percentage increased to 67% in 2008 (P< 0.001) (Table 2).

QI activities

During the 6 years, each of the 12 pilot hospitals initiated QI projects, focusing on areas of their choosing and stimulated by local review of their performance data. The most common indicators selected for projects were CD4 testing, Papanicolou examinations, OI prophylaxis, and TB and syphilis screening. Common improvement strategies used included the development of checklists and flow-charts for physicians. Some of the clinics also addressed structural issues such as establishing a multidisciplinary HIV care team that met regularly to plan appropriate care for patients and linking their HIV QI teams to the hospital-wide QI committees which helped provide resources and support for HIV QI activities (Table 3).

View this table:
Table 3

Examples of QI activities among 12 hospitals following initial HIVQUAL PM, Thailand, 2002–08

Performance scores among hospitals participating in HIVQUAL-T program in different time series

In 2004 and 2006, 29 and 64 hospitals, respectively, began submitting HIVQUAL-T performance data to Thailand MOPH. The median performance scores of the original 12 pilot hospitals were significantly higher when re-measured than of those that initiated in 2004 and 2006 on indicators, including clinical TB screening, pap smear and syphilis screenings (Fig. 2).

Figure 2

Median HIVQUAL-T PM among different cohorts hospitals initiating HIVQUAL in 2002, 2004 and 2006, Thailand, 2006. Wilcoxon rank-sum (Mann–Whitney) test. *Test for trend across ordered groups.

Evaluation of HIVQUAL-T implementation

Twelve clinic staff representing 12 hospitals provided feedback on HIVQUAL-T implementation through formal and informal meetings using both structured questionnaires and unstructured interviews. Nine of these respondents cited the importance of having a PM tool that was easy to use and allowed them to quantitatively assess their clinical service delivery. This tool helped them measure performance rapidly and allowed them to implement QI processes in a timely manner. They also cited the benefits of being tracking longitudinal performance. Other frequently cited benefits of HIVQUAL measurement included clearly defined indicators and the ability to automatically produce performance data reports. Eight participants noted the benefit of developing and sharing QI strategies to improve care with other providers in their own and other hospitals and the use of these strategies to build larger systems of hospital care. Five participants mentioned the opportunity to engage in a team effort in which all of the staff worked together to identify strengths and weaknesses in their clinic systems. Four respondents noted that HIVQUAL-T provided a useful mechanism for linking simple and effective PM methodologies with the hospital-wide QI system for hospital accreditation already in place in their facilities.

The most commonly identified challenge to HIVQUAL-T implementation was the increased workload associated with data abstraction and entry (seven participants), in part related to difficulties in finding medical records and in computer skills. Resource requirements for setting up HIVQUAL-T implementation are summarized in Table 4.

View this table:
Table 4

Resource requirements for HIVQUAL-T implementation in 12 pilot hospitals, Thailand, 2002–08 (Source: informal feedback from HIVQUAL coordinators at central and hospital levels)


The implementation of HIVQUAL-T demonstrates how a PM system can be established and implemented in a period of 1–2 years in a country that is rapidly scaling up HIV care. It further shows how these data can be analyzed locally and used to address issues affecting the quality of the service delivery system that are specific to a single institution, or common to many facilities. Resource requirements for implementing HIVQUAL-T during the pilot phase were relatively low, and the contributory percentage of full-time staff allocations to the work was quite modest.

Among the 12 hospitals reporting through HIVQUAL-T, substantial increases were seen in the proportion of patients receiving CD4 testing annually and every 6 months, the proportion of patients evaluated annually for TB, receiving syphilis serologic screening, and for HIV-positive women, annual Papanicolau testing. QI projects at individual hospitals led to specific actions to address these issues. Dialog between providers and health officials following review of data resulted in policy changes, including revision of clinical guidelines, involvement of provincial public health officials who sponsored improvement campaigns within their provinces and regional meetings for hospitals to exchange their solutions and tools to improve the quality of HIV care. Conversely, some indicators did not significantly improve often because performance scores were already high, or lack of prioritization of QI activities for those indicators. It is important to note that even if performance is initially good, it can decline if continuous assessment and QI are not addressed [17].

In some cases, changes in performance scores were influenced by external factors [6]. The year-to-year changes in the proportion of eligible patients receiving ART reflect increases in the availability of eligibility screening and subsequently, in the availability of treatment through the national program. In 2002, screening and treatment were provided only through a variety of pilot programs; a small number of patients were found to be eligible for ART and these patients were treated. CD4 testing in hospitals gradually became more available and in 2006, routine CD4 monitoring every 3–6 months for untreated HIV-positive patients was recommended and covered under the national health insurance program. As the number of patients with CD4 testing increased from 2003 to 2005, the proportion of those appropriately receiving ART decreased, but that proportion subsequently increased as the demand for ART was met.

During the introduction of HIVQUAL-T, many challenges were initially encountered. First, a quality measurement system based on statistical sampling of records had not previously been implemented in hospitals in Thailand. Prior monitoring systems had relied on universal reporting through systems of logbooks and reports, which were cumbersome and limited the number of useful variables that could be collected reliably. These data collection systems were often characterized by multiple records for an individual patient, posing obstacles to client-level data analysis. Secondly, HIVQUAL-T was introduced shortly after a system had been developed to monitor and evaluate the numbers and longitudinal outcomes of all persons placed on ART through the national program. Although the need for a more comprehensive program for monitoring the quality of HIV/AIDS care was recognized, the initial concern was that any such activities would be difficult to initiate. Thirdly, each facility had a unique system for recording patient data through hardcopy or electronic medical records; however, the sampling system of data collection used for HIVQUAL-T was shown to function effectively through either manual or electronic data extraction from these systems. Finally, many hospitals initially lacked a formal clinical registry to generate the HIV case listing required for HIVQUAL-T sampling; they had to compile lists from multiple sources, such as billing systems, scheduling lists and laboratory or research log books. These challenges highlighted the insufficiency of existing monitoring, reporting and other electronic systems to adequately capture information about processes of clinical care. Based on this experience, several hospitals developed a more systematic process for maintaining these records; improvements in medical record systems were also reflected in the national hospital accreditation program.

There are several limitations to the conclusions drawn from this analysis. First, the HIVQUAL-T assessment is limited to patients already diagnosed with HIV infection who present for clinical care. Separate assessments are needed of the coverage of voluntary HIV counseling and testing services, of other medical services that can identify persons who should be referred for HIV testing, of HIV care utilization among persons found to be HIV-infected and of HIV care provided outside of traditional hospital-based services, including day-care programs and home-based care services. However, the infrastructure and processes for PM and QI can be adapted and utilized for other HIV services and potentially, for other aspects of health-care delivery.

Secondly, accurate measurement using HIVQUAL-T depends on the quality and completeness of medical records. In many instances, medical records systems have major gaps. While those deficiencies are a barrier to HIVQUAL-T implementation, they can also be a primary factor leading to lapses in care for individual patients. We have found that the use of HIVQUAL-T leads to recognition and correction of problems in medical records documentation and systems that are then targeted for improvement. Improving medical record documentation helps health-care providers to assess true service coverage as well as provide appropriate clinical management for each patient, which in turn, improves the validity of subsequent rounds of PM.

Thirdly, we were initially concerned that the implementation of HIVQUAL-T would require commitment and human resources beyond the capacity of already overburdened facilities. However, we found that HIVQUAL-T helped address a need identified by hospital staff for clinical data on their facility's performance, provided a tool for generating discussion about aspects of care that some already knew needed improvement, and was effective in building commitment to the QI process. At the same time, it provided training of frontline staff, particularly nurses, in the national guidelines for HIV care and the importance of adhering to these guidelines, and established various QI approaches through which clinic staff could participate in decisions and changes in the care delivery system.

In contrast with the EQHIV study [21] that reported a QI collaborative did not significantly affect the quality of care, in this report, we found that various QI approaches may have led to improvement in HIV care quality, which is in line with reports of other QI studies [17, 2224].

In this report, hospitals that have a shorter timeframe to develop and implement HIVQUAL activities demonstrated lower performance scores for some indicators when compared with hospitals with earlier initiation of HIVQUAL and QI activities and sufficient time to implement them, suggesting that performance improvement may be attributed to QI activities [17].

As noted earlier, HIVQUAL-T was launched during a period of rapid scale-up and expansion of ART services. In this context, the relative contribution of HIVQUAL-T cannot be isolated. However, in hospitals where HIVQUAL-T has been implemented, the HIVQUAL process has facilitated service delivery and adherence to national HIV guidelines. In some cases, such as with syphilis screening, HIVQUAL-T led to the implementation of this practice at the pilot sites. In other cases, such as with CD4 testing, HIVQUAL-T QI projects were coincident with increasing availability of tests at hospitals, but HIVQUAL-T also facilitated a policy change to provide free CD4 testing for asymptomatic HIV-infected patients, which was subsequently adopted.

HIVQUAL-T promoted multidisciplinary teamwork in HIV clinics, provided an organizational framework for systematic review of performance data, facilitated implementation of delivery system changes to improve care and the organization of these activities within the hospital system. In addition, feedback from hospital staff linked improved care with the processes of PM and QI activities. Overall, a culture of QI practice was created among providers by promotion shared decision-making and team-based use of data. These data were also used to support the hospital accreditation process and for reporting to local and national public health authorities. As shown in other reports, clinics participating in the HIV QI collaborative resulted in organizational changes [25]. Organizations with a more open culture of QI practice tend to have more and comprehensive interventions which may affect the quality of care through their influence on intervention activities [26].

Key factors that facilitated the implementation of HIVQUAL-T were support from the Thailand MOPH and involvement of the National Hospital Accreditation Program, which has integrated QI into its accreditation standards. The implementation of any system of quality assurance requires that there be sustained support from the government or major funding agency, ideally resulting in the integration of the basic elements of QI into the national system for HIV services.

HIVQUAL-T is now being expanded to all 76 Thai provinces, with performance monitoring in place in over half of the nation's 900 HIV clinics [14]. HIVQUAL-T is also being integrated into the national health insurance system as their method for assuring the quality of covered services. The HIVQUAL model has also been adapted to other countries [27] as part of rapid, large ART scale-up programs, adjusting for differences in guidelines, resources and health-care models. In concert with the scale-up of ART and systems for delivery of HIV care, HIVQUAL provides a simple, systematic method to simultaneously monitor and improve the quality of HIV service delivery.

Conflict of interest statement

The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the US Centers for Disease Control and Prevention. The use of trade names is for identification purposes only and does not constitute endorsement by the US Centers for Disease Control and Prevention or the Department of Health and Human Services.


The HIVQUAL program was funded in 2003–05 by the US Centers for Disease Control and Prevention and by the Thai MOPH. From 2005 to 2008, funding was primarily from the Thai government but with continued technical assistance from the US Centers for Disease Control and Prevention.


We would like to acknowledge the staff of the hospitals participating in the HIVQUAL-T project for their hard work and dedication to improving the quality of care for their patients. In addition, we would like to thank the staff of the Bureau of AIDS, TB and STI in the Thailand Ministry of Public Health and the staff of the Regional Offices of Disease Prevention and Control and BMA. We thank Sarika Pattanasin, a statistician, Thailand MOPH-U.S. CDC collaboration for her assistance in data analysis.


View Abstract