OUP user menu

Evaluation of accreditation program in non-governmental organizations' health units in Egypt: short-term outcomes

Mahi Al Tehewy, Bssiouni Salem, Ihab Habil, Sayed El Okda
DOI: http://dx.doi.org/10.1093/intqhc/mzp014 183-189 First published online: 13 May 2009

Abstract

Objectives To determine the effect of accreditation of non-governmental organizations' health units on patient satisfaction and provider satisfaction and the output of accreditation on compliance to some accreditation standards.

Subjects and methods Sixty non-governmental health units were selected as follows: 30 units already submitted for accreditation in three governorates and 30 pair-matched units not programmed for accreditation. Matching was done according to the socioeconomic standard and administration type, and from the same governorate. Satisfaction was measured by an interview questionnaire using the Likert scale. Assessment of compliance to some accreditation standards was done using a checklist.

Results Mean patient satisfaction scores were significantly higher among the accredited non-governmental health units regarding: cleanliness, waiting area, waiting time, unit staff and overall satisfaction. No significant differences were noticed in provider satisfaction except for the overall satisfaction score. Most of the checked standards had compliance above 90% in the accredited units and were significantly higher than in the non-accredited units.

Conclusion Accreditation of the non-governmental health units has a positive effect regarding patient satisfaction and the continuation of performance according to the accreditation standards compared with non-accredited health units. This short-term effect was shown within the first year from accreditation. Future research is needed to assess long duration effects of applying accreditation in non-governmental health units.

Keywords
  • external quality assessment
  • general practice
  • measurement of quality
  • patient satisfaction
  • primary care
  • setting of care
  • specific accreditation (laboratory, etc.)

Introduction

Accreditation of health care organizations has been in practice in many countries over the world [1]. Now in an increasingly evidence-based world, scarce data are available on the impact of such accreditation programs on the health system and customer perception [2]. There is evidence that health care organizations rapidly increase compliance with the published standards in the months prior to the external assessment and improve organizational processes but there is less evidence that this is beneficial in terms of clinical process and outcome [1]. Needless to say that environmental factors such as culture, incentives and regulations are greatly associated with the success or the failure of the general model of accreditation in a given country [3]. On the other hand, accreditation is more difficult to evaluate than a clinical technology as the endpoints of accreditation are hard to define and vary according to the expectations of users and customers [1].

An accreditation program has been implemented in Egypt in the recent years. It was started for primary health care units and now has been extended to hospitals. Recently, a program for preparation of primary health care units administered by non-governmental organizations (NGOs) for accreditation was put into action. Accreditation of primary health care units has been part of the health sector reform in Egypt adopting the family health care approach model. Evaluation of such efforts at the level of NGO-delivered health services will add insight on the progress and value of accreditation.

The aim of the current study was to determine the effect of accreditation of NGO health units on patient satisfaction and provider satisfaction, and to determine the output of accreditation of NGO health units on compliance to certain accreditation standards.

Subjects and methods

The three pilot governorates of the Health Sector Reform Project in Egypt, namely Alexandria, Menofia and Souhag were involved in the study.

Type of study: Quasi-experimental cluster study.

Study setting and sample

Number of clusters: Thirty NGO accredited units and 30 pair-matched NGO units not programmed for accreditation. The control units were selected from the same governorate and matching was done according to the administration type (e.g. Church–Mosque private agency). Matching according to the socioeconomic level was assured by selecting the control unit from the same district.

Comparison between the intervention and control units was done within the first year after the intervention units got the accreditation.

On average, 60 subjects were enrolled using consecutive sampling for the patient satisfaction survey in each NGO unit. All employees in the studied health care units were enrolled for the assessment of provider satisfaction.

Description of the intervention program

The program consists of three phases: preparation phase, accreditation phase and follow-up phase. The preparation phase includes training, improvement of structure and provision of necessary equipment and supplies. The training comprises training of the health care workers on the family health model, records, clinical guidelines, infection control, communication skills and quality in health care. The preparation phase takes 1 year, after which the health unit submits a request to the accrediting body in Egypt to get accreditation. Upon successful accreditation, the heath units are monitored by a set of monitoring indicators till the next due date for re-accreditation. The accreditation standards framework consists of: patient rights, environmental safety, clinical safety, patient care, supportive services (laboratory, radiology, non-medical, e.g. housekeeping, laundry), performance improvement and information management.

Administrative approval

Approval to conduct the study in the selected NGO health units was obtained from the related authorities.

Development of the study tools

Preliminary versions of the study tools were constructed during several meetings of the research team. They included the following:

  1. Patient satisfaction questionnaire: This included patient gender, age, previous visits, cause of the visit and satisfaction questions regarding information given, cleanliness of the unit, attitude of doctors, attitude of nurses and overall satisfaction. Satisfaction was measured using 5- and 10-point Likert scales.

  2. Provider satisfaction questionnaire: This included provider gender, age, profession, years of experience and satisfaction questions regarding the following items: availability of equipment and supplies, work load, interpersonal relationship with colleagues, work constraints, assessment of performance, family health models and financial rewards in the unit.

  3. Checklist for compliance with selected standards: The selection of these measures respected the monitoring indicators set by the General Directorate of Quality in the Ministry of Health and Population (Egypt) for follow-up of the accreditation status in the accredited units. They are basically driven from the accreditation standards. The selected output measures were: clean toilets; properly functioning toilets; appropriate furniture; presence of a functioning alarm system; presence of an incineration contract; presence of a maintenance contract; presence of a complaints box; announced patient rights; analyzed patient satisfaction questionnaire (at least once); analyzed provider satisfaction questionnaire (at least once); presence of a record room; presence of records; presence of guidelines; presence of an emergency drug list; recording at least two visits for patients with hypertension, diabetes and antenatal care; waste segregation; proper sharp disposal; presence of three sterilized sets; and an employee health file.

After testing the tools, some modifications were introduced till the final version was established. The checklist was filled in each selected unit by a surveyor trained in the national accreditation program, who carried out an observation tour over a 2-day visit to the unit. Three to five surveyors were assigned for each governorate.

Training on data collection

One-day training of field investigators on the study tools was conducted to explain the objective of the study, the sampling methodology and how to use the data collection tools.

Data collection

The field investigators from each governorate collected the necessary data from the selected health units. To ensure the quality of collected data, the data collection process was supervised in each governorate through independent supervisors.

Data manipulation and analysis

Collected data was checked, coded and entered in a statistical computer program (SPSS for windows). Analysis of the effect of the accreditation intervention was done by comparison between naturally experimented clusters (submitted for accreditation) and the control clusters (not submitted for accreditation).

Analysis of patient and provider satisfaction: The different items of the questionnaires on patient satisfaction and provider satisfaction were subjected to exploratory factor analysis for the ultimate purpose of data reduction. Initial screening of responses within the items in the questionnaire was done for floor and ceiling effects. None of the items had more than a 50% response of the extreme scale. The principal component analysis method was used to extract the factors, and this was followed by a varimax rotation with Kaiser normalization. In interpreting the rotated factor pattern, an item was considered to load on a given factor if the factor loading was 0.40 or greater for that factor and was <0.40 for the other factors. When an item loaded in more than one factor, it was removed from further analysis. The number of factors retained in the study was based on eigenvalues ≥1. To force an additional factor in patient satisfaction analysis, the cleanliness factor (composed of two related items), which was considered important in the analysis, was added. The items that made up these factors were further examined for the internal consistency of scale scores by means of reliability analysis using Cronbach's alpha coefficient. To enable an easy comparison of the scores on different aspects of satisfaction, all scales were transformed to a scale with a minimum of 0 and a maximum of 100 by dividing the difference of the crude score and the minimum crude score by the score range and multiplying by 100.

Comparison of mean satisfaction scores was done taking into account the clustering effect using complex samples analysis in SPSS version 15.

Analysis of output of accreditation: Comparison between the studied units regarding the output of accreditation was done using the Chi-square or Fisher's exact test.

Results

Table 1 shows the distribution of the patients enrolled for the satisfaction survey. A total of 3895 patients were enrolled in the three governorates selected. The number of females was nearly double that of males in both the study and control groups. There was comparable distribution of providers between both study groups regarding job description, mean age and gender.

View this table:
Table 1

Characteristics of the study population

InterventionControl
Patients sample
 Governorate [no. (%)]
  Alexandria802 (40.6)737 (38.4)
  Menoufia583 (29.5)599 (31.2)
  Sohag591 (29.9)583 (30.4)
 Gender [no. (%)]
  Male628 (32.2)740 (39.0)
  Female1320 (67.8)1159 (61.0)
 Education [no. (%)]
  Essential & preparatory991 (50.8)843 (44.4)
  Secondary677 (34.7)673 (35.5)
  University284 (14.5)381 (20.1)
 Age (mean ± SD)35.4±12.633.9±12.5
Providers sample [no. (%)]
 Profession
  Doctors35 (17.9)33 (23.1)
  Nurses41(21.0)27(18.9)
  Workers34 (17.4)24 (16.8)
  Administrative51 (26.2)39 (27.3)
  Technician34 (17.4)20 (14.0)
 Gender [no. (%)]
  Male88 (45.8)60 (43.2)
  Female104 (54.2)79 (56.8)
 Age (mean ± SD)34.4±11.835.2 ± 11.8

Exploratory factor analysis for patient satisfaction identified four factors that were named according to the composition of its items into the following: cleanliness, unit staff, waiting area and waiting time. The composition of the factors is shown in Table 2. The consistency of the items scale for each factor was good to excellent according to Cronbach's alpha >0.8. The patients in the accredited health units expressed significantly higher satisfaction scores compared with the control group regarding cleanliness, waiting area, waiting time and unit staff as well as regarding overall satisfaction after adjusting the effects of age, gender and education (Table 3). The provider satisfaction questionnaire was classified into three main factors according to analysis of the following factors: administrative environment, social environment and family health model. The components of the three factors are shown in Table 4, with adequate reliability. Although there was no significant difference in satisfaction between providers in the accredited and non-accredited units regarding the administrative environment, social environment and the family health model, the providers in the accredited units expressed significantly higher overall satisfaction scores compared with providers in the non-accredited units (Table 5).

View this table:
Table 2

Factor loadings for the final set of items of patient satisfaction questionnaire

Unit staffCleanlinessWaiting areaWaiting time
1Cleanliness of the unit0.75
2Cleanliness of the toilets0.77
3The space of waiting area0.80
4The seats are comfortable0.77
5The number of seats is sufficient0.79
6Pre-examination waiting0.70
7Pre-investigations waiting0.82
8Waiting for results of investigations0.77
9Examination time0.73
10Doctors' explanation to diagnosis and management0.75
11Doctors' skills0.77
12Nurses' dealing with patients0.68
13Nurses' skills0.68
14Doctors' respect of confidentiality of patient information0.82
15Nurses' respect of confidentiality of patient information0.78
16Privacy during examination0.80
Cronbach's alpha0.940.820.900.90
View this table:
Table 3

Comparison of patient satisfaction scores in the studied health unitsa

Intervention, mean (SE)Control, mean (SE)P value
Unit staff90.6 (0.92)83.2 (2.6)0.005
Cleanliness81.3 (1.2)71.9 (2.6)<0.001
Waiting area85.7 (0.87)73.8 (2.7)<0.001
Waiting time75.2 (1.1)67.8 (2.4)0.005
Overall90.4 (1.07)79.5 (2.7)<0.001
  • aSatisfaction scores were adjusted for gender, education and age.

View this table:
Table 4

Factor loadings for the final set of items of provider satisfaction questionnairea

Administrative environmentSocial environmentFamily health model
1Satisfaction with supplies and equipment0.66
2Satisfaction with work load0.60
3Flexibility of work0.60
4Relationship with colleagues0.84
5Relationship with directors0.73
6Responsiveness of unit to work demand0.76
7Difficulties met during work0.74
8Family medicine system0.84
9Record for the system0.79
10Financial income from working in the center0.53
Cronbach's alpha0.760.720.76
  • aSatisfaction with performance assessment loaded in administrative and social environment. Sense of appreciation loaded in administrative environment and family medicine approach and accordingly these two items were omitted from further analysis.

View this table:
Table 5

Comparison of provider satisfaction scores in the studied health unitsa

Intervention, mean (SE)Control, mean (SE)P value
1Administrative environment70 (1.9)68 (2.5)0.52
2Social environment84 (1.9)80 (1.8)0.15
3Family health model85 (1.6)77 (4)0.07
4Overall satisfaction81 (2)73 (3)0.04
  • aSatisfaction scores were adjusted for gender, job type and age.

Table 6 shows wide discrepancy between both groups regarding different environmental safety standards, especially the presence of a functioning alarm system, maintenance contract and functioning toilet or appropriate furniture. Regarding patient rights, the intervention group had better performance compared with the control group for all studied standards (P < 0.001). As regards the information system and records, 100% of the health units in the intervention group had records for their patients and 96% of them had a separate room for the retention of records compared with 12 and 20% in non-accredited units, respectively. Compared with the non-accredited units, accredited health units showed greater compliance with certain patient care standards, such as the availability of clinical guidelines and an emergency drug list. Recording of patient visits in patient records, especially for antenatal care and chronic diseases such as hypertension and diabetes, is significantly better in accredited health units than in non-accredited units. The accredited units also showed a higher degree of compliance to infection control and clinical safety standards compared with the non-accredited units.

View this table:
Table 6

Output of accreditation in the studied health units

Intervention, (n = 29)a no. (%)Control, (n = 24)a no. (%)P value
Clean toilets27 (93.1)9 (37.5)<0.001
Functioning toilets23 (79.3)8 (33.3)<0.001
Appropriate furniture28 (96.6)6 (25)<0.001
Alarm system27 (93.1)0 (0)<0.001
Incineration contract26 (89.7)5 (20.8)<0.001
Maintenance contract22 (75.9)2 (8.3)<0.001
Complaints box28 (96.6)2 (8.3)<0.001
Announced patient rights29 (100)2 (8.3)<0.001
Analyzed patient satisfaction questionnaire (at least once)25 (86.2)5 (20.8)<0.001
Analyzed provider satisfaction questionnaire (at least once)27 (93.1)5 (20.8)<0.001
Presence of record room28 (96.6)5 (20.8)<0.001
Presence of records29 (100)3 (12.5)<0.001
Presence of guidelines24 (82.8)2 (8.3)<0.001
Presence of emergency drug list22 (75.9)3 (12.5)<0.001
≥50%c of records with recorded two visits (for hypertension)11 (37.9)1 (4.2)0.003
≥50%c of records with recorded two visits (for diabetes)10 (34.4)1 (4.2)0.008b
≥50%c of records with recorded two visits (for antenatal care)5 (17.2)1 (4.2)0.2b
Waste segregation26 (89.7)6 (25)<0.001
Proper sharp disposal22 (75.9)1 (4.2)<0.001
Presence of three sterilized sets12 (41.4)7 (29.2)0.35
Employee health file21 (72.4)1 (4.2)<0.001
  • aThe units who accepted an observation tour.

  • bDone by Fisher's exact test.

  • cPercentage was calculated by checking 10 random records.

Discussion

There is little conclusive evidence that the accreditation process actually improves the quality of care offered in health care settings [46]. Reasons for the lack of evaluation research include the methodological challenges of measuring outcomes and attributing causality to these complex, changing, long-term social interventions to organizations or health systems, which themselves are complex and changing [7].

The current study aimed to evaluate a program of implementing accreditation in NGO health units in Egypt within 1 year of getting the accreditation, compared with control units matched to the socioeconomic level. Three domains were selected for the evaluation: patient satisfaction, provider satisfaction and some selected output measures of accreditation.

Measuring some outputs of the accreditation program in the health units has two objectives: the first one is to find out if there is difference between accredited and non-accredited health units regarding some standards covered in the accreditation program and the second objective was to monitor the maintenance of compliance to some of these standards after 6 months from getting the accreditation status.

Patient satisfaction

The present study showed that patients attending the accredited NGO units expressed significantly higher satisfaction scores than those attending the non-accredited units. This pattern was seen in all areas of the health service: cleanliness, waiting area, waiting time and staff performance (Table 3). Accreditation system of the primary health care in Egypt addresses certain points concerning patient rights beside other standards that may impact patient satisfaction like housekeeping standards, communication skills training and environmental safety. Indeed, the current study showed that accredited units had higher prevalence of clean toilets, appropriate furniture, analyzed patient satisfaction surveys and announced patient rights (Table 6).

In the era of patient-centered care, everyone is looking for patient (customer) satisfaction. Thus, it is quite understandable that any health care intervention seeks to monitor its effect on patient satisfaction [8, 9]. Without acceptable levels of patient satisfaction, health plans may not get full accreditation and will lack the competitive edge enjoyed by fully accredited plans [10]. The link between accreditation and patient satisfaction was addressed in some researches at the hospital level, although that is not sufficient to draw any conclusion in a systemic review [11]. Heuer [8], in his research, concluded no relation between accreditation score and patient satisfaction. Salmon et al. [9] also noted no difference in the effect of accreditation on patient satisfaction between intervention and control groups. The difference between the hospital system and primary health care is quite evident in its complexity and diversity of services: inpatient, ambulatory, diagnostic and rehabilitative. This may explain the discrepancy between satisfaction results at the hospital level and the current study.

Provider satisfaction

The relationship of providers to the health care system is a vital element in the delivery and quality of care rendered to patients [12]. Satisfaction has been identified as a key factor in determining the extent to which physicians choose to participate in certain types of programs, such as managed care [13, 14]. In addition to predicting physician participation in health plans, provider satisfaction research offers insight into how plans are functioning at the point of service [12].

The present study evaluated the provider satisfaction in accredited and non-accredited health units. It encompassed all categories working in the units: doctors, nurses, technicians, workers and administrators. The rationale of selecting all categories is that application of such a new program together with adoption of a new patient care system under the umbrella of the family doctor needs to be communicated and supported by all groups. The subjects enrolled in the intervention and control groups were comparable regarding job classification, age and gender distribution, thus removing potential confounders (Table 1).

The current study showed no significant difference in the mean satisfaction score between the intervention and control groups regarding the social environment, administrative environment and family health model (Table 5)

Overall, the providers in the accredited group showed a significantly higher satisfaction score compared with the non-accredited health units (Table 5). Accordingly, accreditation during the first months from its application may improve the providers' satisfaction not through the social or administrative environment but maybe through adjusting the health system or through otherwise unidentified aspects.

In the survey by Bates et al. [15], the exploratory factor analysis yielded four separate factors determining satisfaction among physicians: relationships with patients, autonomy in clinical decision-making, office resources and professional relationships. In the study of the impact of accreditation on the quality of hospital care in South Africa, the researchers noticed a significant improvement in nurses' perception of quality, especially clinical performance quality, but not for teamwork and cooperation and not for participation in decisions [9]. Duckett (1983) [16] noted that areas that showed the least change with hospital accreditation were those associated with the medical staff. In evaluating the accreditation program in Zambia, little difference in workers' satisfaction was seen between exposed hospitals and the control group [17]. However, Lin et al. [18] found that accreditation was one of the determinants of job satisfaction among physicians and nurses. Greenfield and Braithwaite [11] in their systemic review also confirmed this inconsistent relation between accreditation and professional attitude.

It is to be noted that satisfaction of patients has no relation with providers' satisfaction as providers and consumers of medical services evaluate the quality of services differently. Previous researches showed that surveys of providers and patients at the same clinics found little to no relationship between patient and provider satisfaction [19].

Measures of output of the accreditation program

The results of the present study showed that structural elements (like presence of a complaints box, presence of a functioning alarm system, appropriate equipments, etc.) represent the highest percentage in accredited units meaning that they are better achieved and sustained compared with patient care measures (follow-up of diabetes, hypertension and antenatal care).

The current study selected three items to measure patient care from the review of records: follow-up of diabetes, hypertension and antenatal care. Due to the short-period lag between accreditation and the survey, two visits were chosen as the minimal recording numbers in the related records. More than two-thirds of the accredited units had records with at least two visits documented in each of the three categories of patient care. Compared with the control group, the difference was highly significant. This may imply better follow-up of chronic patients and better recording. However, the main difference between accredited and non-accredited units was the maintenance of records (only 12% of the control group had records), and thus accreditation may be the principal factor underlying the better performance of the accredited units in recording chronic patients.

Salmon et al. selected hospital sanitation as one indicator to monitor the effect of accreditation. Their rationale was that cleanliness of a hospital and the availability of related supplies are important contributors to infection prevention and control efforts. Moreover, good hospital sanitation is an indicator of management effectiveness and proper allocation of resources. In their research, they concluded that accredited hospitals showed better sanitation than the control hospitals but the difference was not significant [9]. Similar results were also noted in the Zambian study [16]. These results measured sanitation in hospitals, which are by far more complex and bigger than primary health units making changes harder to be achieved and maintained.

It is concluded from the current study that accredited NGO health centers showed higher patient satisfaction compared with non-accredited health units. Overall provider satisfaction was higher in accredited health units, and accredited health units continue to comply with the accreditation standards within the first year after getting accreditation.

The present study had a limitation of unavailable pre-intervention measures (pre-accreditation) and it is recommended that future researches with controlled pre- and post-design be carried out to evaluate the effect of accreditation on the health services.

Funding

This study was carried out as part of a project funded by the Social Fund for Development in Egypt.

References

View Abstract