International Journal for Quality in Health Care 15:463-471 (2003)
© 2003 International Society for Quality in Health Care
Results of the first 100 accreditation procedures in France
Comité de Coordination de l’Evaluation Clinique et de la Qualité en Aquitaine (CCECQA), Hôpital Xavier Arnozan, 33604 Pessac Cedex, France
Purpose. To identify the areas of needed improvement that were most frequently identified in the first 100 accredited hospitals by the French Accreditation College (FAC) according to the standards manual. To compare the outcomes of accreditation procedures according to the status and size of the accredited hospitals.
Data sources. We analyzed the first 100 summaries of accreditation reports available on the website of the Agence Nationale d’Accréditation et d’Evaluation en Santé (ANAES).
Data extraction. Data were collected on hospitals, accreditation processes, and outcomes (decisions of the FAC). For each decision, we assessed the relationship with accreditation manual criteria, and analyzed their distribution by chapter.
Results of data synthesis. Among the 100 accredited hospitals (40 public, 43 private, 17 mixed), nine were accredited without recommendations for improvement, 47 with recommendations, 40 with reservations, and four with major reservations. All of them received requests for improvement. No significant difference was found concerning the FAC decisions according to status and size of hospitals, although there was a trend that the larger the hospital, the more numerous and more serious the decisions of the FAC. The main topics addressed by decisions were those given high priority by the FAC (information given to patients and its traceability on patient records, and signing of prescriptions for medication).
Conclusion. Despite wide heterogeneity in the summaries on accreditation and in FAC decision-making, this study provides an initial insight into common quality defects and ANAES priorities for hospitals in France.
Keywords: accreditation, health care, peer review, quality assurance,, quality of health care
Accepted for publication June 24, 2003.
In France, the 1996 law reforming public and private hospitalization stipulates that ‘in order to ensure continuous quality and safety improvement of health care, all public and private hospitals must submit to an external evaluation procedure named accreditation’ [1], which is implemented by the national agency for accreditation and evaluation in health care (ANAES). The French Accreditation College (FAC), a component of the ANAES, is a key element responsible for examining the survey reports, attributing accreditation, and defining recommendations for improvement. The objectives of this procedure are to assess quality and safety of care, to assess a hospital’s ability to ensure continuous improvement in quality of overall patient care, to formulate explicit recommendations, to involve professionals at all stages of the quality initiative, to provide external recognition of the quality of care in hospitals, and to improve public confidence [2]. All French hospitals had to enroll within 5 years of publication of the law on April 25, 1996. The first hospitals enrolled in the procedure in June 1999 and the first summaries of accreditation reports were available in June 2000.
An initial analysis performed on the first 20 summaries of accreditation reports showed that those standards that were most frequently identified by the FAC decisions [3,4] were: requirement for patient’s informed consent and provision to the patient of information about the benefits and risks of the planned procedures; assessment of respect for patient’s rights; prescriptions made by the prescribing practitioner, dated and containing the practitioner’s name and signature; patient care coordination between sectors of clinical activity; continuity of care; production of an annual activity report in each sector; annual assessment of staff; implementation of a system to report undesirable events; implementation of specific safety prevention programs; and all staff involvement in quality policy definition. In its 2000 annual activity report, the FAC confirmed this analysis and highlighted four topics [5]: medical prescriptions; patient information about benefits and risks of the planned procedures (and its traceability in patient records); patient record accessibility and archiving; and specific prevention programs (covering blood derivatives, adverse drug reactions, medical device safety, and risks associated with the use of parts of the human body or products derived from it), which must be operational in a hospital.
The main objective of this study was to identify the areas of needed improvement that were most frequently identified in the first 100 accredited hospitals by the FAC, according to the standards manual. The secondary objective was to compare the outcomes of accreditation procedures according to the status and size of the accredited hospitals.
|
 |
Materials and methods |
|---|
 Top Materials and methods ResultsDiscussionReferences |
|---|
Sample
We analyzed the first 100 summaries of accreditation reports made available on 29 January 2002 on the ANAES web site (http://www.anaes.fr).
Principles of the French accreditation procedure
The procedure begins with the official enrollment of the hospital, which signs a contract with the ANAES [6]. A self-assessment is performed, according to the accreditation manual published by the ANAES [2]. The manual is divided into three sections, and each is subdivided into chapters containing a set of standards and criteria ( Table 1). The self-assessment is followed by a survey visit conducted by three to six peers from the ANAES accreditation department, including at least one physician, one member of the allied health professions, and a hospital manager. Then, the FAC, which is composed of health professionals independent from the accreditation department, writes an accreditation report based on the peers’ findings, on the results of the self-assessment, and on comments supplied by the hospital after the visit. In this report, the FAC accredits the hospital according to four levels of decision: requests for improvement only, recommendations, reservations, and major reservations. The level of decision depends on the frequency and the seriousness of the departure from the standards, and on implementation of corrective measures. Finally, the accreditation report is sent to the accredited hospital and a summary is published on the ANAES website.
Data collection
In each summary, we collected data concerning hospital, accreditation processes, and decisions. Each topic addressed by the decisions was compared with the relevant criteria and standards of the accreditation manual. Therefore, the number of decisions was lower than the number of criteria and standards concerned with these decisions.
Analysis
We compared the distribution of the four grades of decision, and the duration of the accreditation process, to the status and size of the hospital (Table 2). For each grade of decision, the most frequently cited standards and criteria are presented. The number of decisions per chapter was analyzed in two ways: by raw distribution (i.e. the number of citations) and by weighted distribution (i.e. weighted by the relative number of criteria in each chapter in order to take into account the relative size of each chapter in the accreditation manual).
|
Statistical tests comprised the Pearson’s chi-square test for categorical variables and the non-parametric Mann–Whitney U-test for continuous variables. Any P-value <0.05 was considered statistically significant.
|
Results |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
Description of hospitals
The sample differed from the national distribution by overestimating the number of public hospitals (29% in France versus 40% in our sample; P < 0.03) and underestimating the number of small-sized hospitals (37% of hospitals with <100 beds in France versus 49% in our sample; P < 0.02) [15].
Decisions of the FAC
The decisions of the FAC are described in Table 2. The proportion of hospitals accredited with reservations or major reservations was greater in the public and mixed sectors and in larger hospitals, but the difference did not reach statistical significance.
Duration of the accreditation procedure
The median duration of the accreditation procedure (25th and 75th percentiles) was 7.7 months (range 5.7–10.6 months) between enrollment and survey, 7.3 months (6.0–9.3) between survey and FAC recommendations, and 2.0 months (1.3–3.2) between FAC recommendations and summary publication. The duration did not differ between hospitals with <100 beds, 100–399 beds, and 400–799 beds (15.0–15.4 months), but was longer in those with >799 beds (18.1 months; P < 0.05).
The survey visit lasted from 2 to 12 days. The median number of peer-visit days (number of days multiplied by number of peers) was 12 (range 6–74).
Major reservations formulated by the FAC
Major reservations were applied to four hospitals. There was only one major reservation in three hospitals, and two major reservations in one hospital. These concerned the absence of implementation of the recommendations made by external safety control organizations (in two hospitals), insufficient safety in operating theaters, and the declaration of all computer files with nominative data to the National Commission for Computers and Freedoms (CNIL) (whose role is to protect the private lives and the personal and public freedoms of individuals with regard to electronic data).
Reservations formulated by the FAC
Reservations were applied to 44 hospitals. The number of reservations applied to each hospital varied from one to eight (median of two). A total of 97 reservations were formulated, involving 171 criteria and standards in the accreditation manual.
The raw distribution of reservations showed that the chapters most frequently cited in the reservations were (in descending order): patient records [44 (26%) of reservations]; monitoring, prevention, and control of the risk of infection [28 (17%)]; quality management and risk prevention [27 (16%)]; specific prevention programs and transfusion safety [18 (11%)]; and organization of patient care [18 (11%)] (Figure 1). According to the weighted results, the chapters on patient records, quality management and risk prevention, specific prevention programs and transfusion safety, and monitoring, prevention, and control of the risk of infection were those most frequently cited (Figure 1).
|
‘Clear identification of the prescribing practitioner’ in the patient records chapter was the most cited criterion (DPA4c, 16 hospitals). The next most frequently cited criteria are presented in Table 3.
|
Recommendations formulated by the FAC
The number of recommendations related to standards or criteria in the accreditation manual varied from one to 17 in the 91 hospitals that received recommendations (median of five). A total of 499 recommendations were formulated, involving 648 standards and criteria in the accreditation manual.
The raw distribution of recommendations showed that the chapters most frequently cited in the recommendations were (in descending order): organization of patient care [118 (18%) of recommendations]; patient records [84 (13%)]; management of general support services [82 (13%)]; patient rights and information [70 (11%)]; and management of human resources [69 (11%)] (Figure 2). According to the weighted results, the chapters on patient records, quality management and risk prevention, management of human resources, management of general support services, and specific prevention programs and transfusion safety were those most frequently cited (Figure 2).
|
The criterion most cited in the recommendations was GRH7a, ‘conducting an annual interview between each member of staff and the managerial staff ’, in the chapter on management of human resources (21 hospitals). Other frequently cited criteria are presented in Table 4.
|
Requests for improvement formulated by the FAC
The number of requests for improvement related to standards or criteria in the accreditation manual varied from eight to 92 in the 100 hospitals. All were subject to requests for improvement (median 27). A total of 3025 requests for improvement were issued, involving 3317 standards and criteria in the accreditation manual.
The raw distribution of requests for improvement showed that the chapters most frequently cited in requests for improvement were (in descending order): organization of patient care [558 (17%) of the requests for improvement]; management of general support services [368 (11%)]; patient rights and information [355 (11%)]; management of human resources [333 (10%)]; and patient records [324 (10%)] (Figure 3). According to the weighted results, the chapters on management of the information system, patient records, quality management and risk prevention, specific prevention programs and transfusion safety, and management of human resources were those most frequently cited (Figure 3).
|
The standards most often cited were those concerning the policy defined by the hospital and/or activity sector to address the standard (the first standard in each chapter). These represented four of the 18 standards most cited in requests for improvement.
The last standards in each chapter, which concern evaluation of the results concerning continuous quality improvement and the levels of achievement with respect to the objectives set by the hospital, were differentially cited in requests for improvement. These standards were frequently cited only in the chapters on patient rights and information (DIP) and patient records (DPA), and, to a lesser extent, in the chapters on infection risk prevention (SPI), management of general support services (GFL), and information systems (GSI).
The criteria most cited in requests for improvement concerned the assessment of respect for patient rights (DIP9a; 43 hospitals), the involvement of all activities in the information system (GSI1b; 37 hospitals), the assessment of general support services such as catering, laundry, security, and transportation (GFL10; 36 hospitals), the requirement for informed consent and information about the benefits and risks of the planned procedures (DIP5a; 34 hospitals), the assessment of the quality of patient records (DPA7a; 34 hospitals), the up-to-date description of job profiles (GRH4c; 33 hospitals), and the implementation of a document management system (QPR3f; 33 hospitals).
|
Discussion |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
This review pinpoints topics frequently cited in accreditation decisions taken by the FAC. Two are close to those defined as priorities by the FAC in its 2000 annual activity report [5]: information to patients about the benefits and risks of the planned procedures, and its traceability in patient records, and clear identification of the practitioner issuing the prescription. The other two priorities defined by the ANAES (patient record accessibility and archiving, and specific prevention programs which must be operational) were less frequently cited. Other topics outside the priorities of the ANAES appeared frequently: the assessment of the quality of patient records; the implementation of operational procedures, notably prescription rules between the pharmacy and the clinical activity sectors; the drafting and up-to-date description of job profiles; the implementation of a system to report undesirable events; the involvement of all activities in the information system (especially patient records); the conducting of an annual interview between each member of staff and the managerial staff; the assessment of the general support services; and the implementation of a document management system.
Final outcomes of accreditation programs have rarely been reported [7–9], but studies on the intermediate outcomes of accreditation have been published in the United States [10,11], Australia [12,13], and Catalonia [14]. By comparing the intermediate outcomes of hospital accreditation in United States with those in France [10], it is striking that the performance areas that received a low score of compliance (3–5) from the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) were frequently cited in the reservations or recommendations made by the FAC. This was notably the case for ‘medication use’ (FAC decisions in about half of the hospitals), ‘availability of patient-specific information’ (FAC decisions in 25% of hospitals), ‘assessing staff competence’ (FAC decisions in 20% of hospitals), ‘implementation of safety plans’ (FAC decisions in nearly 20% of hospitals), ‘infection control’ (FAC decisions in 15% of hospitals), and ‘credentialing’ (FAC decisions in 10% of hospitals). Other performance areas frequently found to be either partially or non-compliant in the United States—‘initial assessment of patients’ and ‘design of the environment’—were rarely cited in the FAC decisions. On the other hand, the areas concerning patient records (archiving, confidentiality, assessment of quality) and the assessment of patient rights, commonly cited in the FAC decisions, were rarely highlighted by the JCAHO.
The main limitations of this review are the analysis of the summary reports and the limited number of accreditation reports as yet available in France. Errors of judgment in the correspondence between decisions found in the reports, and criteria or standards in the accreditation manual may have occurred, since our work was based on the summaries of accreditation reports. The limited sample size hampered the power of statistical analysis. The close distribution of hospital status in our sample differed from national figures by including a larger proportion of public hospitals and the largest hospitals [15], and the sample also comprised volunteer hospitals able to enroll quickly in the accreditation procedure. This may reflect a higher degree of investment in the continuous quality improvement process.
The content of the summaries of accreditation reports was heterogeneous. Firstly, the length of the summaries varied from 20 to 50 pages and was not related to the results. Moreover, the number of decisions varied substantially. For example, the number of requests for improvement varied in the range of one to four between hospitals with an identical number of reservations and recommendations. This range is not likely to reflect true differences between hospitals, i.e. ‘the health care organization’s ability to ensure continuous improvement in quality of overall patient care’, which is the main objective of the accreditation procedure [2]. In some summaries, a decision was formulated for each specific criterion in the accreditation manual, whereas another concerned improvement plans, thus covering a large number of criteria. For reasons of equity, this heterogeneity should be reduced in the future.
The initial trends observed in this study show that the largest hospitals received more high-grade decisions. According to the ANAES statement that the accreditation procedure is a process for assessing ‘quality and safety of care’ [2], this finding calls into question the widespread opinion that the quality and safety of care is higher in reference hospitals, such as teaching hospitals, than in small hospitals [16,17]. Indeed, seven of the nine teaching hospitals in our sample had reservations or major reservations (78%). Do these differences reflect a selection bias or do they result from unfair assessment? Although these findings in part reflect the real situation, we must bear in mind that the French accreditation procedure investigates macro-processes, not outcomes. Until links between clinical processes and outcomes are studied further [18], we lack information about the relationship between these macro-processes and outcomes. Herein lies an area of research that might even question the overall effectiveness and efficiency of the accreditation process [19].
|
Footnotes |
|---|
Address reprint requests to Valentin Daucourt, Comité de Coordination de l’Evaluation Clinique et de la Qualité en Aquitaine (CCECQA), Hôpital Xavier Arnozan, 33604 Pessac Cedex, France. E-mail: valentin.daucourt{at}ccecqa.asso.fr
An abstract of this study was selected as a poster presentation for the Annual Conference of the International Society for Quality in Health Care in Paris, France, November 6–8, 2002.
|
References |
|---|
|
TopMaterials and methodsResultsDiscussionReferences |
|---|
- Employment and Social Affairs Ministry. Ordonance No. 96-346 on April 24th, 1996, reforming public and private hospitalization. Official Journal of the Employment and Social Affairs Ministry 1996; April 25th: 6324–6336 (available online at http://droit.org/jo/19960425/TASX9600043R.html).
- The National Agency for Accreditation and Evaluation in Health Care (ANAES). Accreditation Manual (English version). Paris: ANAES, 1999 (available online at http://www.anaes.fr/anaes/Publications.nsf/nPDFFile/AT_LILF-4HHKSU/$File/Manual.pdf?OpenElement).
- Daucourt V, Michel P. Analysis of the summaries of accreditation reports. Principles, methods, results. Gestions Hospitalières 2001; 409: 542–550.
- Daucourt V, Michel P. Accreditation: an initial analysis. Concours médical 2001; 123: 1618–1621.
- The National Agency for Accreditation and Evaluation in Health Care (ANAES). 2000 Annual Activity Report of the French Accreditation College. Paris: ANAES, 2001 (available online at http://www.anaes.fr/ANAES/Publications.nsf/b3dfd2cc3f399b37c125658d004132b9/2baeff50d2ece78cc1256b35004887be?OpenDocument ).
- Giraud A. Accreditation and the quality movement in France. Qual Health Care 2001; 10: 111–116.
[Free Full Text] - Duckett SJ. Changing hospitals: the role of hospital accreditation. Soc Sci Med 1983; 17: 1573–1579.[CrossRef][Web of Science][Medline]
- Dean Beaulieu N, Epstein AM. National Committee on Quality Assurance health-plan accreditation: predictors, correlates of performance, and market impact. Med Care 2002; 40: 325–337.[CrossRef][Web of Science][Medline]
- Beaumont M. Canadian Council on Health Services Accreditation (CCHSA). Effectiveness and role in CQI evolution. Paper presented at the Annual Conference of the International Society for Quality in Health Care, November 6–8, 2002, Paris.
- Joint Commission on Accreditation of Healthcare Organizations. 2000 Hospital Performance Report (available online at http://www.jcaho.org/qualitycheck/directry/directry.asp).
- Leatherman S, McCarthy D. Public disclosure of health care performance reports: experience, evidence and issues for policy. Int J Qual Health Care 1999; 11: 93–98.
[Free Full Text] - Amos BJ. Medical problems in hospitals: what has the Australian Council on Hospital Standards discovered? Aust Clin Rev 1985; 16: 47–49.
- Collopy BT. Extending facility accreditation to the evaluation of care: the Australian experience. Int J Health Plann Manage 1995; 10: 223–229.[Medline]
- Bohigas L, Asenjo MA. Hospital accreditation in Catalunya: an assessment of the performance in quality of hospitals. Int J Health Plann Manage 1995; 10: 201–208.[Medline]
- Employment and Solidarity Ministry, Direction of hospitals. Hospitalization in France: Marking Data and Digits. Report No. 53. Paris: Employment and Solidarity Ministry, March 2000 (available online at http://www.sante.gouv.fr/htm/publication/dhos/chiffr_rep/chiff_rep.htm).
- Keeler EB, Rubenstein LV, Kahn KL et al. Hospital characteristics and quality of care. J Am Med Assoc 1992; 268: 1709–1714.
[Abstract/Free Full Text] - Polanczyk CA, Lane A, Coburn M, Philbin EF, Dec GW, DiSalvo TG. Hospital outcomes in major teaching, minor teaching, and nonteaching hospitals in New York state. Am J Med 2002; 112: 255–261.[CrossRef][Web of Science][Medline]
- Dedier J, Singer DE, Chang Y, Moore M, Atlas SJ. Processes of care, illness severity, and outcomes in the management of community-acquired pneumonia at academic hospitals. Arch Intern Med 2001; 161: 2099–2104.
[Abstract/Free Full Text] - Collopy BT. Clinical indicators in accreditation: an effective stimulus to improve patient care. Int J Qual Health Care 2000; 12: 211–216.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Greenfield and J. Braithwaite Health sector accreditation research: a systematic review Int. J. Qual. Health Care, June 1, 2008; 20(3): 172 - 183. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. D. Shaw Evaluating accreditation Int. J. Qual. Health Care, December 1, 2003; 15(6): 455 - 456. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||