OUP user menu

A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?

Cyrille Delpierre , Lise Cuzin , Judith Fillaux , Muriel Alvarez , Patrice Massip , Thierry Lang
DOI: http://dx.doi.org/10.1093/intqhc/mzh064 407-416 First published online: 16 September 2004

Abstract

Purpose. To analyse the impact of computer-based patient record systems (CBPRS) on medical practice, quality of care, and user and patient satisfaction.

Data sources. Manual and electronic search of the Medline, Cochrane, and Embase databases.

Study selection. Selected articles were published from 2000 to March 2003. CBPRS was defined as computer software designed to be used by clinicians as a direct aid in clinical decision making. To be included, the systems should have recorded patient characteristics and offered online advice, or information or reminders specific to clinicians during the consultation.

Data extraction. Keywords used for the search were: electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation.

Results. Twenty-six articles were selected. Use of a CBPRS was perceived favourably by physicians, with studies of satisfaction being mainly positive. A positive impact of CBPRS on preventive care was observed in all three studies where this criterion was examined. The 12 studies evaluating the impact on medical practice and guidelines compliance showed that positive experiences were as frequent as experiences showing no benefit. None of the six studies analysing the impact of CBPRS on patient outcomes reported any benefit.

Conclusions. CBPRS increased user and patient satisfaction, which might lead to significant improvements in medical care practices. However, the studies on the impact of CBPRS on patient outcomes and quality of care were not conclusive. Alternative approaches considering social, cultural, and organizational factors may be needed to evaluate the usefulness of CBPRS.

  • computer-based patient record system
  • evaluation
  • medical practice
  • patient outcomes
  • quality of care
  • randomized controlled trials
  • systematic review

The evolution of medicine during the three past decades is characterized by a paradox. Medical information needed for clinical decision making has increased; however, the organization and accessibility of health data are still poor, resulting in inappropriate decisions and medical errors [1,2]. To increase the accessibility and management of medical information, the use of informatic tools has been promoted [3]. First used for management and administrative purposes, computer-based patient record systems (CBPRS) have been developed to collect and synthesize medical information [2].

Many experiences have been published regarding general practice [4], surgery [5], cardiology [6], psychiatry [7], or HIV infection [8,9] based on electronic medical records. These systems have common characteristics [10,11]. Medical data are conceptually organized as the patient’s paper medical record, and synthesize clinical and therapeutic patient history. CBPRS are designed to be used directly by physicians during consultation and to provide online information and messages to help physicians in their practice.

The aim of CBPRS is to offer support in medical decision making, to increase coordination between different health care providers, and to promote the use of guidelines, thereby improving global quality of care [12,13]. Development of CBPRS in a care unit has even been proposed as a criterion of quality [14,15]. The aim of CBPRS is also: to improve the speed of retrieval of medical records, allowing many persons to have simultaneous access to the same medical record; to improve data confidentiality while tracing who has accessed it; and finally to collect routine data [1618].

During the past three decades, the introduction and development of CBPRS in the health sector has followed the development of data processing [19]. However, few systems have been evaluated with regard to their impact on clinical performance and patient outcomes [2,2022]. Results from the few studies that evaluated CBPRS were not concordant. A systematic review that analysed 28 randomized controlled trials (RCTs) from 1983 to 1992 concluded that compliance with guidelines increased; however, patient outcomes were unchanged [20]. Another systematic review, which analysed 25 studies from 1992 to 1998, confirmed the positive effect of these tools’ compliance with guidelines, but was inconclusive on patient outcomes and quality of care due to lack of data [13]. The last systematic review, which analysed studies until 1999, confirmed these results and concluded that new approaches were needed to evaluate the impact of these tools on medical practice and quality of care [23]. Several studies since then have tried to fill this gap. Our objective was to carry out a systematic review of studies analysing the impact of CBPRS on medical practice, quality of care, and user and patient satisfaction, using papers published since 2000.

Methods

Study selection

Studies published between January 2000 and March 2003 were identified by searching the US National library of Medicine Medline electronic bibliographic database, and the electronic Cochrane and Embase databases. A manual search of studies most frequently cited in automatically selected studies was also performed. The keywords used for the search were electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation.

To be selected, studies had to pertain to an evaluation of the impact of CBPRS on medical practice and/or quality of care and/or user satisfaction and/or patient satisfaction.

CBPRS was defined as follows: ‘computer software designed to be used by a clinician involved in patient care as a direct aid to clinical decision making. Patient data were recorded into the system. Patient-specific information in the form of assessments or recommendations, or alerts or reminders was presented to the clinician during the consultation’.

Descriptive studies and analyses concerning the quality of CBPRS, such as accuracy or completeness of data, were not included. No criteria with respect to trial methodology were defined.

Results

Thirty-nine articles were identified at the end of this process. After careful analysis, 13 of them were excluded: eight did not evaluate the impact of CBPRS on clinician performance and five did not meet the criteria for the definition of a CBPRS. Eventually, 26 articles were selected, corresponding to 25 studies. One of them was published in two papers [24,25]. Selected studies are presented in Tables 14.

View this table:
Table 1

Randomized controlled trials studies

View this table:
Table 2

Before–after studies

View this table:
Table 3

Cross-sectional studies

View this table:
Table 4

Qualitative studies

Study designs

Several study designs were used: 10 were RCTs [2432], 11 were before–after studies [3343], three were cross-sectional studies [4446], and two were based on qualitative interviews [10,47].

Criteria of judgment

Sixteen studies attempted to assess the process of care. Five studies used patient outcomes as a criterion of judgment. Fifteen studies used satisfaction among users and/or patients as the endpoint.

Process of care

Length of the consultation. Six studies analysed the impact of CBPRS on the length of the consultation [31,33,37,39,41,45]. An increase in this length was observed in three studies [31,33,45]. This increase was found to range between 2.2 and 9.3 minutes per patient [31,33]. Makoul et al. [45] found discordant results according to the variety of consultation. Initial visits had a mean length of 35 minutes when using CBPRS compared with 25 minutes without (P < 0.05). No difference was observed during subsequent visits. The length of the consultation was found to be reduced in one study [41] and unchanged in another study [37].

Content of the consultation. Three studies analysed the impact of CBPRS on the content of the consultation [31,33,45]. Some variables, such as questions regarding mental health or behaviour such as tobacco consumption were more frequently recorded as items of the CBPRS than when these tools were not used [33,45]. Clinicians were significantly more likely to address routine health care maintenance topics, including psychosocial issues [relative risk (RR) = 1.4] and smoking at home (RR = 15.6) in a paediatric primary care practice using CBPRS than in one using paper records [33].

Clinical performance

Twelve studies analysed modification of medical practice [2530,3336,40,48]. Three studies analysed the impact of CBPRS on preventive care [26,28,33], and an improvement was observed in all of them. Dexter et al. [26] analysed pneumococcal and influenza vaccination, and prophylactic heparin and aspirin use. An increase was observed in vaccination rates from 0.8% to 35.8% for pneumococcal vaccination, and from 1% to 51.4% for influenza vaccination. Likewise, prophylactic heparin and aspirin prescription at discharge increased from 18.9% to 32.2%, and from 27.6% to 36.4%, respectively.

Eight studies analysed compliance with guidelines [25, 2729,35,36,48]. An increase in the application of guidelines was observed in three studies of the management of breast cancer, venous thromboembolism, and blood test ordering [35,36, 48]. This increase was 14% (85% versus 62%, respectively; P < 10−4) with respect to breast cancer guidelines and 12% (95% versus 83%, respectively; P < 0.001) with respect to venous thromboembolism guidelines [35,36]. In a study of blood test ordering, general practitioners who used CBPRS requested 20% fewer blood tests [48]. Results regarding the management of acute respiratory distress syndrome showed that 95% of computer-generated instructions were followed [29]. However, compliance with guidelines on the management of asthma, angina, diabetes mellitus, and major depression was not found to increase [25,27,28].

Nine studies analysed the impact of CBPRS on treatment prescriptions [25,27,30,3436,40,43,48]. Prescription rates concordant with recommendations increased by 31% for chemotherapy prescriptions and by 61% for prescriptions for venous thromboembolism. Accordingly, the error rate decreased [35,36]. Improvements in appropriateness for prescription of drugs with CBPRS was reported in one study [43]. No significant evolution could be observed in the remaining five studies [25,27,30,34,40].

Patient outcomes

Five studies used patient outcomes as a criterion of judgment [25,2730]. The results were not improved for cardiovascular risk [28,30], survival, length of hospitalization, or acute respiratory distress syndrome [29]. However a significant decrease of 2.3 mmHg diastolic pressure was observed in one study [28]. In two studies, outcomes were assessed by the patients. No benefit with respect to the management of asthma and angina, or major depression was observed [25,27].

User and patient satisfaction

Physician satisfaction. Thirteen studies analysed physician perception for CBPRS use [10,28,3133,3741,4446]. Physician satisfaction with the system was reported in nine studies [10,31,33,3941,4446]. The positive points reported were improved knowledge of patients’ medical history, better medical examination, and improvement in quality of care. In four studies, physicians were asked if they wanted to continue working with CBPRS: all studies reported positive responses [31,39,40,46]. Nevertheless, four studies raised some points of dissatisfaction [28,32,37,38]. CBPRS was perceived as a physical barrier [33,37,38,45] that could have a negative impact on the patient–physician relationship [37,38,46], particularly by reducing eye-to-eye contact [33]. Concerns were expressed about data confidentiality [3739], personal and professional privacy, bug management [32,38,44], and the additional work for physicians [28,31,32]. Factors influencing perception of the system were its characteristics, such as its interface and ease of use, as well as users’ characteristics. Perception of CBPRS was better for community-based physicians than for academic-based physicians. Physicians accustomed to working with a computer had a better perception of CBPRS [46].

Nurse satisfaction. Three studies analysed nurse satisfaction with the introduction of CBPRS in their practice [32,40,47]. A global increase in satisfaction was observed in two studies [32,40]. In another study, reduced administrative work and increased accessibility to care protocols, especially for young nurses, were reported. Negative points were the lack of flexibility of these tools, the loss of nurses’ judgment, and the additional workload [47]. Nevertheless, nurses stated their desire to continue working with an informatic system in all three studies.

Patient satisfaction. Two studies analysed patient satisfaction [37,49], and patients were found to have a positive opinion of CBPRS in both. A mean score of 4.6 on a general satisfaction scale of 0–5 was reported [37]. Patients did not report a reduction in eye-to-eye contact with the use of CBPRS, and found medical visits more effective. However, fear concerning data confidentiality was observed in both studies.

Discussion

Use of a CBPRS seems to be perceived favorably by physicians, with user satisfaction being mainly positive despite some limits and concerns regarding its use [31,39,40,46]. The main constraints reported were the physical barrier and the impact on the patient–physician relationship that could result from the use of CBPRS during the consultation. Technical characteristics of the system and knowledge of data processing by physicians were criteria related to satisfaction with, and thus implementation of, CBPRS [50]. Future research is needed considering qualitative factors that could influence the use of CBPRS [7,51].

Physicians wanted to continue working with CBPRS once such tools were in place in their wards [31,39,40,46], as once exposed, physicians felt a need for an information system. To improve the quality of CBPRS it is essential that the quality of data collected is certified and controlled [52]. Furthermore, to prepare for technical problems with the computer it is advisable always to keep a paper print-out of patient records [53].

Only two studies analysed patient satisfaction. Generally, patients were satisfied and did not complain about changes to their relationship with their physicians [37]. Their greatest fear concerned data confidentiality. It is therefore essential to reassure patients about the confidentiality of their data. Physicians must be able to explain how data confidentiality can be maintained with the system they use. Additional patient-focused studies are thus necessary to explore their feelings about the use of CBPRS in consultation rooms and its consequences with respect to the patient–physician relationship.

The impact of CBPRS on medical practice was more balanced. A clear positive impact of CBPRS on preventive care was noted. This finding is consistent with other systematic reviews [13,20,54]. Improvements in medical practice and the adoption of guidelines was less certain. Positive experiences were as frequent as experiences showing no benefit. In studies of arterial hypertension and major depression [25,30,5558], there was no improvement in medical practice and compliance with guidelines. The recent systematic review of Kaushal et al. showed that CBPRS could decrease prescription error, although most studies were inconclusive [59]. Possible reasons for these inconclusive findings were underuse of the tool, inadequate study design, and lack of study power [28].

Only six studies analysed the impact of the use of CBPRS on patient outcomes and did not show any benefit of CBPRS. In a systematic review [13], 25 RCTs evaluating the contribution of CBPRS to medical practice were analysed. Only eight of these studies used patient outcomes as a criterion of judgment. The relationship between the process of care and health outcomes might explain why improved outcomes are difficult to relate to the implementation of CBPRS [60].

Users of CBPRS acknowledge the usefulness of such tools, whereas the results of clinical performance or patient outcomes are not always conclusive. This paradox draws the attention to the criteria and methodologies used in various studies. RCTs are usually considered the methodological reference [61]. However, this methodological approach has several limits in evaluating the impact of CBPRS on medical practice. In order to organize RCTs and select a criterion of judgment, tracers have to be chosen within complex and multifactorial diseases (e.g. hypertension, depression). Moreover, even if positive results are found in some narrow clinical fields, generalization to the more complex clinical situations is generally difficult [62]. Controlled conditions required by RCTs do not reflect the diversity of medical practices and working conditions. Due to these limitations, RCTs do not offer a sound enough methodology to evaluate the contribution of CBPRS in medical practice, and its impact on clinical performance and patient outcomes [61,63,64].

Two systematic reviews [23,65] describing methodological approaches to the evaluation of systems designed to improve medical decision making emphasized the lack of diversity in the design of studies that evaluate these tools. This phenomenon could lead to lack of conclusive results. Alternatives approaches centered on social, cultural, and organizational factors are thus advocated. Structure of care (general practice, hospital), organization of the ward, and the relationships between various health professionals seem to be key factors to consider when evaluating the impact of CBPRS on medical practice [65].

It is recognized that systematic review may be subject to a possible selection bias. Studies presenting negative findings are indeed published less often [13]. Nevertheless, a high proportion of articles showing no effect of CBPRS on quality of care were found [25,27,28]. A wide range of keywords to find systems with the same characteristics had to be used. A well defined operating definition of CBPRS, sharing the same characteristics, could facilitate future research and produce more exhaustivity [66].

Conclusion

This systematic review, based on 25 papers published between 2000 and 2003, has contributed to a better understanding of the relationship between the use of CBPRS and medical practice. Increased satisfaction of users and patients was noted, which could lead to significant changes in medical practice. However, as in previous reviews, the impact of CBPRS on medical practice and quality of care was not well demonstrated. It is noteworthy that most of the studies did not include qualitative factors such as characteristics of the disease and the tool, the ward in which it is developed, and the relationship between various health care professionals, which can impact upon the use of CBPRS.

A broad review including all the factors that could influence the success or failure of the use of CBPRS in medical practice is indicated in the future.

References

View Abstract