OUP user menu

A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?

Cyrille Delpierre, Lise Cuzin, Judith Fillaux, Muriel Alvarez, Patrice Massip, Thierry Lang
DOI: http://dx.doi.org/10.1093/intqhc/mzh064 407-416 First published online: 16 September 2004

Abstract

Purpose. To analyse the impact of computer-based patient record systems (CBPRS) on medical practice, quality of care, and user and patient satisfaction.

Data sources. Manual and electronic search of the Medline, Cochrane, and Embase databases.

Study selection. Selected articles were published from 2000 to March 2003. CBPRS was defined as computer software designed to be used by clinicians as a direct aid in clinical decision making. To be included, the systems should have recorded patient characteristics and offered online advice, or information or reminders specific to clinicians during the consultation.

Data extraction. Keywords used for the search were: electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation.

Results. Twenty-six articles were selected. Use of a CBPRS was perceived favourably by physicians, with studies of satisfaction being mainly positive. A positive impact of CBPRS on preventive care was observed in all three studies where this criterion was examined. The 12 studies evaluating the impact on medical practice and guidelines compliance showed that positive experiences were as frequent as experiences showing no benefit. None of the six studies analysing the impact of CBPRS on patient outcomes reported any benefit.

Conclusions. CBPRS increased user and patient satisfaction, which might lead to significant improvements in medical care practices. However, the studies on the impact of CBPRS on patient outcomes and quality of care were not conclusive. Alternative approaches considering social, cultural, and organizational factors may be needed to evaluate the usefulness of CBPRS.

  • computer-based patient record system
  • evaluation
  • medical practice
  • patient outcomes
  • quality of care
  • randomized controlled trials
  • systematic review

The evolution of medicine during the three past decades is characterized by a paradox. Medical information needed for clinical decision making has increased; however, the organization and accessibility of health data are still poor, resulting in inappropriate decisions and medical errors [1,2]. To increase the accessibility and management of medical information, the use of informatic tools has been promoted [3]. First used for management and administrative purposes, computer-based patient record systems (CBPRS) have been developed to collect and synthesize medical information [2].

Many experiences have been published regarding general practice [4], surgery [5], cardiology [6], psychiatry [7], or HIV infection [8,9] based on electronic medical records. These systems have common characteristics [10,11]. Medical data are conceptually organized as the patient’s paper medical record, and synthesize clinical and therapeutic patient history. CBPRS are designed to be used directly by physicians during consultation and to provide online information and messages to help physicians in their practice.

The aim of CBPRS is to offer support in medical decision making, to increase coordination between different health care providers, and to promote the use of guidelines, thereby improving global quality of care [12,13]. Development of CBPRS in a care unit has even been proposed as a criterion of quality [14,15]. The aim of CBPRS is also: to improve the speed of retrieval of medical records, allowing many persons to have simultaneous access to the same medical record; to improve data confidentiality while tracing who has accessed it; and finally to collect routine data [1618].

During the past three decades, the introduction and development of CBPRS in the health sector has followed the development of data processing [19]. However, few systems have been evaluated with regard to their impact on clinical performance and patient outcomes [2,2022]. Results from the few studies that evaluated CBPRS were not concordant. A systematic review that analysed 28 randomized controlled trials (RCTs) from 1983 to 1992 concluded that compliance with guidelines increased; however, patient outcomes were unchanged [20]. Another systematic review, which analysed 25 studies from 1992 to 1998, confirmed the positive effect of these tools’ compliance with guidelines, but was inconclusive on patient outcomes and quality of care due to lack of data [13]. The last systematic review, which analysed studies until 1999, confirmed these results and concluded that new approaches were needed to evaluate the impact of these tools on medical practice and quality of care [23]. Several studies since then have tried to fill this gap. Our objective was to carry out a systematic review of studies analysing the impact of CBPRS on medical practice, quality of care, and user and patient satisfaction, using papers published since 2000.

Methods

Study selection

Studies published between January 2000 and March 2003 were identified by searching the US National library of Medicine Medline electronic bibliographic database, and the electronic Cochrane and Embase databases. A manual search of studies most frequently cited in automatically selected studies was also performed. The keywords used for the search were electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation.

To be selected, studies had to pertain to an evaluation of the impact of CBPRS on medical practice and/or quality of care and/or user satisfaction and/or patient satisfaction.

CBPRS was defined as follows: ‘computer software designed to be used by a clinician involved in patient care as a direct aid to clinical decision making. Patient data were recorded into the system. Patient-specific information in the form of assessments or recommendations, or alerts or reminders was presented to the clinician during the consultation’.

Descriptive studies and analyses concerning the quality of CBPRS, such as accuracy or completeness of data, were not included. No criteria with respect to trial methodology were defined.

Results

Thirty-nine articles were identified at the end of this process. After careful analysis, 13 of them were excluded: eight did not evaluate the impact of CBPRS on clinician performance and five did not meet the criteria for the definition of a CBPRS. Eventually, 26 articles were selected, corresponding to 25 studies. One of them was published in two papers [24,25]. Selected studies are presented in Tables 14.

View this table:
Table 1

Randomized controlled trials studies

AuthorsNature of the tested systemName of systemParticipants (n)Length of trialDomainJudgment criteriaResults
Dexter et al. (2001) [26]In-house developedCPOE6371 patients, 202 physicians18 monthsPreventive care for hospitalized patientsRates of use of four preventive therapies: pneumococcal vaccination; influenza vaccination; prophylactic heparin; and prophylactic aspirin at dischargeSignificant increase in the rate of delivery of such therapies (P < 0.001)
Eccles et al. (2002) [27]CommercialCDSS2276 patients2 years: 1 year before and 1 year after the interventionManagement of asthma and anginaCompliance with the guidelinesNo change in the management of asthma and angina
Hetlevik et al. (2000) [28]In-house developedCDSS1034 patients, 53 physicians18 monthsTreatment of diabetes mellitus in general practiceFractions of patients without registration; mean group differences for the same variablesNo significant change in doctors’ behaviour or in patient outcome
McKinley et al. (2001) [29]In-house developedCDSS67 patients63 monthsTrauma intensive care unitPhysician acceptance; survival; length of ICU stay; morbidity; barotraumaPositive effects on FiO2 and P plateau; survival, ICU length of stay, morbidity, barotrauma not significantly different
Montgomery et al. (2000) [30]In-house developedCDSS614 patients, 74 general practitioners, 11 nurses2 yearsManagement of hypertension in primary carePercentage of patients with a 5-year cardiovascular risk ≥10%; systolic blood pressure; diastolic blood pressure; prescribing of cardiovascular drugsNo benefit to absolute risk reduction or blood pressure control
Overhage et al. (2001) [31]In-house developedCPOE744 patients, 34 physicians3 different days over a 16-months periodPhysician time utilization in ambulatory primary care internal medicine practicePhysician time utilization; physician opinionLittle extra time (2.2 minutes per patient); patient care improved
Rollman et al. (2001) [24]CommercialEMR216 patients, 16 physicians20 monthsManagement of major depressionResponse to alert; agreement with diagnosis; treatmentChange in management of major depression
Rollman et al. (2002) [25]CommercialEMR200 primary care patients, 17 physiciansTwo phases: 3 and 6 monthsManagement of major depressionClinical outcomes and care processes for patients with major depressionLittle differential impact on clinical outcomes or on process measures
Van Wijk et al. (2001) [48]CommercialCDSS60 general practitioners11 monthsPrimary careAverage number of blood tests ordered per order form per practice20% fewer tests on average
  • CPOE, computer patient order entry; CDSS, computer decision support system; EMR, electronic medical record; ICU, intensive care unit.

View this table:
Table 2

Before–after studies

AuthorsNature of the tested systemName of systemParticipants (n)Length of trialDomainJudgment criteriaResults
Adams et al. (2003) [33]CommercialEMR1221 patients2 months pre-intervention period, 6 months post-intervention periodPaediatric primary careHealth history; risk assessment; physical examination; anticipatory guidance; immunizations; hearing; vision; lead; anaemia; tuberculosis testing; users’ satisfactionBenefits of EMR on clinical practice; high satisfaction
Bansal et al. (2001) [34]In-house developedCPOENot documented12 weeks: 5 pre-intervention and 7 post-interventionIntensive care unitNumber of arterial blood gasNo significant change
Bouaud et al. (2001) [35]In-house developedCDSS13 physicians, 127 patients4 monthsManagement of breast cancerTreatment decision; physician prescribing behaviourSignificant increase in physician compliance with guidelines (85% versus 61.4%, P < 10−4)
Durieux et al. (2000) [36]In-house developedCDSS1971 patientsThree 10 week intervention periods, four 10 week control periods, 4 week washoutPrevention of venous thromboembolism in orthopedic surgery departmentProportion of appropriate prescriptions ordered for anticoagulationSignificant increase in guidelines compliance with CDSS (94.9% versus 82.8%, P < 10−4); 73% reduction in risk of inappropriate prescription
Gadd et al. (2000) [37]CommercialEMR28 physiciansPhysicians’ and patients’ interview before implementation, after 6 months, and at 1 yearPhysical medicine and rehabilitationPhysician concerns regarding implementation of an EMR in an outpatient clinic; patient satisfactionSignificant negative effects on: patient privacy; overall quality of health care; physician autonomy
Gadd et al. (2001) [38]CommercialEMR170 physiciansPhysicians’ interview before implementation and after 6 monthsPhysical medicine and rehabilitationPhysician attitudesSignificant decrease in overall optimism with regard to EMR
Mikulich et al. (2001) [39]In-house developedEMR142 physicians, 789 patients3 yearsManagement of chronic back painPhysician attitudes; family medicine; patient complaintsIncrease in compliance with guidelines. Considerable heterogeneity in belief among complaints
Penrod et al. (2001) [46]CommercialEMR34 physicians6 monthsCommunity- and academic-based primary careAttitudes of academic-based and community-based primary care physicians; factors influencing their attitudesCommunity-based physicians more satisfied by EMR
Rocha et al. (2001) [40]In-house developedCDSSNot documentedEvaluation before implementation and after 6 monthsManagement of nosocomial infectionsPhysician satisfaction; number of suggestions to treatNo significant change in treatment strategies; significant increase in physicians’ satisfaction
Rousseau et al. (2003) [32]CommercialCDSS47 physicians, 6 nurses2 yearsManagement of asthma and anginaUser satisfactionNegative perceptions by physicians
Sanders et al. (2001) [42]In-house developedCPOENot documented9-week control period, 8-week intervention periodNeuroradiology imagingClinician ordering patternsChange in the distribution of tests ordered; significant increase in compliance with guidelines
Teich et al. (2001) [43]In-house developedCPOENot documented4-week study periods before and after interventionsHospital medical practicePrescribing practices; compliance with formulary and prophylactic heparin use guidelinesIncreased frequency of use of H2 blocker; increased rate of ordering prophylactic heparin; decreased rates of excessive dosing; increased appropriateness for frequency of ondansetron use
  • EMR, electronic medical record; CPOE, computer patient order entry; CDSS, computer decision support system.

View this table:
Table 3

Cross-sectional studies

AuthorsNature of the tested systemName of systemParticipants (n)DomainJudgment criteriaResults
Laerum et al. (2001) [44]CommercialEMR227 physiciansGeneral clinical tasks in Norwegian hospitalsProportion of respondents using the electronic system; physician satisfactionModerate utilization; significant increase in satisfaction score
Makoul et al. (2001) [45]CommercialEMR204 patients, 6 physiciansInternal medicine faculty practicePhysician–patient communicationMore active role in clarifying information; difficulty focusing attention on other aspects of patient communication
Rodriguez et al. (2002) [41]UnknownGraphical-based electronic patient record36 physiciansInternal medicineUsability; learnability; efficiency; physician satisfactionSignificant increase in physician satisfaction
  • EMR, electronic medical record.

View this table:
Table 4

Qualitative studies

AuthorsNature of the tested systemName of systemParticipants (n)DomainJudgment criteriaResults
Lee et al. (2002) [47]UnknownComputerized nursing care plan system12 nursesRespiratory intensive care unitsNursing satisfactionAmbivalent feelings
Matsumura et al. (2001) [10]In-house developedEMR20 in-patientsHospital medical practicePhysician satisfactionUsefulness for patient history
  • EMR, electronic medical record.

Study designs

Several study designs were used: 10 were RCTs [2432], 11 were before–after studies [3343], three were cross-sectional studies [4446], and two were based on qualitative interviews [10,47].

Criteria of judgment

Sixteen studies attempted to assess the process of care. Five studies used patient outcomes as a criterion of judgment. Fifteen studies used satisfaction among users and/or patients as the endpoint.

Process of care

Length of the consultation. Six studies analysed the impact of CBPRS on the length of the consultation [31,33,37,39,41,45]. An increase in this length was observed in three studies [31,33,45]. This increase was found to range between 2.2 and 9.3 minutes per patient [31,33]. Makoul et al. [45] found discordant results according to the variety of consultation. Initial visits had a mean length of 35 minutes when using CBPRS compared with 25 minutes without (P < 0.05). No difference was observed during subsequent visits. The length of the consultation was found to be reduced in one study [41] and unchanged in another study [37].

Content of the consultation. Three studies analysed the impact of CBPRS on the content of the consultation [31,33,45]. Some variables, such as questions regarding mental health or behaviour such as tobacco consumption were more frequently recorded as items of the CBPRS than when these tools were not used [33,45]. Clinicians were significantly more likely to address routine health care maintenance topics, including psychosocial issues [relative risk (RR) = 1.4] and smoking at home (RR = 15.6) in a paediatric primary care practice using CBPRS than in one using paper records [33].

Clinical performance

Twelve studies analysed modification of medical practice [2530,3336,40,48]. Three studies analysed the impact of CBPRS on preventive care [26,28,33], and an improvement was observed in all of them. Dexter et al. [26] analysed pneumococcal and influenza vaccination, and prophylactic heparin and aspirin use. An increase was observed in vaccination rates from 0.8% to 35.8% for pneumococcal vaccination, and from 1% to 51.4% for influenza vaccination. Likewise, prophylactic heparin and aspirin prescription at discharge increased from 18.9% to 32.2%, and from 27.6% to 36.4%, respectively.

Eight studies analysed compliance with guidelines [25, 2729,35,36,48]. An increase in the application of guidelines was observed in three studies of the management of breast cancer, venous thromboembolism, and blood test ordering [35,36, 48]. This increase was 14% (85% versus 62%, respectively; P < 10−4) with respect to breast cancer guidelines and 12% (95% versus 83%, respectively; P < 0.001) with respect to venous thromboembolism guidelines [35,36]. In a study of blood test ordering, general practitioners who used CBPRS requested 20% fewer blood tests [48]. Results regarding the management of acute respiratory distress syndrome showed that 95% of computer-generated instructions were followed [29]. However, compliance with guidelines on the management of asthma, angina, diabetes mellitus, and major depression was not found to increase [25,27,28].

Nine studies analysed the impact of CBPRS on treatment prescriptions [25,27,30,3436,40,43,48]. Prescription rates concordant with recommendations increased by 31% for chemotherapy prescriptions and by 61% for prescriptions for venous thromboembolism. Accordingly, the error rate decreased [35,36]. Improvements in appropriateness for prescription of drugs with CBPRS was reported in one study [43]. No significant evolution could be observed in the remaining five studies [25,27,30,34,40].

Patient outcomes

Five studies used patient outcomes as a criterion of judgment [25,2730]. The results were not improved for cardiovascular risk [28,30], survival, length of hospitalization, or acute respiratory distress syndrome [29]. However a significant decrease of 2.3 mmHg diastolic pressure was observed in one study [28]. In two studies, outcomes were assessed by the patients. No benefit with respect to the management of asthma and angina, or major depression was observed [25,27].

User and patient satisfaction

Physician satisfaction. Thirteen studies analysed physician perception for CBPRS use [10,28,3133,3741,4446]. Physician satisfaction with the system was reported in nine studies [10,31,33,3941,4446]. The positive points reported were improved knowledge of patients’ medical history, better medical examination, and improvement in quality of care. In four studies, physicians were asked if they wanted to continue working with CBPRS: all studies reported positive responses [31,39,40,46]. Nevertheless, four studies raised some points of dissatisfaction [28,32,37,38]. CBPRS was perceived as a physical barrier [33,37,38,45] that could have a negative impact on the patient–physician relationship [37,38,46], particularly by reducing eye-to-eye contact [33]. Concerns were expressed about data confidentiality [3739], personal and professional privacy, bug management [32,38,44], and the additional work for physicians [28,31,32]. Factors influencing perception of the system were its characteristics, such as its interface and ease of use, as well as users’ characteristics. Perception of CBPRS was better for community-based physicians than for academic-based physicians. Physicians accustomed to working with a computer had a better perception of CBPRS [46].

Nurse satisfaction. Three studies analysed nurse satisfaction with the introduction of CBPRS in their practice [32,40,47]. A global increase in satisfaction was observed in two studies [32,40]. In another study, reduced administrative work and increased accessibility to care protocols, especially for young nurses, were reported. Negative points were the lack of flexibility of these tools, the loss of nurses’ judgment, and the additional workload [47]. Nevertheless, nurses stated their desire to continue working with an informatic system in all three studies.

Patient satisfaction. Two studies analysed patient satisfaction [37,49], and patients were found to have a positive opinion of CBPRS in both. A mean score of 4.6 on a general satisfaction scale of 0–5 was reported [37]. Patients did not report a reduction in eye-to-eye contact with the use of CBPRS, and found medical visits more effective. However, fear concerning data confidentiality was observed in both studies.

Discussion

Use of a CBPRS seems to be perceived favorably by physicians, with user satisfaction being mainly positive despite some limits and concerns regarding its use [31,39,40,46]. The main constraints reported were the physical barrier and the impact on the patient–physician relationship that could result from the use of CBPRS during the consultation. Technical characteristics of the system and knowledge of data processing by physicians were criteria related to satisfaction with, and thus implementation of, CBPRS [50]. Future research is needed considering qualitative factors that could influence the use of CBPRS [7,51].

Physicians wanted to continue working with CBPRS once such tools were in place in their wards [31,39,40,46], as once exposed, physicians felt a need for an information system. To improve the quality of CBPRS it is essential that the quality of data collected is certified and controlled [52]. Furthermore, to prepare for technical problems with the computer it is advisable always to keep a paper print-out of patient records [53].

Only two studies analysed patient satisfaction. Generally, patients were satisfied and did not complain about changes to their relationship with their physicians [37]. Their greatest fear concerned data confidentiality. It is therefore essential to reassure patients about the confidentiality of their data. Physicians must be able to explain how data confidentiality can be maintained with the system they use. Additional patient-focused studies are thus necessary to explore their feelings about the use of CBPRS in consultation rooms and its consequences with respect to the patient–physician relationship.

The impact of CBPRS on medical practice was more balanced. A clear positive impact of CBPRS on preventive care was noted. This finding is consistent with other systematic reviews [13,20,54]. Improvements in medical practice and the adoption of guidelines was less certain. Positive experiences were as frequent as experiences showing no benefit. In studies of arterial hypertension and major depression [25,30,5558], there was no improvement in medical practice and compliance with guidelines. The recent systematic review of Kaushal et al. showed that CBPRS could decrease prescription error, although most studies were inconclusive [59]. Possible reasons for these inconclusive findings were underuse of the tool, inadequate study design, and lack of study power [28].

Only six studies analysed the impact of the use of CBPRS on patient outcomes and did not show any benefit of CBPRS. In a systematic review [13], 25 RCTs evaluating the contribution of CBPRS to medical practice were analysed. Only eight of these studies used patient outcomes as a criterion of judgment. The relationship between the process of care and health outcomes might explain why improved outcomes are difficult to relate to the implementation of CBPRS [60].

Users of CBPRS acknowledge the usefulness of such tools, whereas the results of clinical performance or patient outcomes are not always conclusive. This paradox draws the attention to the criteria and methodologies used in various studies. RCTs are usually considered the methodological reference [61]. However, this methodological approach has several limits in evaluating the impact of CBPRS on medical practice. In order to organize RCTs and select a criterion of judgment, tracers have to be chosen within complex and multifactorial diseases (e.g. hypertension, depression). Moreover, even if positive results are found in some narrow clinical fields, generalization to the more complex clinical situations is generally difficult [62]. Controlled conditions required by RCTs do not reflect the diversity of medical practices and working conditions. Due to these limitations, RCTs do not offer a sound enough methodology to evaluate the contribution of CBPRS in medical practice, and its impact on clinical performance and patient outcomes [61,63,64].

Two systematic reviews [23,65] describing methodological approaches to the evaluation of systems designed to improve medical decision making emphasized the lack of diversity in the design of studies that evaluate these tools. This phenomenon could lead to lack of conclusive results. Alternatives approaches centered on social, cultural, and organizational factors are thus advocated. Structure of care (general practice, hospital), organization of the ward, and the relationships between various health professionals seem to be key factors to consider when evaluating the impact of CBPRS on medical practice [65].

It is recognized that systematic review may be subject to a possible selection bias. Studies presenting negative findings are indeed published less often [13]. Nevertheless, a high proportion of articles showing no effect of CBPRS on quality of care were found [25,27,28]. A wide range of keywords to find systems with the same characteristics had to be used. A well defined operating definition of CBPRS, sharing the same characteristics, could facilitate future research and produce more exhaustivity [66].

Conclusion

This systematic review, based on 25 papers published between 2000 and 2003, has contributed to a better understanding of the relationship between the use of CBPRS and medical practice. Increased satisfaction of users and patients was noted, which could lead to significant changes in medical practice. However, as in previous reviews, the impact of CBPRS on medical practice and quality of care was not well demonstrated. It is noteworthy that most of the studies did not include qualitative factors such as characteristics of the disease and the tool, the ward in which it is developed, and the relationship between various health care professionals, which can impact upon the use of CBPRS.

A broad review including all the factors that could influence the success or failure of the use of CBPRS in medical practice is indicated in the future.

References

View Abstract