A community survey of medical errors in New York*
1 New York Academy of Medicine, Division of Health and Science Policy, New York, NY, 2 Mount Sinai School of Medicine, Department of Pediatrics, New York, NY, USA
Objectives. To assess factors related to experiences with medical errors by health care consumers in the community.
Design. Using a random telephone survey of New York State residents screened for knowledge of health care utilization, we gathered information about demographic factors, health care attitudes, experiences with the health care system, and use of information to make health care decisions.
Setting. The State of New York, USA.
Participants. Adults living in the State of New York who possessed a telephone.
Interventions. None.
Results. Approximately one-fifth (21.1%) of New Yorkers reported that either they or someone in their household had experienced a medical error, with logistic regression models for ever experiencing a household medical error revealed that respondents who were divorced/separated/widowed, African American, and those from higher income households were less likely to report medical errors. Conversely, those between the ages of 30 and 65 years, those who had frequent doctor visits, and those who were better informed about health care were more likely to report them. The results were similar for household medical errors in the past 5 years. In all multivariate models, greater use of medical information was consistently related to experiencing household medical errors. Having a regular doctor, having health insurance, and concern about health care delivery were not related to either of these outcomes.
Conclusions. Our study indicated that one-fifth of New York State households had experienced a medical error, with one in 10 reporting experiencing a household medical error within the past 5 years. Greater knowledge about health care increased the likelihood of reporting a household medical error. Thus, a greater consumer orientation in health care and provision of more medical information may increase rather than reduce the reporting of medical errors by the public.
Keywords: adverse medical events, health care consumers, medical errors, patient perceptions, quality improvement, quality of care
Address reprint requests to J. A. Boscarino, New York Academy of Medicine, Division of Health and Science Policy, New York, NY, USA. E-mail: jboscarino{at}nyam.org or ja_boscarino{at}hotmail.com
Accepted for publication May 17, 2004.
With its recent report on medical errors, the Institute of Medicine (IOM) in the United States generated renewed interest in reducing adverse events in medical care settings [1]. Although the IOM study has had both its critics and defenders [2,3], research in the US and other countries has generally supported the IOM’s conclusions [4–9]. The Harvard Medical Practice Study reported that adverse events occurred in 3.7% of hospitalizations in New York in 1984, and that the rate of adverse events due to negligence was 1.0% [6,7]. In Australia, assessments of medical records in 28 hospitals in New South Wales and South Australia revealed that 17% of the admissions were associated with an adverse event, over half of which were considered preventable [8]. Similarly, a study in the United Kingdom indicated that 
10% of in-patient admissions resulted in harmful adverse events, with about half being preventable [9].
Given this interest, it is surprising that almost no studies have focused on the public’s perceptions of medical errors. Previous work has focused on measurement issues in defining and collecting data on medical errors [2,3,10,11] and the policy, legal, and structural changes in health care delivery required to reduce such events [4,6,10]. This emphasis on the health care system, however, neglects input from individual patients. A national study in the US, conducted in 2000 by the Kaiser Family Foundation and the Agency for Healthcare Research and Quality (AHRQ), was one of the few to survey health care consumers [12]. This survey found that 6% of the public reported suffering ‘personal injury or harm’ that resulted from a medical error in the 12 months before the survey [12]. Thus, studies suggest that medical errors have had an adverse impact on patient health, are common, and are of concern to health care consumers [13,14].
In the present study, two questions guided our analyses: (i) how many persons reported that someone in their household experienced a medical error?; and (ii) what were the key correlates of these errors? We were particularly interested in the use of medical information about quality and whether being more informed about health care was related to a lower risk of medical errors in the household. The model underlying our analyses posits the patient as a ‘consumer’ of health care who engages in doctor and hospital ‘shopping’ [15], and who hence can be a potential change agent in the health care system.
 |
Methods |
|---|
 Top Methods ResultsDiscussionReferences |
|---|
Using random-digit dialing of telephone numbers, we interviewed a sample of English-/Spanish-speaking adults (
18 years of age) living in New York State over the telephone. The study population was stratified by the five regions of New York State and was sampled proportionately. The interview was translated by bilingual Spanish/English speakers and back-translated to insure linguistic and cultural appropriateness. Once an eligible household was contacted, one adult was randomly selected for an interview. At the beginning of the survey, potential respondents also were screened for familiarity with household health care utilization and only these persons were interviewed. We followed the same procedures used by the Kaiser Family Foundation/AHRQ in their study of health care consumers in the US [12], and treated the respondent as representative of the entire household. The Institutional Review Board of the New York Academy of Medicine approved our study’s protocols.
Altogether, 1001 interviews were completed in September 2002, with each interview lasting 20 min. Of the 13 545 telephone numbers we attempted to contact, 6902 (51%) were either invalid numbers (e.g. non-resident number) or non-contacts (i.e. no answer after 10 calls). The remaining 6643 were classified as ‘not meeting language, age, or health care screening criteria’ (1006), ‘answering machine/voice mail only with no contact’ (1046), callbacks not reached by end of the study (2460), screened-out because the quota for that region was met (138), refusals (992), and completed interviews (1001). Thus, the survey cooperation rate, using industry standards, was 53% (completed + screenouts/completed + screenouts + refusals) [16]. Sampling weights were developed to adjust the data for potential bias due to the number of household telephone numbers and persons in the household. An age weight was also developed and used to adjust the sample for slightly over-sampling older adults, resulting from screening respondents for knowledge of health care utilization. The result of this weighting did not significantly change the distributions of other key variables and all results are based on these weighted data.
Dependent variables
The main variable of interest was medical errors experienced by the respondent or someone in their household. The survey defined ‘medical error’ and gave several examples so that the respondent would know what this term meant. We then asked respondents ‘Have you or anyone in your household ever suffered injury or harm that resulted from a medical error?’. Respondents were also asked if the household medical error had occurred <5 years ago, and about the type of error (medication, surgical, or diagnostic). Our medical error questions were used in a previous national survey in the US and adopted for use in our study [17].
Independent variables
Our analyses included 13 independent variables. The first set of variables comprised demographic characteristics: gender, marital status, education, age, race/ethnicity, and 2001 household income. Based on preliminary analyses, marital status and education were dummy coded into divorced/separated/widowed versus married/living together/never married, and high school or less versus some college or more, respectively. Age was divided into three categories: 18–29, 30–64, and 65 years. Race/ethnicity was coded into four categories: white, African American, Hispanic, and ‘other’. Income groups were <$30 000, $30 000–$100 000, $100 000+, and ‘not reported’. Given that >20% of the respondents did not report household income, we included them in a separate category to avoid eliminating them from our study. Finally, region of the state where the respondent lived was coded into two categories: Upstate New York (Hudson Valley and areas of the state north and west of New York City) and Downstate New York (New York City/Long Island).
The second set of independent variables concentrated on health care factors. We dummy coded whether or not respondents had a regular doctor, if they had health insurance coverage, if there was a person with a chronic illness living in the home, and the number of visits made to doctors by household members in the past 12 months (coded as 0–10 or 11). Respondents’ attitudes about the health care system were based on the sum of a three-item Likert scale (concern about health care costs, doctor services, and hospital services), rated from ‘very concerned’ to ‘not at all concerned’ on a 5-point scale (Cronbach’s alpha = 0.70). We dichotomized these results into high concern versus low concern. Finally, the survey inquired whether or not respondents recently used health care quality information to make a decision about the use of doctors and hospitals. Specifically, the survey asked ‘Did you personally use information you saw or heard about quality among doctors in making any decisions about doctors in the past 12 months?’. There was a similarly worded question about hospitals. Based on these quality recall questions, we developed a dichotomous variable indicating that the respondent recently used, or did not use, information about doctors or hospitals to make a medical decision. Again, the concern about health care and information use questions were adopted from a previous national survey in the US and pre-tested before use in our study [12].
Analytic strategy
Firstly, frequency distributions for selected demographic variables (region, gender, age, and race/ethnicity) were compared with the 2000 New York Census to ensure that the sample reflected the state’s population. Secondly, we examined bivariate associations between household medical errors and the 13 independent variables discussed. Finally, multivariate logistic regressions were used to estimate the associations between the predictor variables and any household medical errors experienced, as well as household medical errors in the past 5 years. For our statistical analyses we used the survey estimator commands in Stata, version 7 [18], which adjusted our data for survey stratification based on 5 New York State regions and the case weights discussed for telephone lines, adults in the household, and age.
|
Results |
|---|
|
TopMethodsResultsDiscussionReferences |
|---|
The basic characteristics of our final weighted sample in terms of region, gender, age, and race/ethnicity did not significantly diverge from the 2000 census data for New York State (Table 1). Overall, 21.1% [95% confidence interval (CI) 18.3–24.0] of New York adults indicated that either they or someone in their household had experienced a medical error, with 11.4% (95% CI 9.2–13.6) experiencing an error in the past 5 years (Table 2). In terms of specific medical errors, 5.9% (95% CI 4.4–7.5) stated that they experienced a household medication error, 7.2% (95% CI 5.4–9.0) a household surgical error, and 6.3% (95% CI 4.7–7.7) a household diagnostic error. An examination of the 13 predictor variables indicated that those aged between 30 and 64 years, whites, households that had frequent doctor visits, and those who used information to make decisions about doctors and hospitals were more likely to report that someone in their household had experienced a medical error.
|
|
Results for medical errors in the past 5 years (Table 2, column 3) revealed that married and never married respondents were more likely to report household errors than their divorced, separated, or widowed counterparts. Having a high number of household doctor visits was also related to recent household medical errors. Finally, respondents who used quality information about doctors and hospitals to make medical decisions were more likely to report experiencing a household medical error in the past 5 years.
As Table 2 shows, few predictor variables were associated with a specific type of household medical error. More specifically, having a person with a chronic illness in the home was related to reporting household medication and surgical errors. Respondents with a regular doctor experienced more surgical errors in the household, as did those from households that visited the doctor often over the previous year. Lastly, using information to make medical decisions about doctors and/or hospitals was associated with a greater likelihood of reported household medication and diagnostic errors. None of the other predictor variables were statistically associated with any of the three types of household medical errors examined.
For ever having experienced a household medical error (Table 3, column 1), the multivariate logistic analyses suggested that controlling for all other variables in the model, marital status, age, race/ethnicity, income, and using quality information to make medical decisions were significantly associated with reporting household medical errors. In particular, being an African American or residing in middle- to high-income households lowered the likelihood of reporting medical errors in the household, while being married/living together/never married increased the likelihood. Thirdly, and most importantly, respondents who used quality information to make medical decisions were those most likely to report household medical errors.
|
Few predictor variables were statistically associated with experiencing household medical errors in the past 5 years (Table 3, column 2). The significant relationships found were for marital status, income, and use of quality information to make health care decisions. Respondents who were married/living together/never married were more likely to report experiencing a household medical error during the past 5 years compared with those who were divorced/separated/widowed. Being from wealthier households (income $100 000+) was associated with a lower likelihood of household medical errors over the past 5 years, compared with respondents from poorer households. Finally, use of quality medical information was associated with reporting a household medical error and, in fact, almost tripled [odds-ratio (OR) 2.8] the likelihood of reporting such errors in the past 5 years.
To assess the robustness of our findings at the household level, we re-analyzed the data, focusing only on medical errors experienced by the respondent (results not shown but available upon request from the corresponding author). We were limited in these analyses, since only 103 respondents indicated that they had personally experienced a medical error. Despite reduced power, these analyses still indicated that use of medical information was related to the respondent ever experiencing a medical error (P = 0.051) and related to experiencing a medical error in the past 5 years (P = 0.011). College-educated respondents and those who had a person with a chronic illness in the household were also more likely to report ever experiencing a medical error, while respondents who had a person with a chronic illness in the household were more likely to report personally experiencing a medical error in the past 5 years.
|
Discussion |
|---|
|
TopMethodsResultsDiscussionReferences |
|---|
Using data from a representative cross-sectional survey of New York State residents, we found that household medical errors were common. Overall, one in five respondents reported ever experiencing a household medical error, while one in 10 indicated that the household medical error happened within the past 5 years. Unexpectedly, the variable most consistently associated with reporting household medical errors was recently using quality information to make health care decisions. It is interesting to note that having a regular doctor, having insurance, having a person with a chronic illness in the home, and having higher concerns about health care quality were not related to reporting medical errors.
The association between use of medical information and the increased likelihood of reporting medical errors did not appear to be due to greater overall reporting of medical errors in our survey. In the national study conducted by the Kaiser Family Foundation/AHRQ [12], 6% of the respondents reported experiences of medical errors in the past year, while this figure was 4% in our New York survey. The use of health care quality information was slightly higher in our New York study than in the national study [12] (19% compared with 15%, respectively), but only marginally so. Lastly, the association between recent use of quality information and reported medical errors held in our study, even when key demographic, regional, and health care variables were controlled.
There are several explanations for the relationship between reporting household medical errors and use of medical information. Firstly, knowledge about the health care system may have made individuals more aware of health care professionals’ behavior and more likely to connect injury to mistakes by these professionals. Secondly, patients who use information may also be more critical of physicians and more likely to link physical problems to poor or inappropriate medical care [17,19].
Another possibility is that past experience with medical errors results in health care consumers seeking greater information and using that information to make decisions about health care. The use of health care information might be a defensive strategy employed by health care consumers related to perceived problems with the US health care system. In our study, respondents who reported using medical information were also more likely to score high on concerns about health care quality (
2 = 23.99, P < 0.001). Nevertheless, given that information use was still significant in the multivariate model, its association with household medical errors was independent of concerns about health care delivery. At this point, due to the limitations of our data, we cannot fully assess whether use of health care information is a cause or an effect in this situation.
Our survey has several limitations. Firstly, we do not have independent verification that someone in the household actually suffered a medical error. On the other hand, observational studies using self-reported measures are common. Criminologists, for example, use self-report victimization surveys and attach a high level of validity to these data [20,21]. They stress that victimization studies are a necessary alternative to official crime statistics. Given the greater consumer orientation of modern health care delivery, client surveys offer a different and equally informative perspective on quality of care when compared with other sources of information. In addition, our medical errors survey questions come from a national survey and give explicit examples of medical errors before asking the respondent if any household members suffered injury or harm due to these kinds of events.
Secondly, telephone surveys miss households without working telephones, which tend to be poorer. We also excluded individuals who spoke a language other than English or Spanish. Thus, the sample may under-represent low-income households and those from recent immigrant groups. The sample, however, reflected the state’s distributions for region, gender, age, and race/ethnicity and, therefore, does not appear to suffer from any overall demographic biases. These findings, however, should not be considered representative of the United States as a whole. Each state has a unique set of health care policies and programs, which suggests that future studies need to focus their data collection efforts at this level.
Thirdly, the data used in our study were cross-sectional and not longitudinal, therefore cannot be used to assess causal direction. The association between use of health care information and reporting household medical errors may be due to a knowledge-to-error link or vice versa, an error-to-knowledge link. In addition, although longitudinal data can shed light on some causal connections, a greater use of social psychological theory and methods will also be required.
Fourthly, the survey’s cooperation rate was lower than desirable and may contain some unknown biases. As noted earlier, the sampling method somewhat under-represented younger adults. Although we weighted the data to correct for this, it may not have prevented survey bias. Thus, these results should be accepted with some caution until other studies replicate them. Nevertheless, the study’s findings can be generalized to households in New York State and may be most representative of those knowledgeable about health care utilization in the household. There is no reason, however, to suspect that the study’s data collection methods and response rate introduced a systematic bias into the data.
Finally, how individuals in the household actually responded to the medical error is unknown. We do not have information about whether or not they reported the medical error to some oversight organization, or the long-term consequences of the medical error. Future surveys need to delve into the specifics of the medical errors experienced by respondents and members of their households.
The IOM’s report has spurred renewed attention to adverse medical events in the US. Errors such as prescribing wrong medications or making diagnostic errors have not been unique to the US, but have been found in other industrialized countries [8,9]. To reduce such adverse events, a number of investigators have urged evidence-based medicine to focus on safety practices and a shift from risk management to prevention [22–24]. Patient advocates have promoted health care report cards as a means to equip consumers with information about service quality that would enable them to make informed decisions about health care [25,26]. In addition to not being supported by the present study, many researchers have criticized health care report cards because the information does not discriminate well between doctors or hospitals, and the data necessary for such performance evaluations are poor [27–29]. Furthermore, actual use of this information by consumers has appeared to be questionable [29].
One of the most pressing areas for future research is to assess predictors of adverse events found in patient records and those reported on patient surveys. In the Harvard medical study, 3.7% of the records from New York hospitals were classified as having an adverse event in 1 year (1984) [6,7]. This percentage is very close to the 4% of respondents in our survey who reported that they or someone in their household had experienced a medical error. A critical question is whether or not patients reporting adverse events are the same ones judged by chart review to have suffered from a medical error. Studies that start with a sample of patients’ charts with and without an adverse event and then survey those patients may give insight into differences in ‘official’ versus public perceptions of medical errors.
Overall, our survey of residents living in New York State indicated that medical errors resulting in harm to patients were common and that use of health care information by New Yorkers was related to reporting such errors. Advocates of disseminating health care information contend that combining such information with doctor shopping should improve the overall quality of health care in a particular area [25,26]. Results from our study, however, suggest that information dissemination, in and by itself, may not lower the risk of medical errors or lower individuals’ concerns about the quality of health care in the US. In fact, the opposite may be true. Additional research is required to determine more fully if this is actually the case in the US and internationally.
|
Footnotes |
|---|
* Early versions of this paper were presented at the Annual Meeting of Academy Health, Nashville, TN, 27–29 June 2003, and the Annual Symposium on Health Services Research in New York, 5 November 2003.
|
References |
|---|
|
TopMethodsResultsDiscussionReferences |
|---|
- Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safe Health System. Washington DC: Committee on Quality of Health Care in America, Institute of Medicine, National Academy Press, 1999.
- McDonald CJ, Weiner M, Hui SL. Deaths due to medical errors are exaggerated in Institute of Medicine report. JAMA 2000; 284: 93–95.
[Free Full Text] - Leape LL. Institute of Medicine medical error figures are not exaggerated. JAMA 2000; 284: 95–97.
[Free Full Text] - Phillips DP, Bredder CC. Morbidity and mortality from medical errors: an increasingly serious public health problem. Ann Rev Publ Health 2002; 23: 135–150.[CrossRef][Medline]
- Barker KN, Flynn EA, Pepper GA, Bates DW, Mikeal RL. Medication errors observed in 36 health care facilities. Arch Intern Med 2002; 162: 1897–1903.
- Brennan TA, Leape LL, Laird NM et al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard medical practice study I. New Engl J Med 1991; 324: 370–376.[Abstract]
- Leape LL, Brennan TA, Laird N et al. The nature of adverse events in hospitalized patients: results of the Harvard medical practice study II. New Engl J Med 1991; 324: 377–384.[Abstract]
- Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian health care study. Med J Aus 1995; 163: 458–471.
- Department of Health. An Organisation with a Memory. London: The Stationery Office, 2000.
- McNutt RA, Abrams R, Aron DC. Patient safety efforts should focus on medical errors. JAMA 2002; 287: 1997–2001.
[Free Full Text] - Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. JAMA 1997; 277: 312–317.[Abstract]
- Kaiser Family Foundation. National Survey on Americans as Health Care Consumers: An Update on the Role of Quality Information. Menlo Park, CA: Kaiser Family Foundation/Agency for Healthcare Research and Quality, 2000.
- Blendon RJ, DesRoches CM, Brodie M et al. Views of practicing physicians and the public on medical errors. New Engl Med 2002; 347: 1933–1940.
- Robinson AR, Hohmann KB, Rifkin JI et al. Physician and public opinions on quality of health care and the problem of medical errors. Arch Intern Med 2002; 162: 2186–2190.
[Abstract/Free Full Text] - Boscarino JA, Steiber SR. Hospital shopping and consumer choice. J Health Care Market 1982; 2: 12–23.
- American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcomes Rates for Surveys. Ann Arbor, MI: American Association for Public Opinion Research, 2000.
- Bagozzi RP, Guerhan-Canli Z, Priester JR. The Social Psychology of Consumer Behaviour. Buckingham, UK: Open University Press, 2002.
- StataCorp. Stata Statistical Software: Release Version 7. College Station, TX: Stata Corporation, 2001.
- Bodenhausen GV, Macrae CN, Hugenberg K. Social Cognition. In Millon T, Lerner MJ, eds. Handbook of Psychology: Personality and Social Psychology, Vol. 5. New York, NY: John Wiley and Sons, Inc., 2003, pp. 257–282.
- United States Department of Justice. The Nation’s Two Crime Measures. Fact Sheet NCJ-122795. Washington, DC: Bureau of Justice Statistics, 2000.
- Rand MR, Rennison CM. True crime stories? Accounting for differences in our national crime indicators. Chance 2002; 15: 47–51.
- Leape LL, Berwick DM, Bates DW. What practices will most improve safety?: evidence-based medicine meets patient safety. JAMA 2002; 288: 501–507.
[Free Full Text] - Shojania KG, Duncan BW, McDonald KM, Wachter RM. Safe but sound: patient safety meets evidence-based medicine. JAMA 2002; 288: 508–513.
[Free Full Text] - Chassin MR. Achieving and sustaining improved quality: lessons from New York State and cardiac surgery. Health Aff 2002; 21: 40–51.
[Abstract/Free Full Text] - Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Publ Health 2003; 24: 413–433.[CrossRef][ISI][Medline]
- Lansky D. Improving quality through public disclosure of performance information. Health Aff 2002; 21: 52–62.
[Abstract/Free Full Text] - Boscarino JA, Chang J. Commentary: inaccurate data on the quality of care may do more harm than good—an alternative approach is required. Am J Med Qual 1997; 12: 196–200.
- Boscarino JA. The public’s perception of quality hospitals II. Implications for patient surveys. Hosp Health Serv Admin 1992; 37: 13–35.[Medline]
- Boscarino JA, Adams RE. Perceptions of quality care and provider profiling in New York: implications for improving quality care and public health. J Publ Health Man Pract 2004; 10: 241–250.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. H Wasson, T. A MacKenzie, and M. Hall Patients use an internet technology to report when things go wrong Qual. Saf. Health Care, June 1, 2007; 16(3): 213 - 215. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||