International Journal for Quality in Health Care Advance Access originally published online on February 20, 2007
International Journal for Quality in Health Care 2007 19(2):120-124; doi:10.1093/intqhc/mzm001
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Electronic questionnaires for measuring parent satisfaction and as a basis for quality improvement
1 Department of Paediatrics, Kolding Hospital, Denmark
2 Research Initiative of Health Services, Kolding Hospital, Denmark
Address reprint requests to: Jette Ammentorp. E-mail: ammentorp{at}tdcadsl.dk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
Background. Using paper questionnaires to measure quality of care from the perspective of the patient is a time consuming procedure resulting in very slow feedback. Response rates are low and patients who cannot read the local language are usually excluded.
Objective. To investigate the applicability of an electronic questionnaire by evaluating the response rate. To study whether computer-based continuous monitoring could elucidate reasons for parents being less satisfied with care and treatment and to compare parent satisfaction with the results of a study performed in 2003.
Methods. Parents were asked to assess the quality of care and treatment by answering questions on a touch-screen computer. The questions, which were translated into seven languages, corresponded to the indicators selected by the department for monitoring parents' satisfaction. The system was developed in cooperation with a software company.
Results. A total of 780 parents answered (69%). Of these, 2% parents answered in a foreign language. The main reasons for being less satisfied were perceived difficulties in getting in contact with the staff, having experienced unnecessary long waiting time and having the impression that the nurses did not have enough time. Significant improvements in satisfaction had occurred as compared to a study from 2003.
Conclusion. By using electronic questionnaires, it was possible to focus on the small percentage of parents not satisfied, to identify reasons for being less satisfied and to respond immediately to the feedback from the parents. Electronic surveys produce a satisfactory response rates.
Keywords: electronic assessment, electronic data capture, patient satisfaction
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
The patient perspective constitutes an important element of the overall quality of care. Patients can make valuable contributions when evaluating interpersonal aspects of care, communication, information and organisation of care [1].
The quality of care as experienced by the patients is traditionally studied using paper questionnaires. However, the increasing focus on quality assessment and the demands for monitoring standards of quality of care strongly emphasize the need for taking into account the disadvantages of this method.
Handling of paper questionnaires is time and resource consuming, especially as questionnaires should be coded and reminders send to non-responders to assure a reasonable response rate [2]. Data can be entered manually or by the use of optical scanning technology, both requiring man-hours. Furthermore, these methods are associated with data entry errors. The error rates using optical scanning methods are slightly lower than for single data entry but higher than for double data entry compared to manual entry method [3].
The time consuming procedures, handling paper questionnaires and entering data, result in very slow feedback, making it difficult to take immediate action on problems identified. The possibilities of elucidating reasons for a negative experience for patients might be reduced due to this late feedback.
Response rates below 50% are often accepted and response rates of 50–60% even considered as ‘quite high’ [2, 4]. Some evidence suggest that non-responders seem to be less satisfied than responders [2]. If this is the case, a low response rate will affect the ratings of care favourably. Selection bias may also occur, because questionnaires are usually written in the local language excluding patients who cannot read this language.
This emphasizes the need for investigating alternative methods that can eliminate or reduce the disadvantages described. Using electronic questionnaires could be such an alternative. Reviewing the literature in Pub Med using the keywords patient satisfaction, electronic assessment, health care surveys, electronic data capture, automatic data processing, computer-assisted methods and electronics in different combinations resulted in 10 studies investigating this relatively new methodology. Different types of questionnaires were investigated by asking patients to fill in the questionnaire both on paper and electronically; the majority of these questionnaires were used for the assessment of quality of life and symptoms/behaviour and a few for the assessment of patient satisfaction. The scores obtained by the computer-assisted method were equivalent to that obtained by the traditional method [5–10]. Furthermore, electronic questionnaires decreased the number of missing data [5, 11, 12] and were preferred by the majority of the patients, as they found them easier to fill in [5–7, 9, 11–14], regardless of age and previous experience of working with computers [8, 9, 12, 13]. However, neither of these studies have described the validity of the electronic survey with respect to the response rate nor have described how to use the electronic questionnaires as an interactive quality assessment tool.
To investigate the applicability of an electronic questionnaire, we wanted to (i) evaluate the response rate for both Danish and non-Danish speaking parents, (ii) study if computer-based continuous monitoring could elucidate reasons for parents being less satisfied with care and treatment and (iii) evaluate the satisfaction as compared to the study performed in 2003, when indicators of quality of care were for the first time evaluated electronically [15].
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
The present study was carried out in the receiving ward for all non-neonatal children referred to the Department of Paediatrics at Kolding Hospital in Denmark. In the receiving ward, the children are examined and treatment started. Following a period of observation, it is decided whether the child should be transferred to the stationary paediatric ward or can be discharged. At least one of the parents always stays with the child. As most patients are young children, in this study, we chose to ask the parents to answer the questions as they were considered to be the ones most capable of evaluating the quality of care.
From 1 January to 30 June 2005, all acutely referred children were enrolled consecutively. On discharge, the parents staying with the child were asked to assess the quality of care by answering questions on a touch-screen computer situated centrally in the ward. They were given a card with a bar code that allowed them to enter their responses on the computer on which the instruction and the questions had been translated into seven languages. The questions were based on 14 quality standards previously developed for emergency admittances to a paediatric department. The standards and three related indicators had been identified in inter-disciplinary audits; based on interviews with parents and by focus group interviews including doctors and nurses from the department [16]. The three questions asked on the computer corresponded to these indicators of quality of care selected by the department for monitoring parents' satisfaction: ‘have you experienced kindness?’, ‘have you experienced that you were taken care of?’ and ‘have you felt that you were well-informed?’. Parents were initially asked to assess the three questions on a five-point Likert scale from ‘to a great extent’ to ‘not at all’. If their responses were in the category ‘to a great extent’, they were not asked any further questions; in all other cases, they were asked to answer up to 16 clarifying questions based on parents' needs as identified previously [16, 17]. Parents answering the clarifying questions were offered to enter their telephone number on the screen and, if accepting, contacted for a follow-up interview within 1 month following discharge. If the parents were not present, the interviewer tried to call for up to five times. The purpose of the telephone interview was, as a part of the continuous monitoring, to detect any reason for being less satisfied not elucidated by the 16 questions already asked, to give the less satisfied parents the possibility of elaborating on their experiences and to supply the quantitative monitoring with qualitative inputs from the parents. The interview was conducted as a semi-structured interview by a clinical development nurse, who by asking open-ended descriptive questions as ‘can you tell me about your stay in the ward?’ gave the parents the opportunity to focus on the areas they found most important. By structuring the interview in accordance with the three main topics of the electronic questionnaire, the parents were given the opportunity to further elaborate their initial answers. A summary of the interview was written as a narrative using the parents own words. Afterwards the data were coded and analysed according to the main problems/issues elucidated.
The system to monitor parents' satisfaction was developed in cooperation with a software company. Scanning the bar code label with a laser scanner to get access to the questionnaire assured that each parent could answer once. A special feature allowed bar codes to change the language of the questionnaire into one of seven possible languages giving non-Danish speaking patients the possibility to answer in their own language. All registration and basic analysis were stored on a central server. For statistical analysis, data were transferred to STATA 9. The response rate was calculated on a monthly base.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
Of the 1128 parents included, 780 answered the electronic questionnaire corresponding to a response rate of 69% (range per month, 60–96%). Of these, 14 (2%) chose a language other than Danish (Somali, Turkish, Tamil, Arabic, Farsi, Bosnian or English).
The results were dichotomized, categorizing ‘to a great extent’ as ‘satisfied to a great extent’ and ‘to some extent’, ‘to a minor extent’ and ‘not at all’ as ‘satisfied to a lesser extent’. The dichotomized results of the three main questions show that out of the 780 parents, 88% (687/780) had experienced kindness, 72% (565/780) felt that they were taken care of and 69% (535/780) felt that they were well-informed ‘to a great extent’.
Of the 215 parents who were ‘satisfied to a lesser extent’ with the main question ‘have you experienced that you were taken care of’, 213 parents answered the nine sub-questions as indicated in Table 1. Of these, 60% were less satisfied with the time the nurses spent on them, 58% with the time they had to wait and 55% with the possibility of getting in touch with the staff. The questions whether the doctors and the nurses had good contact with the child and whether they understood the parents' situation obtained most scores in the category ‘satisfied to a great extent’.
|
Of the 245 parents who were less satisfied with the question ‘have you felt that you were well-informed?’, 242 parents answered the seven sub-questions as indicated in Table 2. Of these, 80% of the parents were less satisfied with the information they had received concerning the time they had to wait (one parent did not answer this question), and 57% with the information about the plan for the admittance. For the other indicators, the proportion of parents being less satisfied ranged from 36% to 48%. Information given in connection with their transfer to the ward and the information given by the doctors obtained most scores in the category ‘satisfied to a great extent’.
|
Parents who were ‘satisfied to a lesser extent’ with the main indicators of care were given the opportunity to enter their telephone number on the screen. Out of 215 parents, 79 entered their telephone number. Of the 43 reached for a telephone-interview all accepted to participate. In the interviews, problems experienced by the parents were further clarified and the department was given advice on how to improve the services provided. The problems elucidated were identical to the themes in the questionnaire, and no new problem areas were identified.
In 2003, a study including a comparable sample of parents assessed two of the main indicators by means of the same electronic questionnaire [15]. Comparing this assessment to the results of the present study from 2005, showed that the proportion of parents satisfied with the information had increased from 60% (415/690) in 2003 to 69% in 2005 (535/780) (P < 0.01), and the proportion of parents satisfied with the care also showed a small but significant improvement from 68% (466/690) in 2003 to 72% (565/780) in 2005 (P = 0.04).
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
Using continuous electronic assessment of the quality of care, a response rate of 69% was obtained over a period of 6 months. The monthly response rate varied from 60% to 96%, stressing the importance of a constant focus on this potential source of error. Using the same methodology, an average response rate of 85% over 12 months was obtained in the same ward in 2003 [15]. Therefore, the present result might underestimate the response rate that can be obtained. Still, compared to paper questionnaires [2, 4] a response rate of 69% must be considered satisfactory. Only 2% answered in a foreign language as compared to 1.5% in the 2003 study, corresponding to a response rate of 21% for non-Danish speaking parents [15]. Although the literature reviewed does not consider the anxiety about using computer and the working experience with computer as a problem in general, it might, however, be a problem for subgroups such as non-Danish speaking parents. The facts that non-Danish speaking parents did not receive verbal instruction in their own language and that they due to language difficulties did not have the same opportunity to ask questions, could also have influenced their possibility and willingness to respond. Finally, the low response rate might also reflect a greater proportion of illiteracy in this subgroup. Previous studies indicate that this sub-group is less satisfied than the responders without language problems [18], and that special efforts are needed to include non-Danish speaking parents into the monitoring of quality of care.
The purpose of the study was to monitor quality standards by means of few indicators. Therefore, only three key questions were used increasing the risk of misinterpretation, response errors, etc. This might have hampered the reliability of the results. Adding two or three questions to each satisfaction domain would have eliminated the problem, but would also have increased the length of the questionnaire and thereby the risk of obtaining a lower response rate.
The questionnaire used in the present study was designed using indicators identified by several other studies on parents' needs, including our own [16, 17]. Furthermore, telephone interviews were performed in order to ensure that the questions covered sufficiently the experiences of the parents. Therefore, in lack of a validated scale from a similar setting, the questionnaire can be considered as a tool with content validity for evaluating the expectations and needs of the parents in a paediatric ward. All data in this study were obtained at the time of discharge and are thus comparable. As opposed to this, most paper questionnaires are often answered varying time intervals following discharge. Considering that the cognitive and emotional states of the parents might influence the responses, and that emotions may change during the period after discharge, a fixed time point for responding might reduce the possibility of information bias. Still whether discharge is the most optimal time for the assessment of parents' satisfaction (having the experiences fresh in the mind) or whether the responses could be biased due to feelings such as relief, fatigue or anxiety is not clear and could be an area for further research.
The main reasons for parents being less satisfied were perceived difficulties in getting in contact with the staff, having experienced unnecessary long waiting time and having the impression that the nurses did not have enough time, all issues related to the parents' experience of the available staff-time. These observations corroborate a previous study showing a strong negative association between high activity at the ward and the overall satisfaction of the parents [15]. Dissatisfaction with longer waiting time and the importance of receiving care and treatment without waiting too long have been documented in other studies evaluating parents' priorities and satisfaction [17, 19].
Nationwide Danish surveys and an international review of patient satisfaction have emphasized that information and communication are areas with great potential for improvement [20, 21]. In the present study, a main complaint was in fact the lack of information about waiting time and about the plan for examination and treatment of the child, indicating that the satisfaction of the parents could be improved by focusing on the communicative skills of the staff.
The results of the present study were compared to those of a study performed in the same ward in 2003 and demonstrated small but significant improvements in the quality of care and in the information given. Even though comparisons over time might be questionable, they might be considered as an indication of changes in parents' satisfaction, especially considering the difficulties in performing benchmarking between different departments.
In the present study, it was thus possible to elucidate some of the main problems related to paediatric care and to the information given. The results obtained during the first 6 months were used to develop and implement an action plan intended to improve the areas having the lowest satisfaction scores. Audits using patients' files were performed to create consensus among staff members and to prepare in detail the action plan. Among other activities, the action plan comprised training in communication skills for all staff members, improving the facilities of waiting rooms and receiving rooms and improving the ease of access to the staff and of the information given to the parents about waiting time.
By using electronic questionnaires, it was possible to focus the quality development on selected important issues; it was possible to create an action plan based on both quantitative and qualitative data and to ensure a continuous follow-up.
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
The use of electronic assessment of quality of care is a relevant method, which can be used not only to monitor parents' satisfaction, but also to focus on the small percentage of parents not satisfied and to identify their reasons for being less satisfied. The electronic questionnaire produced a satisfactory response rate, though with a high variability. Finally, it made it possible to respond immediately to the feedback from the parents.
|
Acknowledgement |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
The authors thank the staff in the paediatric ward for their assistance with the study.
The research committee of Frederica and Kolding hospitals supported this research.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAcknowledgementReferences |
|---|
- Donabedian A. (1980) The Definition of Quality an Approaches to Its Assessment.(Health Administration Press, Michigan).
- Sitzia J and Wood N. (1998) Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care 10:311–7.
[Abstract/Free Full Text] - Jorgensen CK and Karlsmose B. (1998) Validation of automated forms processing. A comparison of Teleform with manual data entry. Comput Biol Med 28:659–67.[CrossRef][Web of Science][Medline]
- Rubin HR. (1990) Patient evaluations of hospital care. A review of the literature. Med Care 3:267–326.
- Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S, et al. (2002) Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. Rheumatology 41:268–73.
[Abstract/Free Full Text] - Bushnell DM, Martin ML, Parasuraman B. (2003) Electronic versus paper questionnaires: A further comparison in persons with asthma. J Asthma 40:751–62.[CrossRef][Web of Science][Medline]
- Pouwer F, Snoek FJ, van der Ploeg HM, Heine RJ, Brand AN. (1998) A comparison of the standard and the computerized versions of the well-being questionnaire (WBQ) and the diabetes treatment satisfaction questionnaire (DTSQ). Qual Life Res 7:33–8.[CrossRef][Web of Science][Medline]
- Kleinman L, Leidy NK, Crawley J, Bonomi A, Schoenfeld P. (2001) A comparative trial of paper-and-pencil versus computer administration of the Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire. Med Care 39:181–9.[CrossRef][Web of Science][Medline]
- Bischoff-Ferrari HA, Vondechend M, Bellamy N, Theiler R. (2005) Validation and patient acceptance of a computer touch screen version of the WOMAC 3.1 osteoarthritis index. Ann Rheum Dis 64:80–4.
[Abstract/Free Full Text] - McBride JS, Anderson RT, Bahnson JL. (1999) Using a hand-held computer to collect data in an Orthopedic outpatient clinic. Med Care 37:647–51.[CrossRef][Web of Science][Medline]
- Caro Sr JJ, Caro I, Caro J, Wouters F, Juniper EF. (2001) Does electronic implementation of questionnaires used in asthma alter responses compared to paper implementation? Qual Life Res 10:683–91.[CrossRef][Web of Science][Medline]
- Drummond HE, Ghosh S, Ferguson A, Brackenridge D, Tiplady B. (1995) Electronic quality of life questionnaires: a comparison of pen-based electronic questionnaires with conventional paper in a gastointestinal study. Qual Life Res 4:21–6.[CrossRef][Web of Science][Medline]
- Greenwood MC, Hakim AJ, Carson E, Doyle DV. (2005) Touch-screen computer systems in the rheumatology clinic offer a reliable and user-friendly means of collecting quality-of-life and outcome data from patients with rheumatoid arthritis. Rheumatology 45:66–71.[CrossRef][Web of Science][Medline]
- Velikova G, Wright E, Smith A, Cull A, Gould A, Forman D, et al. (1999) Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J Clin Oncol 17:998–1007.
[Abstract/Free Full Text] - Ammentorp J and Kofoed PE. (2005) Continuous IT-based monitoring of the quality of treatment and care seen from parents' perspective[Danish]. Ugeskr Læger 167:1950–3.
- Ammentorp J, Rørmann D, Mainz J, Larsen L. (2001) Measurement of the quality of care in a paediatric department [Danish]. Ugeskr Læger 163:7048–53.
- Ammentorp J, Mainz J, Sabroe S. (2005) Parents priorities and satisfaction with acute pediatric care. Arch Pediatr Adolesc Med 159:127–31.
[Abstract/Free Full Text] - Weech-Maldonado R, Morales LS, Elliott M, Spritzer, Marshall G, Hays RD. (2003) Race/ethnicity, language, and patients' assessments of care in Medicaid managed care. Health Serv Res 38:789–808.[CrossRef][Web of Science][Medline]
- Ygge B and Arnetz JE. (2001) Quality of paediatric care: application and validation of an instrument for measuring parent satisfaction with hospital care. Int J Qual Health Care 13:33–43.
[Abstract/Free Full Text] - Freil M and Knudsen JL. (2004) Quality control in the national health service[Danish]. Ugeskr Læger 166:1796–9.
- Ong LML, De Haes JCJM, Hoos AM, Lammes FB. (1995) Doctor–patient communication: A review of the literatur. Soc Sci Med 40:903–18.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. B. Schefte and M. L. Hetland An open-source, self-explanatory touch screen in routine care. Validity of filling in the Bath measures on Ankylosing Spondylitis Disease Activity Index, Function Index, the Health Assessment Questionnaire and Visual Analogue Scales in comparison with paper versions Rheumatology, November 17, 2009; (2009) kep333v1. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||