OUP user menu

Psychometric evaluation of the primary health-care satisfaction scale in Turkish women

Behice Erci, Sibel Çiftçioğlu
DOI: http://dx.doi.org/10.1093/intqhc/mzq058 500-506 First published online: 19 October 2010

Abstract

Objective This paper reports an adaptation of the English version of the primary health-care satisfaction scale for use with Turkish women and an evaluation of its psychometric properties.

Design A psychometric study.

Setting and Participants A convenience sample of 381 women at a primary health-care centre completed a structured questionnaire including demographic characteristics and the primary health-care satisfaction scale for women in 2008. Item analysis, principal components analysis, internal consistency reliability and Cronbach's alpha were used to measure the psychometric properties of the items of the scale.

Results In the assessment of construct validity, three factors were identified with a: (1) administration and office procedures, (2) communication, (3) care coordination and comprehensiveness. These factors together explained 54.8% of the total variance. Internal reliability coefficients of these three factor-based scales were 0.80 and 0.93, respectively.

Conclusions The present study provides evidence of the primary health-care satisfaction scale's validity, reliability and acceptability. This scale should be further evaluated with a larger representative sample of women seeking primary care in Turkey and diverse populations of the world. The scale has potential applications for use in research. The scale may be used both in studies to evaluative alternative models of primary care delivery for women and in quality improvement programs in women's primary care.

  • psychometric evaluation
  • primary health care
  • satisfaction
  • nursing

Introduction

In the USA, studies show that women make more visits than men per year, use a wider array of primary care providers (including physician generalists and specialists, advanced practice nurses), and in Turkey, women have similar patterns of seeking health care that men have [1, 2]. Women have perceived more care than men in using a wider array of primary care providers, and often use more than one regular source of care in per year. In Turkey, research has shown that rates of women's primary care utilization increased from 54.1 to 68.7%, and women used primary health care for prenatal and child care, immunization, routine examination and prescription of medication [35]. In the 1998 Commonwealth Fund Survey of Women's Health, 37% of women ages 18 and older used two types of physicians for their regular care: a generalist and an obstetrician-gynaecologist. Some women also may seek specific services (e.g. family planning) outside their health plans, for reasons of continuity, confidentiality or availability of methods. In addition, while current definitions and measures of primary care contextualize it within a sustained physician–patient relationship [6, 7], women may have multiple simultaneous relationships with providers [8].

These complex patterns of care mean that satisfaction measures that focus on care received at a specific visit or episode may be more useful for evaluating women's health care than those that require respondents to rate their experience over time and across providers. Visit-specific information may also be more interpretable in research and quality improvement efforts, since the information provides a direct link with an episode of care and with a provider. Women's expectations of health care, their differing roles in using the health-care system for themselves and family members, and their unique health needs also have implications for the design of satisfaction instruments. Expectations may be shaped by past experiences with the health-care system (either through their own care or experiences getting care for a family member) or by an idealized conception of what care should be. The use of expectancy models in assessing satisfaction for women may be particularly helpful, since gender has an important impact on expectations of care, orientations to care, and interactions between patients and providers. For example, women's expectations of care may be shaped by their unique roles and needs in health care [8].

Primary care is characterized and distinguished from other types of care (specialty, inpatient hospital, emergency care, etc.) by several attributes. Theses attributes include accessibility, comprehensiveness of services, coordination, continuity and accountability. Ideally, primary care is the point of first contact in the health service system and provides person-focused (rather than disease-oriented) care over time [9]. Prior research on preferences for the delivery of primary care has focused on patient expectations for specific tests and procedures [10] or on their expectations for communication around specific medical problems within the clinic encounter [1113]. Weisman and colleagues [14] found that women's satisfaction with primary care visits is more dependent on continuity of care and informational content than is men's satisfaction. A qualitative study found that women valued easy access to the primary care site, and comprehensive and coordinated care [15].

The primary care satisfaction survey for women (PCSSW) is a 24-item survey tool consisting of three scales that have been shown to be psychometrically valid. The PCSSW was developed by Scholle and colleagues in the year 2004. The two scales measuring visit-specific satisfaction include the 8-item communication scale and the 6-item administration and office procedures scale. The 10-item care coordination and comprehensiveness scale measures satisfaction with health care during the past 12 months. Each PCSSW item is rated on a 5-point scale: 1 = not at all satisfied; 2 = somewhat satisfied; 3 = satisfied; 4 = very satisfied; and 5 = extremely satisfied. For the validity analyses, a score for each scale was calculated by summing the items. Each of the PCSSW scales has high internal consistency, with coefficient alpha of 0.96 for the communication scale, 0.88 for the administration and office procedures scale and 0.95 for the care coordination and comprehensiveness scale [16].

No study focused specifically on PCSSW, and current scales were not designed specifically for women in Turkey. Health-care researchers who work with culturally diverse communities need to be aware that the measurement of the primary care satisfaction may vary in different cultural groups. Therefore, the primary care satisfaction may be the best representation of the constructs of primary care satisfaction from a Turkish perspective, and thus may be culturally sensitive. Turkish cultural values may influence the measurement of primary care satisfaction; this study was conducted to determine whether the scale structure of the primary care satisfaction in its present form taps into these culturally salient values, and thus whether it is appropriate for use with Turkish women. Cross-cultural influences affect people's perceptions and health practices. So, the PCSSW was adapted to Turkish women for primary care satisfaction survey.

Aim

This paper is a report of a study to adapt The PCSSW for Turkish women and to evaluate its psychometric properties.

Methods

Design

The study used a psychometric design. To ensure the quality of the adapted scale, international norms were performed while carrying out the adaptation. The phases carried out were: (i) translation into the Turkish language from the English version and back translation into English; (ii) content analysis by a panel of specialists; and (iii) pre-test and psychometric testing (factor analysis, a reliability coefficient and inter-item correlations).

Participants

The population for this study consisted of women in Erzurum, Turkey. A convenience sample was recruited in 2008 from women attending a primary health-care centre in the town. Women (n = 381) were requested to participate in the study by the researchers and to complete the primary health-care satisfaction scale during their appointment. The eligibility criteria were: (1) being married and (2) having at least one child.

Translation procedures

Two people translated the scale and that back translation was checked. The two translated versions were compared by the author and analysed until there was a consensus regarding the initial translation. The initial translation into Turkish was back translated into English. The translation phase checked discrepancies between content and meaning of the original version, and the translated instrument. They assessed the accuracy of the translation or the appropriateness of the scale item. All versions were evaluated by the authors and a final version was formed.

Content validity

To test item clarity and content validity, the translated version was submitted to a panel consisting of seven specialists who were working in the area of knowledge of the instrument. They were informed concerning the measures and concepts involved by the author. This panel comprised three public health specialists, two who had published works on instrument development, and two nurses who had conducted research in the fields of primary health care. Each of the panel members was asked to evaluate of the final translated version of the PCSSW compared with the original instrument. Experts were asked to evaluate each item at the scale by using a five-point Likert scale: 1 = not at all satisfied, 2 = somewhat satisfied, 3 = satisfied, 4 = very satisfied, 5 = extremely satisfied. Any changing was done for the rankings of the scale items. The items of the scale were made any changing based on these evaluations. The panel members were agreement for these evaluations. Conceptual adjustments did not require after translation and review.

Pre-test

Once the final version had been developed, a pilot study on subjects selected from the target population should be undertaken to test the equivalency, reliability and score distribution. The final version of the translated instrument was applied to a small pilot group consisting of 15 women in order to pre-test the instrument. Pre-test was conducted at the primary health-care centres where the original study was planned to be done. In order to simplify the recording of doubts and suggestions concerning the scale, a questionnaire for this research phase was used. The questionnaire requested general information from the interviewee, such as gender, age, civil status and occupation. An open-ended question to record doubts and suggestions was provided for each one of the items.

Psychometric testing

Internal consistency and homogeneity

Cronbach's alpha was calculated to determine internal consistency. Clark and Watson [17] indicated that internal consistency may be a necessary condition for homogeneity or unidimensionality of a scale and Cronbach's alpha should be 0.70 and more. Besides, the item-total correlations and the mean inter-item correlations were included in the analysis. Clark and Watson [17] recommended using the inter-item correlation as a criterion for internal consistency. This should be ≥0.15. They pointed out that this average value could be a bias and all individual inter-item correlation should be within these limits. One can only be ensured of unidimensionality if all individual inter-item correlations are clustered closely around the mean inter-item correlation.

Construct validity

The data were analysed using factor analysis (principal component analysis and varimax rotation). To attain the best fitting structure and the correct number of factors, the following criteria were used: eigenvalues >1.0, factor loadings >0.40 and the so-called elbow criterion regarding the eigenvalues [18]. Before conducting the factor analysis of the PCSSW, Kaiser–Meyer–Olkin and Bartlett's test was calculated to evaluate whether the sample was large enough to perform a satisfactory factor analysis. The Kaiser–Meyer–Olkin and Bartlett's test measures the sampling adequacy that P-value should be >0.05 for a satisfactory factor analysis to proceed.

Ethical considerations

Permission to undertake this study was gained from the ethical committee at the Atatürk University and informed consent was obtained from each participant. The patients were informed about the purpose of the research. The participants were assured of their right to refuse to participate or to withdraw from the study at any stage. Anonymity and confidentiality of participants were guaranteed.

Procedure and data collection

Data were collected using questionnaire including demographic characteristics and the primary health-care satisfaction scale. The researchers visited the primary health-care centre two days (Monday and Friday) in every week and conducted interviews with the patients. The researcher introduced the questionnaire to the participants and explained the material covered. Then, the participants read the questionnaire and marked their answers on the sheets. The questionnaire took ∼20 min to complete and could be understood by people with minimal reading ability. The questionnaire was given to the women in a separate quiet room of the primary health-care centre. All of the participants completed the questionnaire.

Data analysis

Pearson's product–moment correlation was used to determine correlation scores of items and the total scale. Factor analysis was used to establish the construct of the scale and factor loadings of items of the scale. Cronbach's alpha was calculated to find internal consistency reliability. The stability of the scale was established by measuring the test–retest reliability. Pearson correlation was used for the test–retest reliability assessment.

Results

Participant demographics

The demographic characteristics of the participants were shown in Table 1. The mean age of the women was 33.3 ± 10.5 years. Among participating women, 33.5% had a monthly income between 501 and 999 USD. This monthly income was less than monthly income average of Turkey [19]. The majority of the mothers (60.7%) graduated from primary school, and 85.6% of them were working in the home. The mean number of children was 2.8 ± 1.4.

View this table:
Table 1

Demographic and health-care utilization characteristics of participating women (n = 381)

Demographic characteristicsX ± SD
Age (years)33.3 ± 10.5
Monthly income (USD)794.3 ± 465.7
Number of children2.8 ± 1.4
N%
Education level
 Primary school23160.7
 Secondary school5514.5
 High school4912.7
 University degree4612.1
Occupational status
 Work in the home32685.6
 Education professional143.7
 Civil servant112.8
 Commerce61.6
 Student246.3
Monthly income (USD)
 500 and less12633.0
 501–99913535.5
 1000 and more12031.5
Health insurance
 Yes36094.5
 No215.5
Health-care utilization
Overall satisfaction with health care visit
 Poor41.0
 Some good4211.0
 Good21556.4
 Much good9926.0
 Perfect215.5
Length of time attending to site
 First time today4211.1
 Less than 1 year15039.5
 1–2 years6316.6
 More than 2 years12532.9
Clinical site is usual source of care
 Yes20453.7
 No17646.3
Main reason for health care visit
 Maternal and child health16944.4
 New health problem348.9
 Routine exam12933.9
 Prenatal or postpartum care246.3
 Other (injection. Blood pressure examination)256.5
Office visits, past year
 5 or less20253.0
 6–109023.7
 11–15195.0
 16–20184.7
 21 or more5213.6
Number of office visits, past yearMean: 9.7SD: 11.0

Content validity

The translated scale, consisting of 36 items, was judged by the expert panel on relevance and phrasing of the instrument items. For each item, experts could suggest possible improvements in wording. Subsequent wording revisions of the Turkish instrument were made and discussed each time by the panel members until agreement about the content was reached. The panel then reviewed the content of Turkish version until there was no further need to modify its translation and content.

Internal consistency

The analysis revealed three factors that were administration and office procedures, communication, and care coordination and comprehensiveness. The PCSSW was found to have an overall coefficient alpha of 0.93. Alphas of the three factors ranged from 0.80 to 0.90 (see Table 2). The corrected item-total correlations were acceptable, and the item-total correlations ranged from 0.43 to 0.73 for the remaining 24 items.

View this table:
Table 2

Principal components analysis followed by varimax rotation factor loadings and item-total correlations of items of the scale (n = 381)

Items of the scale and the items of factorsFactor loadingItem-total correlationsAlphaVariance
First item set: today's visit
Administration and office procedures0.8022.6
(a) The courtesy of the staff…0.4100.566**
(b) The staff's flexibility in scheduling my appointment around my needs…0.5170.550**
(c) Privacy when talking to the receptionist0.5530.583**
(d) How well the staff kept you informed about the waiting time…0.4820.670**
(e) Help with scheduling my next visit…0.4680.597**
(f) The chance to talk to my health professional with my clothes on…0.5770.682**
Communication0.9018.9
(g) The amount of time I had to talk with my health professional…0.5920.708**
(h) My health professional's ability to answer questions in a sensitive and caring way…0.6930.748**
(i) My health professional's ability to explain things clearly…0.5800.711**
(j) My health professional's ability to help me feel comfortable talking about my concerns…0.6000.721**
(k) The chance to ask all of my questions…0.6430.729**
(l) My health professional's ability to take what I say seriously…0.5610.733**
(m) My health professional's willingness to explain different options for my care…0.5900.731**
(n) My health professional's interest in how my life affects my health…0.5760.660**
Second item set: care during past year
Care coordination and comprehensiveness0.8313.3
(a) The health professionals’ focus on prevention…0.3900.399**
(b) The health professionals’ knowledge of women's health issues…0.5940.632**
(c) The information I get about healthy living (such as diet and exercise)…0.5260.644**
(d) The health professionals’ interest in my mental and emotional health…0.5400.622**
(e) Help with finding information resources in women's health…0.6840.681**
(f) How well my health care fits my stage of life…0.5810.663**
(g) Information about how to get the results of my tests…0.5990.671**
(h) How well the health professionals explain the results of tests or procedures…0.3460.432**
(i) The chance to get both gynaecological and general health care here…0.6320.675**
(j) My overall trust in the health professionals here…0.6510.659**
Total0.9354.8%
  • **P < 0.01.

Construct validity

The calculated Kaiser–Meyer–Olkin and Bartlett's test was 0.93 with a P-value is P < 0.001, indicating that the sample was large enough to perform a satisfactory factor analysis. The first step of the factor analysis was a principal component analysis. Eigenvalues >1 were used to determine the number of factors. The analysis revealed three factors with an eigenvalue >1 (Table 2). Factor loadings of all items were adequate, and were above 0.30 and ranged 0.34–0.69. Alphas for the retained items were calculated. This showed that Cronbach's alphas for three factors were >0.70. Thus, the scale was formed from three dimensions and 36 items. Principal components analysis was used to explain the variations in the total scale and its factors. The three factors together explained 54.8% of the variance. Internal consistency reliability was 0.93 for the whole scale. For the first factor, Cronbach's alpha was 0.80 and factor loadings were found to be related to the administration and office procedures subscale. This factor explained 22.6% of the variance. Item loadings for the second factor with an alpha of 0.90 were found to be related to the communication subscale. This factor explained 18.9% of the total variance. The third factor, with an alpha of 0.83, exclusively referred to items which referred to the care during past year subscale. The explained variance of this factor was 13.3% of the total variance. Table 2 shows the principal components analysis, followed by varimax rotation factor loadings for the scale items.

The stability of the scale was established by measuring the test–retest reliability. Sixty-two women from the same sample group took part in the test–retest reliability assessment. The respondents completed the same instrument again after 4 weeks. Test–retest reliability were 0.85 for the PCSSW, r = 0.76 for the administration and office procedures, r = 0.78 for the communication and r = 0.81 for the care coordination and comprehensiveness of PCSSW retest all of which are considered good.

Discussion

The results of this study show that the psychometric characteristics of the Turkish version of the PCSSW are promising. The panel review of its content indicated that there was no need to modify its translation or content. In addition, its internal consistency seems sufficient in terms of item correlations. Internal consistency and inter-item correlations had adequate criteria [20, 21]. Additionally, Scholle et al. [16] reported that inter-factor correlation was 0.60 for PCSSW.

Using varimax rotation, the factor analysis indicated three factors: administration and office procedures, communication, and care coordination and comprehensiveness. These three factors together explained 54.8% of total variance. Internal consistency reliability was 0.93 for the whole scale. In the original scale [16], three factors were found with same content: administration and office procedures, communication, and care coordination and comprehensiveness. Scholle et al. [16] found that each of the PCSSW scales has high internal consistency, with coefficient alpha of 0.96 for the communication scale, 0.88 for the administration and office procedures scale and 0.95 for the care coordination and comprehensiveness scale and the scale explained 66% of total variance. In the current study, internal coefficients were adequate for the whole scale and its subscales.

Looking specifically at the items in the Turkish scale compared with the original scale, cultural characteristics may have been an influencing factor. This also questions the Kaiser–Meyer–Olkin procedure. This indicated that the sample was large enough to perform a satisfactory factor analysis, but further validation of the original scale clearly showed that factor solutions were associated with sample size. In our study, factor analysis yielded factor loadings above 0.30, and factor loading of the items in the scale ranged 0.34–0.69. Scholle et al. [16] found that factor loading of the items ranged 0.33–0.98. Factor loadings in this were similar to those in other study [16].

Women's frequent encounters with health care may raise (or lower) their expectations for some aspects of care, such as the timeliness of appointments, the clarity of communication with providers or the availability of timely follow-up care. In addition, more preventive care is recommended for women than men in early adulthood and in connection with reproduction (e.g. routine gynaecological exams, prenatal care). Based on their experiences, women may develop greater expectations for preventive services. Little conceptual validation has been performed with patient satisfaction instruments currently in use, however, and none has considered gender issues in expectations for care or in the gap between expectations and perceived care [8]. Finally, some attention should be addressed to the unique concerns of women. Women-specific aspects of care (e.g. such as access to female providers or comfort during gynaecological/pelvic examinations) are not included in instruments intended for use in both women and men, and thus require special instrumentation. The construction, analysis and interpretation of satisfaction instruments should consider women's unique needs and roles [8].

Several studies suggest that different aspects of care influence outpatient satisfaction ratings among women versus men [14, 15, 22, 23]. These data suggest existing measurement tools may neglect the context of women's primary care experience. Women make more health care visits than men; the majority of patients seen in many primary care settings are women. For that reason alone, information about the quality of their primary care experiences is important. However, women's primary care utilization patterns are complex owing to the structural fragmentation of reproductive and general health care [24]. To obtain comprehensive care, many women use two physicians for their regular care [25]. Today, most health-care programs use patient satisfaction surveys to assess the quality of care from patients’ perspectives. Many surveys are available, some focusing on satisfaction with a specific health care visit or inpatient episode, and some focusing on health care received over a period of time [16].

The content of the PCSSW differs from existing satisfaction tools in several ways. Some of the items are specific to women (e.g. the ability to get obtain gynaecologic and general health care at the same site; the health professional's knowledge of women's health issues). Other items are new topics not typically included in patient satisfaction surveys but potentially applicable to all patients (e.g. how well office staff keeps the patient informed about waiting time; the health professional's interest in the patient's mental and emotional health). Additional items are similar to items in generic patient satisfaction surveys but are worded to be consistent with women's framings as discovered in the focus groups (e.g. the health professional's ability to answer questions in a sensitive and caring way). The items also address both a specific visit and care coordination and comprehensiveness during the past year. Because many women seek health care from more than one professional or site, the latter component of the PCSSW is particularly innovative. The PCSSW adds sensitivity to satisfaction measurement and can be useful in evaluations of the quality of primary care and in quality improvement programs [16].

Test–retest reliability of the scale was 0.85. Scholle et al. [16] reported a test–retest reliability of 0.61. According to the present results, therefore, the scale had construct validity.

The results of this project, which demonstrate the usefulness of the tool, provide a basis for translating the tool and testing it in other languages.

Limitation

The findings must be interpreted cautiously because of the study limitations. The sample was selected by convenience sampling, the most of the women low education levels (60.7% primary school) and work in the home (85.6%). The sample reflects only one area of Turkey and therefore cannot be generalized to all women in Turkey.

Conclusion

The Turkish version of the PCSSW should be further evaluated with a large sample size in different regions of Turkey and diverse populations in other in different cultures. The existing Turkish scale can be used for further validation and also the usage of the scale will be available at outcome research. Outpatient satisfaction tools designed to be used in general populations fail to capture the full range of health-care concerns of women. The Turkish version of the PCSSW is a psychometrically valid survey tool for assessing women's satisfaction with primary care. It may be self-administered or conducted by telephone interview. The PCSSW may be used both in studies to evaluative alternative models of primary care delivery for women and in quality improvement programs in women's primary care.

References

View Abstract