OUP user menu

Psychometric evaluation of an instrument to assess patient-reported ‘psychosocial care by physicians’: a structural equation modeling approach

Oliver Ommen, Markus Wirtz, Christian Janssen, Melanie Neumann, Elke Driller, Nicole Ernstmann, Sabine Loeffert, Holger Pfaff
DOI: http://dx.doi.org/10.1093/intqhc/mzp010 190-197 First published online: 12 March 2009

Abstract

Objective The objective of our study was to develop a theory-based and empirically tested instrument for measuring patient-reported ‘psychosocial care by physicians’. We propose a model integrating patients' perceptions with respect to: (i) devotion by physicians, (ii) support by physicians, (iii) information by physicians and (iv) shared decision-making (SDM).

Design Data were gathered during 2001 within a cross-sectional, retrospective mail survey.

Participants and setting A total of 4192 inpatients of six German hospitals.

Main outcome measure Specific scales of the Cologne Patient Questionnaire were used. A two-step structural equation model procedure was applied. In the first structural equation model, all items were modeled as indicators of the intended underlying latent construct, ‘psychosocial care by physicians’. In the second structural equation model, criterion-related validity of the intended construct was tested with respect to patients' ‘satisfaction’, ‘trust in physicians’ and the ‘image of the hospital’.

Results The results confirmed that the aspects of psychosocial care provided by physicians measured by the scale items are indeed indicators of the same construct. Furthermore, indicator reliabilities and selectivities revealed that the content of all 13 items was highly representative of the underlying construct. The second structural equation model showed that ‘psychosocial care by physicians’ is related to ‘patients’ satisfaction', ‘trust in physicians’ and ‘hospital-image’ in a significant and relevant manner.

Conclusion On the basis of our instrument's reported psychometric characteristics and of the initial validity indicators, it may be regarded as an adequate measure for further use in outcome and intervention research, and as a quality indicator for the physician-patient relationship.

Keywords
  • physician–patient interaction
  • psychosocial care
  • quality measurement
  • psychometric evaluation
  • structural equation model

Introduction

The increasing importance of patient-reported outcomes results in a greater need for valid and reliable instruments [14]. The technical aspects of care are increasingly subjected to empirical studies within the scope of evidence-based medicine [5]. However, research to date can be characterized by a lack of comprehensive instruments for measuring psychosocial care by physicians. Furthermore, there is no universal definition of psychosocial care in the literature or a definition which is used by other researchers in the field. Only a few indicators of quality have concentrated explicitly on aspects of the psychosocial interaction between patients and medical staff [6]. This is all the more surprising because it is already well known that psychosocial quality of care may impact outcomes just as much as the technical aspects of medical care [7, 8]. DiBlasi et al. [9] were able to show that an empathic communication employed by physicians was able to shorten the length of illnesses and reduce side effects. Beck et al. [10] found that health-related outcomes are positively associated with empathic and patient-centered questioning techniques by physicians. Several studies have verified that the existence of social resources, e.g. supportive relationships, are of decisive benefit to the treatment of chronic illness and that social resources may help to promote health [11, 12]. Individual indications of such effects achieved with physicians' support have also come from placebo-based research [13] and from studies involving rehabilitation motivation [14]. Besides empathic and supporting fundamental attitudes in physicians, it is important for patients to receive comprehensible and comprehensive information with respect to their current state of health, the progress of therapy and treatment prospects. There is strong evidence that patients want more and better information about their illness and results of their treatment, as well as advice about what they can do for themselves [1517]. Furthermore, many studies have shown that patients' participation in treatment and decision-making processes has a positive impact on recovery and satisfaction [1821]. These findings are expressed in the scientific model of so-called ‘shared-decision-making’ [22]. Summing up the above-mentioned empirical results with respect to psychosocial care by physicians, we have hypothesized that four elements of physician-related psychosocial interaction have a positive impact on treatment and patient-reported outcomes (e.g. satisfaction and trust in physicians): (i) physicians' support of patients' coping, (ii) empathy by physicians, (iii) clear and understandable medical information by physicians and (iv) ‘shared decision-making’ (SDM) behavior by physicians. Furthermore, we assume that these four dimensions may be assigned to a superior construct, i.e. ‘psychosocial care by physicians’.

Aim of the study

The current study suggests a theory-based and empirically tested instrument for measuring patient-reported ‘psychosocial care by physicians’. We propose a conceptual framework of ‘psychosocial care by physicians’. Within this framework, the psychosocial care by physicians is supposed to be significantly correlated with other relevant patient-reported outcomes, such as ‘patient satisfaction’, ‘trust in physicians’ and patients' evaluation of the ‘hospital image’.

Methods

Sampling and data collection

Data were gathered during 2001 within a cross-sectional, retrospective mail survey with 4192 inpatients (internal medicine and surgical wards) from six German hospitals. Patients were eligible if they were 18 years of age or older, and were excluded when they died or moved without a forwarding address (n = 474).

Statistical analysis

Missing values were imputed by the expectation-maximization method [23]. For the descriptive statistics of the scales, SPSS 15.0 for Windows was used. The maximum-likelihood estimation procedure [24, 25] implemented in the software, AMOS 5.0 [26], was used to develop and test all structural models. In the first structural equation model, all items were modeled as indicators of the intended underlying latent construct, ‘psychosocial care by physicians’ (second-order factor model). In agreement with Kline [24], a two-step structural equation model procedure was applied. In the first step, a confirmatory factor analysis was conducted to determine whether the intended construct was indeed measured by the underlying latent variables. Confirmatory factor analysis assumes each manifest variable to be a distinct indicator of an underlying latent construct, whereby different constructs are permitted to be intercorrelated. Moreover, a Δχ2 test was carried out to determine if the latent variables were acceptable as independent constructs that could be sufficiently distinguished from each other. Significant Δχ2 test values indicate discriminate validity [27]. The appropriateness of a specific confirmatory factor analysis model was assessed by measures of global and local fit. Measures of global fit indicate whether the empirical associations among the manifest variables are appropriately reproduced by the model [24]. For a variety of these global fit measures, certain criteria have to be met in order to accept the model under study as plausible and parsimonious. For the root mean square error of approximation (RMSEA), the model can be classified as good if the RMSEA is ≤0.05, and as acceptable if it is ≤0.08. Furthermore, measures of incremental fit were used [24]; specifically, the comparative fit index (CFI) and the Tucker–Lewis index (TLI). A rule of thumb for incremental fit measures is that values ≥0.97 are indicative of good fit relative to the independence model, while values ≥0.95 may be interpreted as an acceptable fit [28]. Measures of local fit evaluate whether each construct can be reliably estimated from its indicators [27] and whether the constructs within the model are sufficiently distinguishable [27]. After having ensured an acceptable measurement quality of the confirmatory factor analysis model, the second-order factor model was used to validate all first-order factors as indicators of an underlying latent construct. The quality of the second-order factor model was determined with global fit measures.

Second path analysis was used to investigate the criterion-related validity of the intended construct, ‘psychosocial care by physicians’. The corresponding structural equation model is based on the assumption that the intended construct is significantly related to ‘patients’ satisfaction', patientś ‘trust in physicians’ and ‘image of the hospital’. The significance of the relationships between the exogenous and endogenous latent variables, as well as the amount of variance explained in the endogenous variables, was examined. Once again, a two-step structural equation model procedure was applied [24].

Measures

The instrument used in this study is based on the Cologne Patient Questionnaire [29, 30]. The questionnaire was developed by explorative factor analysis and has been validated in several research projects. This analysis is based on specific scales that were used to operationalize the theoretical concept of patient reported ‘psychosocial care by physicians’ and to assess the criterion-related validity of the tested instrument. For the operationalization of the theoretical concept, the following four scales were used: (i) ‘devotion by physicians’ was comprised of five items and was designed to measure patients' subjective perceptions regarding physicians' empathy and the establishment of a trusting relationship, (ii) ‘support by physicians’ was comprised of three items and measured patients' subjective perceptions of the supportive behavior of physicians, (iii) ‘information by physicians’ was comprised of three items and measured patients' subjective perceptions of the informational behavior of physicians, and (iv) SDM was comprised of four items and was designed to measure patients' perceptions regarding SDM behavior of physicians.

For the assessment of criterion-related validity, the following scales were used: (i) ‘satisfaction’ was comprised of 12 items and measured the satisfaction or dissatisfaction of patients with individual aspects of hospital care (e.g. satisfaction with care by nurses/physicians, organizational aspects or medical treatment), (ii) ‘trust in physicians’ was comprised of five items and measured various aspects of trust in the physicians providing the treatment, such as general trust, as well as trust in the professional competence of medical staff (e.g. ‘I completely trusted my physicians’) and (iii) hospital ‘image’ is comprised of three items and was designed to measure a patients' tendency to recommend the hospital (e.g. ‘I would recommend this hospital to my best friend’).

Results

Sample characteristics

Examining the ‘Total-Design-Method’ [31], a return rate of 52.9% (n = 2470) was realized. Eleven percent of the respondents (n= 273) were excluded due to the limited data quality (missing values, >30% in scale items). All in all, the sample was comprised of 2197 patients in the present analysis. For the patients included in the analysis, a mean average of 4% missing values in the items of the scales was observed. The participants' mean age was 50 years (range, 18–97 years) with 48% of the patients aged >50 years. Females made up 26.3% of the respondents. For 44.6% of the respondents, the highest level of education was elementary school, for 33.2% of the respondents it was secondary school and for 17.5% of the respondents it was grammar school (high school) qualification and 2.3% of the respondents had other schooling qualifications while 2.3% had none at all. The sample consisted of 19.7% workers, 34.1% salaried employees, 28.1% civil servants, 7.9% self-employed, 5.3 apprentices, school or university students, and 4.9% with other professional statuses.

Confirmatory factor analysis and the second-order factor model

Table 1 provides an overview of the scales, items and indices of the ‘Confirmatory Factor Analysis Model 1’.

View this table:
Table 1

Overview of the scales and items of the ‘Confirmatory Factor Analysis Model 1’

Scales and itemsItem labelsγITCMCIMMdSD
Devotion by physicians
Physicians carried out regular conversations, which also took place outside of ward roundsdevo10.770.653.043.00–3.083.000.99
Physicians carried out conversations with me in a very empathic mannerdevo20.830.723.053.01–3.083.000.89
It was possible to talk with the physicians about personal mattersdevo30.790.672.572.53–2.623.001.0
There was one physician who was my point of contactdevo40.850.753.032.99–3.063.000.89
Physicians gave me time to think important decisions overdevo50.780.653.203.17–3.243.000.81
Support by physicians
I was able to rely on the physicians when I had problems with my illnesssupp10.890.763.333.30–3.363.000.77
Physicians supported me so that it was easier to deal with my illnesssupp20.920.813.233.19–3.263.000.81
Physicians were willing to listen to my problems relating to my illness.supp30.900.783.253.21–3.283.000.81
Information by physicians
Physicians used visual aids (pictures, drawings and outlines)inf10.730.502.612.56–2.653.001.1
Physicians explained everything in a clear and understandable wayinf20.910.723.233.19–3.273.000.88
Physicians gave an illustrative picture of illness.inf30.910.723.203.16–3.233.000.92
Shared decision-making
My wishes were taken into account during treatmentcoth10.780.603.203.16–3.233.000.80
I was able to influence the treatment processcoth20.760.572.512.47–2.552.000.93
Physicians wanted me to be actively involved in the treatment processcoth30.790.602.872.83–2.913.000.93
Physicians agreed treatment targets with mecoth40.820.653.053.01–3.093.000.90
  • γ, loadings in accordance with main component analysis; ITC, item-total-correlation; M, mean value; CIM, 95% confidence interval for the mean value; Md, median; SD, standard deviation

With respect to theoretical considerations, the indicators of the ‘Confirmatory Factor Analysis Model 1’ with insufficient model compatibility were sequentially eliminated from the model until criteria for a good model-fit were reached. Items were eliminated if indicator reliabilities were low (<0.40) [27]. Checking the ‘Confirmatory Factor Analysis Model 1’ before this background, only one item of the scale ‘information by physicians’ and one item of the scale SDM had to be eliminated.

The resulting ‘Confirmatory Factor Analysis Model 1’ was comprised of 13 items and exhibited an acceptable global data fit (Table 2). Furthermore, the indices of local fit proved that each latent construct was reliably measured by its indicators; for each manifest item, >40% of the information was associated by the underlying construct (indicator reliability, >0.40) and all factor loadings were significant. The factor reliabilities (acceptable fit threshold, ≥0.60), as well as the average proportions of indicator variance extracted by the corresponding latent construct (acceptable fit threshold, ≥0.50), exceeded the recommended critical values [27, 33]: ‘devotion by physicians’ (factor reliability, 0.86; average variance extracted, 0.56), ‘support by physicians’ (factor reliability, 0.89; average variance extracted, 0.73), ‘information by physicians’ (factor reliability, 0.89; average variance extracted, 0.81) and SDM (factor reliability, 0.77; average variance extracted, 0.52). Finally, each latent variable was tested to determine if the information provided was not covered by the most strongly correlated alternative construct. The significant values according to Δχ2 tests (min[Δχ2df=1] = 81.74; P < 0.001) indicate that the corresponding latent variables could be considered sufficiently distinguishable from each other [27]. Thus, discriminant validity of the structural components can be inferred. Although the ‘Confirmatory Factor Analysis Model 1’ was modified in order to achieve an appropriate model fit, the central properties of the Structural Model remained virtually identical after modification. The same structural path coefficients were significant and a similar amount of variance was explained in the endogenous constructs. Hence, modifications did not lead to meaningful changes in the theoretical definition of the proposed model. After having ensured an acceptable measurement quality of the ‘Confirmatory Factor Analysis Model 1’, the second-order factor model (Fig. 1) was evaluated. Measures of global fit are indicated in Table 2. According to the TLI, the CFI and the RMSEA of the second-order factor model exhibits an acceptable-to-good global data fit and the model can be accepted. Because of the large sample size of the study (n > 300), χ2 and Δχ2/df were not considered valid model-fit-indicators [28]. Additionally, factor analysis (results not shown here) confirmed a four-factor structure.

Figure 1

Second-order factor model: estimated path coefficients, indicator reliability and percentage of explained variance for the endogenous structural constructs.

View this table:
Table 2

Relevant measures (TLI, CFI and RMSEA) and fit thresholds for confirmatory factor analysis models, second-order factor model and full path model

χ2dfPχ2/dfTLICFIRMSEA
Acceptable fit threshold(<3)≥0.95≥0.95≤0.08
Good fit threshold(≥0.05)(<2)≥0.97≥0.97≤0.05
Confirmatory factor analysis model 145359<0.0017.60.970.980.06
Second-order factor model (Fig.1)49061<0.0018.00.970.980.06
Confirmatory factor analysis model 22553303<0.0018.40.940.950.06
Full-path model (Fig. 2)2541315<0.0018.00.940.950.06

Full path model to verify criterion-related validity

Indicators of the confirmatory factor analysis model with insufficient model compatibility were sequentially eliminated from the model until criteria for a good model-fit were reached. The resulting ‘Confirmatory Factor Analysis Model 2’ comprised 27 items and exhibited an all-in-all acceptable global data fit (Table 2). Although the TLI is not reached applying the fit-thresholds of Shermelleh et al. [28] (≥0.95), other authors suggest less restrictive fit-thresholds for the TLI and the CFI, such as Hair et al. [32] (≥0.90), which are both reached by the proposed model.

Figure 2

Full path model: estimated path coefficients and percentage of explained variance for the endogenous structural constructs.

Furthermore, indices of local fit proved that each latent construct was reliably measured by its indicators. For each manifest item, >40% of the information was measured by its underlying construct (indicator reliability, >0.40), and all factor loadings were significant. The factor reliabilities (acceptable fit threshold, ≥0.60), as well as the average proportions of the indicator variance extracted by the corresponding latent construct (acceptable fit threshold, ≥0.50), exceed the recommended critical values [27, 33]: ‘satisfaction’ (factor reliability, 0.91; average variance extracted, 0.59), ‘trust in physicians’ (factor reliability, 0.90; average variance extracted, 0.64) and ‘image’ of hospital (factor reliability, 0.90; average variance extracted, 0.82). The factor reliabilities and average variance extracted measurement values of the second-order factor model remained virtually identical. Finally, each latent variable was tested to determine if the information provided was not covered by the most strongly correlated alternative construct by the Δχ2-statistic [27]. Neither of the corresponding pairwise restricted models yielded non-significant Δχ2 test values (min[Δχ2df=1] = 81.74; P < 0.001). Thus, the discriminant validity of the structural components can be inferred. Descriptive results for all scales of the ‘Confirmatory Factor Analysis Model 2’ are as follows: ‘devotion by physicians’ (number of items, 5; mean value, 2.97; standard deviation, 0.75; Cronbach's alpha, 0.862), ‘support by physicians’ (number of items, 3; mean value, 3.26; standard deviation, 0.72; Cronbach's alpha, 0.89), ‘information by physicians’ (number of items, 2; mean value, 3.21; standard deviation, 0.85; Cronbach's alpha, 0.89), SDM (number of items, 3; mean value, 3.03; standard deviation, 0.72; Cronbach's alpha, 0.76), ‘satisfaction’ (number of items, 7; mean value, 4.13; standard deviation, 0.80; Cronbach's alpha, 0.91), ‘trust in physicians’ (number of items, 5; mean value, 3.50; standard deviation, 0.58; Cronbach's alpha, 0.90), and ‘image’ of the hospital (number of items, 2; mean value, 3.36; standard deviation, 0.73; Cronbach's alpha, 0.90). Table 3 shows the bivariate correlations of all scales in the ‘Confirmatory Factor Analysis Model 2’ and the results of the discriminant validity test.

View this table:
Table 3

Bivariate correlations and results of the discriminant validity test [in brackets: χ2-values, df =1] of the scales used in the confirmatory factor analysis model 2

SupportInformationSDMSatisfactionTrustImage
Devotion0.72 [720.05]0.55 [1437.32]0.64 [402.14]0.67 [1507.29]0.71 [1011.64]0.56 [1549.78]
Support0.57 [1419.08]0.59 [592.46]0.65 [1672.06]0.73 [1001.84]0.59 [1509.76]
Information0.53 [784.34]0.57 [1453.99]0.62 [1300.24]0.49 [1679.51]
SDM0.58 [660.67]0.57 [669.89]0.49 [909.07]
Satisfaction0.72 [1521.63]0.75 [862.52]
Trust0.67 [1177.28]
  • All results are significant with P < 0.001.

By integration of the two residual correlations between items in the ‘satisfaction’ scale (‘sat2’ and ‘sat4’, as well as ‘sat7’ and ‘sat8’), an additional marginal improvement in the full-path model properties was achieved [27]. The correlated items are representing a common source of variance that is not represented by other items. Thus, organizational aspects of the ward are significantly influenced by the work of the nurses, which is not reflected in the total satisfaction score. Informational aspects of the hospital are related to integration in the treatment and are also reflecting a common source of variance that is not reflected in the total satisfaction score.

Table 2 shows the measures of global fit for the confirmatory factor analysis models, the second-order factor model and the full-path model. Figure 2 shows the full-path model with the resulting parameter estimates. Although the TLI of the full-path model is hardly not reached applying the fit-thresholds of Shermelleh et al. [28] (≥0.95), other authors suggest less restrictive fit-thresholds for the TLI and the CFI, such as Hair et al. [32] (≥0.90), which are both reached by the full-path model.

The construct of ‘psychosocial care by physicians’ shares with the construct of patients' ‘satisfaction’ approximately 79% of variance (Pearson correlation coefficient = 0.89; critical ratios = 26.84; P < 0.001), with the construct of ‘trust in physicians’ approximately 84% of variance (Pearson correlation coefficient = 0.92; critical ratios = 31.42; P < 0.001) and with the construct of ‘image’ approximately 64% of variance (Pearson correlation coefficient = 0.80; critical ratios = 30.37; P < 0.001).

Discussion

A major finding of the present study is that it is possible to adequately model the four hypothetical dimensions of psychosocial care provided by physicians. Those four hypothetical dimensions are: (i) supportive behavior of physicians that make it easier for patients to cope with their illness, (ii) physicians' empathy, (iii) physicians' informational behavior, e.g. explaining medical issues in a clear and understandable manner and (iv) SDM behavior of physicians. Results of confirmatory factor analyses confirmed that the aspects of psychosocial care provided by physicians who are operationalized in the scale items are indeed indicators of the same construct. In addition to very good indicators of the model's one-dimensional structure, indicator reliabilities and selectivities reveal that the content of all of the items is highly representative of the underlying construct of ‘psychosocial care by physicians’. A second path analysis was used to investigate the criterion-related validity of the intended construct. The corresponding structural equation model (full path model) confirmed initial assumptions that the intended construct is significantly related to ‘patientś satisfaction’, patientś ‘trust in physicians’ and ‘image of the hospital’ with a very satisfactory proportion of explained variance of 64–84%. This can be seen as a significant result for ensuring the criterion validity of the proposed instrument.

Limitations of the study

The fact that we analyzed a local random sample, which was defined by inclusion and exclusion criteria, means that the population of the analysis was a well-defined sub-population of patients. The results may therefore only be carefully transferred to the situation of other patients, e.g. in primary care. Since all variables were assessed using self-reports, results may have been biased by the common method variance. This level of bias is a real cause for concern in survey-studies because the common method variance may enhance the observed correlation between variables [34]. Despite the above-mentioned methodologic limitations, the overall findings of the study suggest that the validation results help to develop theories on psychosocial care and physician–patient communication. Future research should be aimed at testing the practicality of the 13-item instrument for measuring the psychosocial care provided by physicians within the clinical setting.

Conclusion

Many studies in the area of health care have already proven the particular significance of psychosocial care provided by medical personnel. To establish and maintain a trusting and mutually satisfying physician–patient relationship, it is important for the physician to consider different points. The physician should behave in an empathic and supportive way. The physician has to provide patients with comprehensive and extensive information about their current state of health, the course of their therapy and their treatment prospects. The physician has to actively involve patients in the treatment process given that it is wanted by the patient. The psychosocial care provided by physicians is therefore a communicative and interactive service that takes place on both the relationship and informational levels. The objective of the present study was to contribute to the development of a psychometrically tested instrument for measuring the patient reported ‘psychosocial care by physicians’. The instrument may be used in medical practice as a timesaving feedback instrument for assessing the strengths and weaknesses of psychosocial care provided by physicians within the scope of quality management. In particular, routine surveys, such as annual or semi-annual organization development surveys, could be conducted and used for the early detection and counteraction of possible undesirable trends in the quality of psychosocial care provided by physicians. One possible result of assessing the quality of this psychosocial care may be a direct improvement via training in physicians' communication behavior. The goal of such training could possibly include teaching physicians how to actively involve patients in the consultation and in treatment planning, to provide patients with clear and understandable medical information and to take possible psychosocial dimensions of an illness into consideration.

Funding

The data of this work originate from a study supported by a grant from the Ernst- und Berta-Grimmke-Foundation (Application-Number: 98130).

References

View Abstract