Skip Navigation


International Journal for Quality in Health Care Advance Access originally published online on July 11, 2008
International Journal for Quality in Health Care 2008 20(5):324-330; doi:10.1093/intqhc/mzn029
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
20/5/324    most recent
mzn029v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Sousa, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Sousa, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2008. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved

Risk-adjustment model in health outcomes evaluation: a contribution to strengthen assessment towards quality improvement in interventional cardiology

Paulo Sousa1,2, António Sousa Uva1, Fausto Pinto3 on behalf of the Investigators of PCI Registry of Portuguese Society of Cardiology

1 National School of Public Health, New University of Lisbon, Lisbon, Portugal
2 School of Health Technologies of Lisbon, Lisbon, Portugal
3 Faculty of Medicine, University of Lisbon, Lisbon, Portugal

Address reprint requests to: Paulo Sousa, School of Health Technologies of Lisbon, Lisbon, Portugal. Tel: +351 8980423; Fax: +351 7221392; E-mail: paulo.sousa{at}ensp.unl.pt


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 Conclusions
 Acknowledgements
 References
 
Objective. The aim of this study was to develop a risk adjustment model for major adverse cardiac and cerebrovascular events following percutaneous coronary intervention (PCI), using data from a national registry, and to highlight the use of the risk adjustment when we evaluate the quality of care in interventional cardiology.

Design. The study design was based on a Coorte study. Bivariate and multivariate logistic regression models were used to identify independent risk factors for these major adverse events.

Setting. A total of 19 hospitals from the Portuguese National Registry of Interventional Cardiology.

Participants. Data from 10.641 consecutives procedures collected between June 30, 2003 and June 30, 2006.

Intervention. Build a risk adjustment model for these major adverse events, following percutaneous coronary intervention.

Main Outcome Measure. Factors that were associated with major adverse cardiac and cerebrovascular events following percutaneous coronary intervention.

Results. The rate of in-hospital major adverse cardiac and cerebrovascular events was 1.9%. Factors associated with major adverse cardiac and cerebrovascular events included, among others: age >80 years (adjusted odds ratio = 3.91); female gender (1.72); and cardiogenic shock (6.05). Overall, a good discrimination was achieved with receiver operating characteristics curve = 0.84 and Hosmer-Lemeshow goodness of fit statistic across groups of risk was not significant (P = 0.18) indicating little departure from a perfect fit.

Conclusions. These findings will represent an important contribution to quality and safety improvement and should help driving new research and innovative approaches to different subgroups of patients who have higher chances of having an adverse event or poorer outcomes following this intervention.

Keywords: risk adjustment, PCI, quality improvement, safety, outcomes research


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 Conclusions
 Acknowledgements
 References
 
The growing emphasis and interest in Quality are relatively recent phenomena in health systems. In fact, it is a current theme that has become particularly important in the agendas and the policies of many countries all over the world [1, 2].

The concept of quality in health is nowadays seen from different viewpoints and is defined in different ways. There are several models to assess quality of care, each one with different dimensions and approaches. The mostly used worldwide is the model defined, some years ago, by Donabedian, based on three dimensions: structure, process and outcomes [3]. In the last years, particularly in the United States of America, Canada, Australia, and some European countries, the emphasis is increasingly on outcomes evaluation [4, 5].

The establishment of quality standards based on patient outcome data is a rational means for differentiating the quality of health care in the marketplace. However, variation in patient's baseline clinical risks precludes the direct comparison of outcomes across operators, institutions and health care plans.

Moreover, according to Iezzoni [6], meaningful comparison within the health care system, generally requires risk adjustment – accounting for patient associated factors before comparing outcomes across different patients, treatments, providers, health plans, or populations. The rationale is obvious; on an average higher-risk patients typically generate larger costs, and persons with complex illnesses, multiple coexisting diseases, or other significant risk factors, generally develop more complications and have poorer outcomes, even with excellent care.

There have been several attempts, in recent years, particularly in the United States of America and Europe, to incorporate risk-adjustment methodology in the evaluation of percutaneous coronary intervention (PCI) outcomes [7–10]. These attempts have been limited by small sample sizes, patient samples that do not reflect contemporary PCI, limited geographic representation, inconsistent definitions and coding of the variables, poor quality of the data used to build these models, and consequently, by the lack of applied data standards.

In 2002, the Portuguese Society of Cardiology launched the national registry on interventional cardiology, not a mandatory registry like in Sweden, with the aim to obtain knowledge about the profile of the patient, patterns of disease and treatment strategies; to assess adherence to guidelines of cardiovascular disease in clinical practice; and to stimulate clinical research in this important area [11].

Since 2005, the Portuguese registry uses the Euro Heart Survey on PCI platform, which reinforce more the quality of data [since it uses the Cardiology Audit and Registration Data Standards system (CARDS)] and have opened a window to future comparisons among European countries, which used similar data [12].


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 Conclusions
 Acknowledgements
 References
 
Study population
Retrospective analysis of prospective collected data from 10 641 consecutive patients who underwent PCI, in a total of 19 Portuguese centres who participated in the Portuguese PCI registry, between June 30, 2003 and June 30, 2006.

As a major adverse event, for the purpose of this study, we considered a composite variable comprising, death, acute myocardial infarction, need for a new revascularization by urgent coronary artery bypass graft, and stroke.

We considered as independent variables those that characterize the individuals and treatment aspects, namely demographic (e.g. age, gender); clinical aspects (e.g. diabetes, hypertension, peripheral disease) and technical and functional aspects (e.g. number of vessels diseased, lesion type, ejection fraction, defined as normal >50%; slightly reduced 41–50%; moderately reduced 31–40%; severely reduced <30%).

Data collection
The data were collected from the PCI database of the national centre of data collection in cardiology, the structure of the Portuguese Society of Cardiology, which is responsible for the Portuguese registry. The Portuguese database is based on the Euro Heart Survey of the European Society of Cardiology. Data collection methods and definition of the variables follow the policy of CARDS system of European Society of Cardiology. The form, for each procedure, is filled in a web platform and the quality control of the data and audit is done by European Society of Cardiology. After that, the data are sent to the Portuguese national centre and then distributed to each participant centres. Data are also available in the webpage of the European Society of Cardiology.

Statistical analysis
Firstly, we have started with a descriptive analysis with the aim to get knowledge about the population patterns with respect to each variable.

Subsequently, a bivariate analysis was done crossing the independent (those who characterize the individuals and treatment aspects) with the dependent variable, with the aim of identifying those that have the strongest statistical association with major adverse cardiac and cerebrovascular events.

To build the model, and to identify independent risk factors for major adverse cardiac and cerebrovascular events, a multivariate logistic regression analysis was undertaken, using the stepwise forward technique, which includes all variables that showed, in the bivariate analysis, an odds ratio >1 (which indicates a positive association for the occurrence of major adverse cardiac and cerebrovascular events), and were statistically significant (with P-value <0.05).

To assess the performance and calibration of the model, the area under the receiver operating characteristics (ROC) curve and the Hosmer–Lemeshow goodness of fit statistic were calculated.

All statistical analysis were conducted using the Statistical program® SPSS 14, for a level of significance of 0.05 (P = 0.05) and a confidence interval (CI) of 95%. The association measure used was the odds ratio.


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 Conclusions
 Acknowledgements
 References
 
In the 10 641 patients who underwent PCI, and were included in the register, the primary success rate was 98.1% and the rate of in-hospital major adverse cardiac and cerebrovascular events was 1.9%. In these, 1.4% was death; 0.4% developed an acute myocardial infarction; 0.2% had a stroke; and 0.1% emergency coronary artery bypass graft. These rates are not mutually exclusive, which means that in some patients occurred more than one major adverse event.

The first step of the model development was to examine the bivariate relationship between major adverse cardiac and cerebrovascular events and each pre-procedural risk factor (Table 1). In the multivariate logistic regression model we included all the variables that showed, in the bivariate analysis, an odds ratio >1 and were statistically significant (with P-value <0.05), such as age, gender, acute myocardial infarction, cardiogenic shock, congestive heart failure, diabetes, peripheral disease, renal failure (creatinine >2.0 mg/dl), ejection fraction, number of diseased vessels, intra-aortic balloon pump, type of lesion, non-stenting, priority of procedure and left main treated.


View this table:
[in this window]
[in a new window]

  		Table 1 Univariate and bivariate analysis between dependent and independent variables

 
In Table 2 we can see the variables that resulted from the multivariate logistic regression analysis, using the stepwise forward technique, with their respective coefficient, odds ratio, significance value and CI.


View this table:
[in this window]
[in a new window]

  		Table 2 Independent risk factors for major adverse cardiac and cerebrovascular events

 
Overall, a good discrimination was achieved with ROC curve = 0.84 (Fig. 1), and a Hosmer–Lemeshow goodness of fit statistic across groups of risk was not significant (P = 0.18), indicating little departure from a perfect fit.


Figure 1
View larger version (19K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 1 Area under the receiver operating characteristics (ROC) curve for multivariate prediction model

 

   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
Conclusions
 Acknowledgements
 References
 
In today's world, the rapid spread of information, growing level of knowledge and greater requirements of patients, strong financial constraints, increasing call for accountability, and the need to introduce criteria and quality indicators in the health care provided, have contributed to some change in the dynamics of health institutions [13, 14]

These dynamics have evolved in the direction of giving greater value to the collection and treatment of credible standardized data which makes possible the evaluation and monitor of services regarding the volume of activity and the quality of results achieved [15].

The final goal of risk adjustment is to account for pertinent patient characteristics before making inference about the effectiveness of care.

In the recent past we have seen great progress in PCI that has led to a widening range of situations with well-established clinical and angiographic indications [16–18]. However, PCI still carries significant risk, especially in subgroups in which a more complex clinical condition may lead to higher adverse event rates.

In Portugal, the number of procedures and centres where PCI is performed has exponentially increased in the past decade, from 3017 interventions (302 per million inhabitants) in 12 centres, in 1997 [19] to an estimation of about 11 500 (1150 per million inhabitants) in 24 centres, in 2006.

With this rapid increase in the number of procedures and centres which perform PCI in Portugal, which is similar to the trend of other European countries, and since there is a national continuous registry, it would be important to build a risk-adjustment model. This could greatly help to do reliable comparisons of results among institutions, regions or populations, contributing for the development of multicentre studies, and establishing credible and rigorous benchmarking values in the country.

Information obtained from registries is increasingly being used to assess the process of care and patient outcomes, supported by the paradigm that the future of healthcare is increasingly in the hands of those who are effective users of clinical data [20].

Moreover, this brings up another important question that Portugal and all European countries must face in the short term – which is the need for reliable and standardized data systems to collect, systematize and analyse data which could help to monitor the health care system either, partially or as a whole. For these have concurred different reasons such as: (i) strong movements for accountability and pressing for outcome data, due to the fact that the consumers of today are more informed and demanding than ever, and call for a description of the recommended treatment and its advantages and risks [21]; (ii) legal questions, the example of United Kingdom, since the introduction of freedom of information act, that became mandatory the individual disclosure information about the surgeons results, in different areas [9]. We believe that any attempt to produce unadjusted outcome analysis and comparisons, by named operator, or institutions, may be misleading and will therefore encourage adverse selection practices, and will jeopardize the relationship between patients and health care professionals; (iii) economic implications, which point out not only the importance of effectiveness but also the efficiency of the health care delivered [22].

It is well known that the absence of guidelines for data collection and clear definitions of data items are very important causes of data errors in medical registries, making it difficult to develop multicentre studies and also hampering the comparisons of results between different regions and countries [12, 15].

By developing and using data standards, based on the CARDS system, the Euro Heart Survey and all registries that use their platform, avert these questions and have greatly expanded the pool of patients and geographical area in which data outcomes can be analysed and compared [12]. Furthermore this emphasizes the importance of health outcome analysis, based on credible, standardized and audited data, as a central point in a quality assurance programme and also as a key pathway in the direction of reducing the burden of cardiovascular disease in Europe.[23, 24].

Bearing in mind that cardiovascular disease, including coronary artery disease, affects millions of patients worldwide, with serious consequences in economic and social weight, evaluation of the results of PCI using credible information and systematic databases is crucial for the achievement of the greater objective which is to guarantee and improve the quality of health care provided [17, 21, 24]. Therefore the use of risk adjustment in outcomes evaluation for PCI is essential to assess different quality dimensions such as effectiveness, efficiency and safety [23, 25, 26].

There are multiple areas where the risk-adjustment methodology can be used to contribute to strengthen the quality assessment and by this means reach the quality improvement, including the following: to detect potential adverse selection (based on the severity of the disease); to identify and define, in a more rigorous way, quality indicators; for quality assessment purposes, providing a comparison of outcomes among providers (hospitals or physicians) after adjusting for risk and assessing changes in risk-adjusted outcomes for a single provider over time; and to help explain and understand the impact of practice variations among providers, and their implication in different clinical and economic outcomes [6, 27].

The risk-adjustment model developed in this study, by allowing the identification and evaluation of patient risk factors that are associated with poor outcomes or adverse events, constitutes a potential contribution to quality improvement in interventional cardiology. However, it needs to be tested in future prospective studies.

Generally, the models for risk adjustment are built based on clinical data. However, it will be important, in the future, to include also economic data (costs per patient and per procedure, or cross-data from diagnostic-related groups, for instance) with the aim to close the gap between effectiveness and efficacy and, at the same time, to obtain a global and integrated perspective of the quality of health care delivered [26, 28].

The current study has contributed to demonstrate the potential value of using a continuous national database, with timely data analysis, for developing a risk-adjustment model for major adverse cardiac and cerebrovascular events.

Perhaps the most significant limitation of the current study is the lack of a systematic approach for auditing the data. In our opinion, one of the bigger challenges for the future is the need to submit all databases for a valid and objective audit process, in order to guarantee the quality of the data.

With the proliferation of efforts to report publicly the outcomes of healthcare providers and institutions, most of them using risk-adjustment models, there was a growing need to define standards for the methods that are being employed. According to this, the interdisciplinary writing group for quality of care and outcomes research of the American Heart Association identified, recently, seven preferred attributes of statistical models used [20].

It is our belief that this study includes all of the seven attributes, namely, clear and explicit definition of an appropriate patient sample; clinical coherence of the model variables; sufficiently high-quality and timely data; designation of an appropriate reference time before which covariates are derived and after which outcomes are measured; use of an adequate outcome and standardized period of outcome assessment; application of an analytical approach that takes into account the multilevel organization of data; and disclosure of the methods used to compare outcomes, including disclosure of performance of the risk-adjustment methodology in derivation and validation samples.

Nevertheless, the variables that were generated from our model are consistent with a number of other studies recently published [8, 9, 18, 29].


   Conclusions
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conclusions
Acknowledgements
 References
 
The risk-adjustment model developed in this study, by allowing the identification and evaluation of patient risk factors that are associated with poor outcomes or adverse events represent a potential contribution to improve quality of care through a more rigorous assessment of outcomes.

The variables that were generated from our model are consistent with a number of other studies recently published. These findings will likely represent a potential contribution to improve quality and should help driving new research and innovative approaches to different subgroups of patients who have higher chances of having an adverse event or poorer outcomes following PCI.

Despite the existence of models already described in the literature, to our knowledge there is none based on data from Euro Heart Survey database. This could be seen as an effort to improve quality in a very relevant public health burden disease, such as coronary artery disease.

In our opinion, this issue will be intensified in the years ahead and should be studied more deeply because, on an average, the occurrence of adverse events or poor results is linked with an increase in financial and social costs.

Developing risk-adjustment models for major adverse cardiac and cerebrovascular events following PCI is an important part of the quality and safety improvement process for cardiac revascularization procedures, and for establishing credible and rigorous benchmarking values.


   Acknowledgements
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Acknowledgements
References
 
We thank Prof. Lino Gonçalves, head of the National Centre for Data Collection in Cardiology and the Portuguese Society of Cardiology for authorizing the use of the data for this study, as well as the centres that contributed to the registry, namely, Hospital de São João; Hospital Geral de Santo António; Centro Hospitalar de Vila Nova de Gaia; Hospital de São Teotónio; Hospitais da Universidade de Coimbra; Centro Hospitalar de Coimbra; Hospital Curry Cabral; Hospital Pulido Valente; Hospital de Santa Cruz; Hospital de Santa Marta; Hospital Garcia de Orta; Hospital de São Bernardo; Hospital Distrital de Faro; Hospital do Divino Espírito Santo (Ponta Delgada); Centro Hospitalar do Funchal; Hospital da Cruz Vermelha; Hospital da CUF; Hospital do SAMS; Serviço Médico de Imagem Computorizada (SCIMC).


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Acknowledgements
 References
 

  1. Petersen S, Peto V, Rayner M, et al. European Cardiovascular Disease Statistic (2005) London: British Heart Foundation.

  2. Larsson B, Larsson G, Chantereau M, et al. International comparison of patient's view on quality of care. Int J Health Care Assur (2005) 18:66–73.

  3. Donabedian A. La Calidad De La Atención Médica (1984) México DF: La Prensa Mexicana.

  4. Shaughnessy PW, Hittle DF. Overview of Risk Adjustment and Outcomes Measures for Home Health Agency OBQI Reports: Highlights of Current Approaches and Outline of Planned Enhancements (2002) Denver: Centre for Health Services Research, UCHSC.

  5. Arca M, Fusco D, Barone AP, et al. Risk adjustment and outcome research. J Cardiol Med (2006) 7:682–90.

  6. Iezzoni L. Reasons for risk adjustment. In: Risk Adjustment for Measuring Healthcare Outcomes—Iezzoni L, ed. (2003) 3rd edn.

  7. Maynard C, Richard JG, Malenka DJ, et al. Adjusting for patient differences in predicting hospital mortality for percutaneous coronary interventions in the Clinical Outcomes Assessment Program. Am Heart J (2003) 145:658–64.[CrossRef][Web of Science][Medline]

  8. Shaw RE, Anderson HV, Brindis RG, et al. Development of a risk adjustment mortality model using the American College of Cardiology – National Cardiovascular Data Registry (ACC-NCDR) Experience: 1998–2000. J Am Coll Cardiol (2002) 39:1104–12.[Abstract/Free Full Text]

  9. Grayson AD, More RK, Jackson M, et al. Northwest quality improvement programme in cardiac interventions. Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England. Heart (2006) 92:658–63.[Abstract/Free Full Text]

  10. Singh M, Rihal CS, Lennon RJ, et al. A critical appraisal of current models of risk stratification for percutaneous coronary intervention. Am Heart J (2005) 149:753–60.[CrossRef][Web of Science][Medline]

  11. Pereira H. on behalf of the investigators of PCI registry of the Portuguese Society of Cardiology. Registo Português de Cardiologia de Intervenção. Rev Port Cardiol (2004) 23:7–14.[Medline]

  12. Flynn MR, Barret C, Cosio FG, et al. The cardiology audit and registration data standards (CARDS), European data standards for clinical cardiology practice. Euro Heart J (2005) 26:308–13.[Abstract/Free Full Text]

  13. World Health Organization. Measuring Hospital Performance to Improve the Quality of Care in Europe: A Need for Clarifying the Concepts and Defining the Main Dimensions (2003) Barcelona: WHO Regional Office for Europe.

  14. Fergunson TB Jr, Peterson ED, Coombs LP, et al. Use of continuous quality improvement to increase use of process measures in patients undergoing coronary artery bypass graft surgery: a randomised controlled trial. JAMA (2003) 290:49–56.[Abstract/Free Full Text]

  15. Weintraub WS, McKay CR, Riner R, et al. The American College of Cardiology National Database: progress and challenges. J Am Coll Cardiol (1997) 29:459–65.[CrossRef][Web of Science][Medline]

  16. Topol EJ. Quality of care in interventional cardiology. In: Textbook of Interventional Cardiology—Topol EJ, ed. (2003) 4th edn. Philadelphia: Saunders. 1021–32.

  17. Anderson HV, Shaw RE, Brindis RG. A contemporary overview of percutaneous coronary intervention. J Am Coll Cardiol (2002) 39:1096–103.[Abstract/Free Full Text]

  18. Chuntao W, Hannan EL, Walford G, et al. A risk score to predict in-hospital mortality for percutaneous coronary interventions. J Am Coll Cardiol (2006) 47:654–60.[Abstract/Free Full Text]

  19. Maier W, Camici P, Windecker D, et al, on behalf of the Working Group Coronary Circulation of the European Society of Cardiology. The European Registry of Cardiac Catheter Intervention 1997. Eur Heart J (2002) 23:1903–07.[Free Full Text]

  20. Krumholz HM, Brindis RG, Brush JE, et al. Standards for statistical models used for public reporting of health outcomes. Circulation (2006) 113:456–62.[Abstract/Free Full Text]

  21. Marshall MN, Shekelle PG, Huw TO, et al. Public reporting on quality in the United States and the United Kingdom. Health Aff (2003) 22:134–48.[Abstract/Free Full Text]

  22. Jacobson KM, Long KH, McMurtry EK, et al. The economic burden of complications during percutaneous coronary intervention. Qual Saf Health Care (2007) 16:154–9.[Abstract/Free Full Text]

  23. Scholte OP, Reimer WJW, Gitt AK, et al. Cardiovascular Diseases in Europe. Euro Heart Survey 2006 (2006) Sophia Antipolis: European Society of Cardiology.

  24. Brindis RG, Fitzgerald S, Anderson HV, et al. The American College of Cardiology – National Cardiovascular Data Registry: building a national clinical data repository. J Am Coll Cardiol (2001) 37:2240–45.[Free Full Text]

  25. Leal J, Luego-Fernández R, Gray A, et al. Economic burden of cardiovascular disease in the enlarged European Union. Eur Heart J (2006) 27:1610–19.[Abstract/Free Full Text]

  26. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century (2001) Washington, D.C.: National Academy Press.

  27. Ferraris AI, Edwards FH, Shahian DM, et al. Risk stratification and comorbdity. In: Cardiac Surgery in the Adult—Cohn LH, ed. (2008) New York: McGraw-Hill. 199–246.

  28. Kugelmass AD, Cohen DJ, Brow PB, et al. Hospital resources consumed in treating complications associated with percutaneous coronary interventions. Am J Cardiol (2006) 97:322–27.[CrossRef][Web of Science][Medline]

  29. Block PC, Peterson EC, Krone R, et al. Identification of variables needed to risk adjust outcomes of coronary intervention: evidence-based guidelines for efficient data collection. J Am Coll Cardiol (1998) 32:275–82.[Abstract/Free Full Text]

Accepted for publication June 18, 2008.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
20/5/324    most recent
mzn029v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Sousa, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Sousa, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?