OUP user menu

Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups

Allison Tong, Peter Sainsbury, Jonathan Craig
DOI: http://dx.doi.org/10.1093/intqhc/mzm042 349-357 First published online: 14 September 2007

Abstract

Background Qualitative research explores complex phenomena encountered by clinicians, health care providers, policy makers and consumers. Although partial checklists are available, no consolidated reporting framework exists for any type of qualitative design.

Objective To develop a checklist for explicit and comprehensive reporting of qualitative studies (indepth interviews and focus groups).

Methods We performed a comprehensive search in Cochrane and Campbell Protocols, Medline, CINAHL, systematic reviews of qualitative studies, author or reviewer guidelines of major medical journals and reference lists of relevant publications for existing checklists used to assess qualitative studies. Seventy-six items from 22 checklists were compiled into a comprehensive list. All items were grouped into three domains: (i) research team and reflexivity, (ii) study design and (iii) data analysis and reporting. Duplicate items and those that were ambiguous, too broadly defined and impractical to assess were removed.

Results Items most frequently included in the checklists related to sampling method, setting for data collection, method of data collection, respondent validation of findings, method of recording data, description of the derivation of themes and inclusion of supporting quotations. We grouped all items into three domains: (i) research team and reflexivity, (ii) study design and (iii) data analysis and reporting.

Conclusions The criteria included in COREQ, a 32-item checklist, can help researchers to report important aspects of the research team, study methods, context of the study, findings, analysis and interpretations.

Keywords
  • focus groups
  • interviews
  • qualitative research
  • research design

Qualitative research explores complex phenomena encountered by clinicians, health care providers, policy makers and consumers in health care. Poorly designed studies and inadequate reporting can lead to inappropriate application of qualitative research in decision-making, health care, health policy and future research.

Formal reporting guidelines have been developed for randomized controlled trials (CONSORT) [1], diagnostic test studies (STARD), meta-analysis of RCTs (QUOROM) [2], observational studies (STROBE) [3] and meta-analyses of observational studies (MOOSE) [4]. These aim to improve the quality of reporting these study types and allow readers to better understand the design, conduct, analysis and findings of published studies. This process allows users of published research to be more fuller informed when they critically appraise studies relevant to each checklist and decide upon applicability of research findings to their local settings. Empiric studies have shown that the use of the CONSORT statement is associated with improvements in the quality of reports of randomized controlled trials [5]. Systematic reviews of qualitative research almost always show that key aspects of study design are not reported, and so there is a clear need for a CONSORT-equivalent for qualitative research [6].

The Uniform Requirements for Manuscripts Submitted to Biomedical Journals published by the International Committee of Medical Journal Editors (ICMJE) do not provide reporting guidelines for qualitative studies. Of all the mainstream biomedical journals (Fig. 1), only the British Medical Journal (BMJ) has criteria for reviewing qualitative research. However, the guidelines for authors specifically record that the checklist is not routinely used. In addition, the checklist is not comprehensive and does not provide specific guidance to assess some of the criteria. Although checklists for critical appraisal are available for qualitative research, there is no widely endorsed reporting framework for any type of qualitative research [7].

Figure 1

Development of the COREQ Checklist. *References [26, 27], References [6, 2832], Author and reviewer guidelines provided by BMJ, JAMA, Lancet, Annals of Internal Medicine, NEJM.

We have developed a formal reporting checklist for in-depth interviews and focus groups, the most common methods for data collection in qualitative health research. These two methods are particularly useful for eliciting patient and consumer priorities and needs to improve the quality of health care [8]. The checklist aims to promote complete and transparent reporting among researchers and indirectly improve the rigor, comprehensiveness and credibility of interview and focus-group studies.

Basic definitions

Qualitative studies use non-quantitative methods to contribute new knowledge and to provide new perspectives in health care. Although qualitative research encompasses a broad range of study methods, most qualitative research publications in health care describe the use of interviews and focus groups [8].

Interviews

In-depth and semi-structured interviews explore the experiences of participants and the meanings they attribute to them. Researchers encourage participants to talk about issues pertinent to the research question by asking open-ended questions, usually in one-to-one interviews. The interviewer might re-word, re-order or clarify the questions to further investigate topics introduced by the respondent. In qualitative health research, in-depth interviews are often used to study the experiences and meanings of disease, and to explore personal and sensitive themes. They can also help to identify potentially modifiable factors for improving health care [9].

Focus groups

Focus groups are semi-structured discussions with groups of 4–12 people that aim to explore a specific set of issues [10]. Moderators often commence the focus group by asking broad questions about the topic of interest, before asking the focal questions. Although participants individually answer the facilitator's questions, they are encouraged to talk and interact with each other [11]. This technique is built on the notion that the group interaction encourages respondents to explore and clarify individual and shared perspectives [12]. Focus groups are used to explore views on health issues, programs, interventions and research.

Methods

Development of a checklist

Search strategy

We performed a comprehensive search for published checklists used to assess or review qualitative studies, and guidelines for reporting qualitative studies in: Medline (1966—Week 1 April 2006), CINAHL (1982—Week 3 April 2006), Cochrane and Campbell protocols, systematic reviews of qualitative studies, author or reviewer guidelines of major medical journals and reference lists of relevant publications. We identified the terms used to index the relevant articles already in our possession and performed a broad search using those search terms. The electronic databases were searched using terms and text words for research (standards), health services research (standards) and qualitative studies (evaluation). Duplicate checklists and detailed instructions for conducting and analysing qualitative studies were excluded.

Data extraction

From each of the included publications, we extracted all criteria for assessing or reporting qualitative studies. Seventy-six items from 22 checklists were compiled into a comprehensive list. We recorded the frequency of each item across all the publications. Items most frequently included in the checklists related to sampling method, setting for data collection, method of data collection, respondent validation of findings, method of recording data, description of the derivation of themes and inclusion of supporting quotations. We grouped all items into three domains: (i) research team and reflexivity, (ii) study design and (iii) data analysis and reporting. (see Tables 24)

Within each domain we simplified all relevant items by removing duplicates and those that were ambiguous, too broadly defined, not specific to qualitative research, or impractical to assess. Where necessary, the remaining items were rephrased for clarity. Based upon consensus among the authors, two new items that were considered relevant for reporting qualitative research were added. The two new items were identifying the authors who conducted the interview or focus group and reporting the presence of non-participants during the interview or focus group. The COREQ checklist for explicit and comprehensive reporting of qualitative studies consists of 32 criteria, with a descriptor to supplement each item (Table 1).

View this table:
Table 1

Consolidated criteria for reporting qualitative studies (COREQ): 32-item checklist

NoItemGuide questions/description
Domain 1: Research team and reflexivity
Personal Characteristics
1.Interviewer/facilitatorWhich author/s conducted the interview or focus group?
2.CredentialsWhat were the researcher's credentials? E.g. PhD, MD
3.OccupationWhat was their occupation at the time of the study?
4.GenderWas the researcher male or female?
5.Experience and trainingWhat experience or training did the researcher have?
Relationship with participants
6.Relationship establishedWas a relationship established prior to study commencement?
7.Participant knowledge of the interviewerWhat did the participants know about the researcher? e.g. personal goals, reasons for doing the research
8.Interviewer characteristicsWhat characteristics were reported about the interviewer/facilitator? e.g. Bias, assumptions, reasons and interests in the research topic
Domain 2: study design
Theoretical framework
9.Methodological orientation and TheoryWhat methodological orientation was stated to underpin the study? e.g. grounded theory, discourse analysis, ethnography, phenomenology, content analysis
Participant selection
10.SamplingHow were participants selected? e.g. purposive, convenience, consecutive, snowball
11.Method of approachHow were participants approached? e.g. face-to-face, telephone, mail, email
12.Sample sizeHow many participants were in the study?
13.Non-participationHow many people refused to participate or dropped out? Reasons?
Setting
14.Setting of data collectionWhere was the data collected? e.g. home, clinic, workplace
15.Presence of non-participantsWas anyone else present besides the participants and researchers?
16.Description of sampleWhat are the important characteristics of the sample? e.g. demographic data, date
Data collection
17.Interview guideWere questions, prompts, guides provided by the authors? Was it pilot tested?
18.Repeat interviewsWere repeat interviews carried out? If yes, how many?
19.Audio/visual recordingDid the research use audio or visual recording to collect the data?
20.Field notesWere field notes made during and/or after the interview or focus group?
21.DurationWhat was the duration of the interviews or focus group?
22.Data saturationWas data saturation discussed?
23.Transcripts returnedWere transcripts returned to participants for comment and/or correction?
Domain 3: analysis and findingsz
Data analysis
24.Number of data codersHow many data coders coded the data?
25.Description of the coding treeDid authors provide a description of the coding tree?
26.Derivation of themesWere themes identified in advance or derived from the data?
27.SoftwareWhat software, if applicable, was used to manage the data?
28.Participant checkingDid participants provide feedback on the findings?
Reporting
29.Quotations presentedWere participant quotations presented to illustrate the themes / findings? Was each quotation identified? e.g. participant number
30.Data and findings consistentWas there consistency between the data presented and the findings?
31.Clarity of major themesWere major themes clearly presented in the findings?
32.Clarity of minor themesIs there a description of diverse cases or discussion of minor themes?
View this table:
Table 2

Items included in 22 published checklists: Research team and reflexivity domain

ItemReferences
[26]a[27]a[6]b[28]b[32]b[13][15][14][17][33][34][35][16][19][36][7][37][23][38][39][22]BMJ
Research team and reflexivity
Nature of relationship between the researcher and participants
Examination of role, bias, influence
Description of role
Identity of the interviewer
Continued and prolonged engagement
Response to events
Prior assumptions and experience
Professional status
Journal, record of personal experience
Effects of research on researcher
Qualifications
Training of the interviewer/facilitator
Expertise demonstrated
Perception of research at inception
Age
Gender
Social class
Reasons for conducting study
Sufficient contact
Too close to participants
Empathy
Distance between researcher and participants
Background
Familiarity with setting
  • aOther publications, bSystematic review of qualitative studies; BMJ, British Medical Journal—editor's checklist for appraising qualitative research); •, item included in the checklist.

View this table:
Table 3

Items included in 22 published checklists: Study design

ItemReferences
[26]a[27]a[6]b[28] b[32]b[13][15][14][17][33][34][35][16][19][36][7][37][23][38][39][22]BMJ
Study design
Methodological orientation, ontological or epistemological basis
Sampling—convenience, purposive
Setting
Characteristics and description of sample
Reasons for participant selection
Non-participation
Inclusion and exclusion, criteria
Identity of the person responsible for recruitment
Sample size
Method of approach
Description of explanation of research to participants
Level and type of participation
Method of data collection, e.g. focus group, in-depth interview
Audio and visual recording
Transcripts
Setting and location
Saturation of data
Use of a topic guide, tools, questions
Field notes
Changes and modifications
Duration of interview, focus group
Sensitive to participant language and views
Number of interviews, focus groups
Time span
Time and resources available to the study
  • aOther publications, bSystematic review of qualitative studies; BMJ, British Medical Journal—editor's checklist for appraising qualitative research; •, item included in the checklist.

View this table:
Table 4

Items included in 22 published checklists: Analysis and reporting

ItemReferences
[26]a[27]a[6]b[28]b[32]b[13][15][14][17][33][34][35][16][19][36][7][37][23][38][39][22]BMJ
Respondent validation
Limitations and generalizability
Triangulation
Original data, quotation
Derivation of themes explicit
Contradictory, diverse, negative cases
Number of data analysts
In-depth description of analysis
Sufficient supporting data presented
Data, interpretation and conclusions linked and integrated
Retain context of data
Explicit findings, presented clearly
Outside checks
Software used
Discussion both for and against the researchers' arguments
Development of theories, explanations
Numerical data
Coding tree or coding system
Inter-observer reliability
Sufficient insight into meaning/perceptions of participants
Reasons for selection of data to support findings
New insight
Results interpreted in credible, innovative way
Eliminate other theories
Range of views
Distinguish between researcher and participant voices
Proportion of data taken into account
  • aOther publications, bSystematic review of qualitative studies; BMJ, British Medical Journal—editor's checklist for appraising qualitative research, •, item included in the checklist.

COREQ: content and rationale (see Tables 1)

Domain 1: research team and reflexivity

(i) Personal characteristics: Qualitative researchers closely engage with the research process and participants and are therefore unable to completely avoid personal bias. Instead researchers should recognize and clarify for readers their identity, credentials, occupation, gender, experience and training. Subsequently this improves the credibility of the findings by giving readers the ability to assess how these factors might have influenced the researchers' observations and interpretations [1315].

(ii) Relationship with participants: The relationship and extent of interaction between the researcher and their participants should be described as it can have an effect on the participants' responses and also on the researchers' understanding of the phenomena [16]. For example, a clinician–researcher may have a deep understanding of patients' issues but their involvement in patient care may inhibit frank discussion with patient–participants when patients believe that their responses will affect their treatment. For transparency, the investigator should identify and state their assumptions and personal interests in the research topic.

Domain 2: study design

(i) Theoretical framework: Researchers should clarify the theoretical frameworks underpinning their study so readers can understand how the researchers explored their research questions and aims. Theoretical frameworks in qualitative research include: grounded theory, to build theories from the data; ethnography, to understand the culture of groups with shared characteristics; phenomenology, to describe the meaning and significance of experiences; discourse analysis, to analyse linguistic expression; and content analysis, to systematically organize data into a structured format [10].

(ii) Participant selection: Researchers should report how participants were selected. Usually purposive sampling is used which involves selecting participants who share particular characteristics and have the potential to provide rich, relevant and diverse data pertinent to the research question [13, 17]. Convenience sampling is less optimal because it may fail to capture important perspectives from difficult-to-reach people [16]. Rigorous attempts to recruit participants and reasons for non-participation should be stated to reduce the likelihood of making unsupported statements [18]. Researchers should report the sample size of their study to enable readers to assess the diversity of perspectives included.

(iii) Setting: Researchers should describe the context in which the data were collected because it illuminates why participants responded in a particular way. For instance, participants might be more reserved and feel disempowered talking in a hospital setting. The presence of non-participants during interviews or focus groups should be reported as this can also affect the opinions expressed by participants. For example, parent interviewees might be reluctant to talk on sensitive topics if their children are present. Participant characteristics, such as basic demographic data, should be reported so readers can consider the relevance of the findings and interpretations to their own situation. This also allows readers to assess whether perspectives from different groups were explored and compared, such as patients and health care providers [13, 19].

(iv) Data collection: The questions and prompts used in data collection should be provided to enhance the readers' understanding of the researcher's focus and to give readers the ability to assess whether participants were encouraged to openly convey their viewpoints. Researchers should also report whether repeat interviews were conducted as this can influence the rapport developed between the researcher and participants and affect the richness of data obtained. The method of recording the participants' words should be reported. Generally, audio recording and transcription more accurately reflect the participants' views than contemporaneous researcher notes, more so if participants checked their own transcript for accuracy [1921]. Reasons for not audio recording should be provided. In addition, field notes maintain contextual details and non-verbal expressions for data analysis and interpretation [19, 22]. Duration of the interview or focus group should be reported as this affects the amount of data obtained. Researchers should also clarify whether participants were recruited until no new relevant knowledge was being obtained from new participants (data saturation) [23, 24].

Domain 3: analysis and findings

(i) Data analysis: Specifying the use of multiple coders or other methods of researcher triangulation can indicate a broader and more complex understanding of the phenomenon. The credibility of the findings can be assessed if the process of coding (selecting significant sections from participant statements), and the derivation and identification of themes are made explicit. Descriptions of coding and memoing demonstrate how the researchers perceived, examined and developed their understanding of the data [17, 19]. Researchers sometimes use software packages to assist with storage, searching and coding of qualitative data. In addition, obtaining feedback from participants on the research findings adds validity to the researcher's interpretations by ensuring that the participants' own meanings and perspectives are represented and not curtailed by the researchers' own agenda and knowledge [23].

(ii) Reporting: If supporting quotations are provided, researchers should include quotations from different participants to add transparency and trustworthiness to their findings and interpretations of the data [17]. Readers should be able to assess the consistency between the data presented and the study findings, including the both major and minor themes. Summary findings, interpretations and theories generated should be clearly presented in qualitative research publications.

Discussion

The COREQ checklist was developed to promote explicit and comprehensive reporting of qualitative studies (interviews and focus groups). The checklist consists of items specific to reporting qualitative studies and precludes generic criteria that are applicable to all types of research reports. COREQ is a comprehensive checklist that covers necessary components of study design, which should be reported. The criteria included in the checklist can help researchers to report important aspects of the research team, study methods, context of the study, findings, analysis and interpretations.

At present, we acknowledge there is no empiric basis that shows that the introduction of COREQ will improve the quality of reporting of qualitative research. However this is no different than when CONSORT, QUOROM and other reporting checklists were introduced. Subsequent research has shown that these checklists have improved the quality of reporting of study types relevant to each checklist [5, 25], and we believe that the effect of COREQ is likely to be similar. Despite differences in the objectives and methods of quantitative and qualitative methods, the underlying aim of transparency in research methods and, at the least, the theoretical possibility of the reader being able to duplicate the study methods should be the aims of both methodological approaches. There is a perception among research funding agencies, clinicians and policy makers, that qualitative research is ‘second class’ research. Initiatives like COREQ are designed to encourage improvement in the quality of reporting of qualitative studies, which will indirectly lead to improved conduct, and greater recognition of qualitative research as inherently equal scientific endeavor compared with quantitative research that is used to assess the quality and safety of health care. We invite readers to comment on COREQ to improve the checklist.

References

View Abstract