Whether you’re a student or an experienced practitioner, it’s not unusual to have a crisis of confidence during a qualitative research project.
This credibility checklist will help keep you on track.
1. Have I engaged with the literature?
A systematic literature review demonstrates your familiarity with the topic and positions you as a credible expert.
So, consider whether you've read and reported on the significant works in your field. Venturing out across different fields and disciplines will help you position yourself in the wider scholarly landscape.
You may need to revisit the literature at various points in your research process. It's not just a one-time process that you do at the beginning.
As your own research develops, you may need to go back to the literature for a deeper read and reflect on how it relates to your own research.
You might also want to seek out other literature that addresses issues that were not on your radar when you started the study.
>> Read how NVivo helps with literature reviews.
2. Do I have a sound research question?
Let's say you're at a party and someone asks about your research.
Can you explain it in clear everyday language that doesn’t send them scurrying to the bar?
If the answer is yes, remember to point out why your research is important and mention who would be interested in the outcomes.
Your research question should uncover new ground and move the conversation forward in some way but it doesn’t need to be “ground breaking, unprecedented or paradigm shifting” (Golding & Sharmini & Lazarovitch, 2014, p. 569).
And it goes without saying, your final thesis or report should address and answer your research question.
3. Are my methods transparent?
Reviewers need to easily trace the steps you took to arrive at your results. They’ll want to know how your data was collected, recorded, coded and analyzed.
They’ll also want to understand the choices you made along the way.
For example, you might explain your reason for running focus groups:
“Focus groups work well for encouraging participants to explore topics that have shared social meaning but are seldom discussed” (Bailey, 2012, p. 3).
Analytical memos are the key to transparency.
If you’re wondering about what sort of memos you should be keeping, invoke this clever mnemonic developed by Birks, Chapman and Francis (2008):
- M – Mapping research activities (documentation of the decision-making processes of research design and implementation as an audit trail).
- E – Extracting meaning from the data (analysis and interpretation, concepts, assertions, theories).
- M – Maintaining momentum (researcher perspectives and reflexivity throughout the evolutionary journey of the study).
- O – Opening communication (for research team member exchanges).
4. Have I laid my reflexive cards on the table?
Credibility depends on honesty and reflexivity, so you’ll need to openly acknowledge and address your biases and sociocultural position.
Be clear about:
- Your awareness of pre-existing knowledge and philosophical frameworks.
- Your own position in relation to research participants.
- The limitations of your method.
- Your openness to contrary evidence.
5. Have I collected enough of the right kind of data?
“The quality of your study starts with the data, as does its credibility. The depth and scope of the data make a difference. A study based upon rich, substantial, and relevant data stands out” (Charmaz, 2014, Location No. 1275).
Whatever your approach to doing qualitative research, the purpose for collecting qualitative data is to generate in-depth and rich data.
So if you are doing interviews, make sure that you have probed sufficiently so you truly understand what your respondent is saying.
Active listening and judicious probing are important skills to develop as a qualitative researcher. If you feel there are gaps in your understanding of an interview, it may be possible to go back to your respondents, or if that is not possible, to do a few more additional interviews to fill those gaps.
A common question is whether you have collected enough data.
The answer depends on the approach to your analysis. If you need to generalise your findings to a particular population, which may be the case in a mixed methods study, then you'll need to have a representative sample of the population that you're studying.
If you're doing a public consultation, you need to make sure that you have reached out to all the relevant stakeholders.
If you are doing a grounded theory study, then you'll be doing theoretical sampling – which means you stop collecting data when your sample is saturated i.e. when the data you are collecting does not reveal anything new. That might be after eight interviews or after fifty.
Taking an iterative approach to data collection can help to alleviate short comings.
For example, you could conduct a pilot set of interviews and do some preliminary coding - this might help you determine whether you need to tweak your questions or broaden the spread of participants.
If the time for interviewing has passed and you’re worried about the depth of your data - look at ways you can triangulate it:
- Find complementary studies or alternative theories.
- Explore it with a different methodological lens.
- Include the perspective of other researchers.
Just ensure you understand (and report on) the limits of your data.
Only make claims where there is clear empirical evidence to support them.
6. Is my coding up to scratch?
In the words of coding guru, Johnny Saldana, “the excellence of the research rests in large part on the excellence of the coding.” (Saldana, 2016, p. 2)
In practical terms this involves:
- Creating codes that adequately reflect what is happening in the data.
- Aiming for coding consistency (especially important for teams).
- Moving beyond purely descriptive codes to analytical concepts.
- Looking for the relationships between analytical concepts.
- Checking that these analytical concepts offer fresh insights about your topic.
- Reading The Coding Manual for Qualitative Researchers by Johnny Saldana.
To score even higher on the credibility scale, go back to the study participants and get their input on your interpretations (sometimes called member checking).
Wherever possible, seek the advice of trusted peers, supervisors and mentors.
7. Am I continually striving for ethical excellence?
Do you have ethics approval from relevant committees covering confidentiality and informed consent?
Credibility means treating your participants with respect and approaching your research with scholarly integrity.
Be careful not to plagiarize the work of others, make unsupported claims or ignore ‘problematic’ passages of text that don’t quite fit your theories.
8. Have I got a strong narrative to tell?
Ultimately, credibility is about creating a coherent narrative that stands up to challenges and brings fresh ideas to the table.
"When I silently think or verbally whisper “Wow!” at the conclusion of someone’s work, I know that I have been given not just new knowledge but new awareness, and am now the better for it" (Saldana, 2016, p. 289).
Your results should represent the experience of your participants in a believable way and convincingly address the research question.
Along with ‘credibility’ you might want to consider a few other measures of quality, including: Dependability, accountability, confirmability, believability, trustworthiness, expandability, resonance, creativity, relevance, clarity, confirmability, significance, artfulness, authenticity, criticality, vividness, thoroughness, congruence, sensitivity, believable, completeness, publishable, generalizability, transferability, replicability and rigor.
Enough to make your head spin (but fun when sung to the tune of ‘I am the very model of Modern Major General’).
What’s your benchmark for research quality?
I’d love to hear your thoughts below.
NVivo is software that helps researchers to organize and analyze their qualitative data. Read more about how NVivo supports literature reviews.
References
Bailey, D. C. (2012). Women and Wasta: The Use of Focus Groups for Understanding Social Capital and Middle Eastern Women . The Qualitative Report, 17(33), 1-18. Retrieved from Nnsuworks.nova.edu
Birks, M., Chapman, Y., & Francis, K. (2008) Memoing in qualitative research: Probing data and processes. Journal of Research in Nursing 13(1), 68–75.
Charmaz, K. (2014). Constructing Grounded Theory (2nd ed.) [Kindle version]. Retrieved from Amazon.com
Clinton Golding, Sharon Sharmini & Ayelet Lazarovitch (2014) What examiners do: what thesis students should know, Assessment & Evaluation in Higher Education, 39:5, 563-576, DOI: 10.1080/02602938.2013.859230
Saldana, J. (2016). The Coding Manual for Qualitative Researchers (3rd ed.) [Kindle version]. Retrieved from Amazon.com
ABOUT THE AUTHOR
Kath McNiff
Kath McNiff is on a mission to help researchers deliver robust, evidence-based results. If they’re drowning in a sea of data (or floods of tears) she wants to throw them an NVivo-shaped life raft. As an Online Community Manager at QSR, she knows that peers make the best teachers. So, through The NVivo Blog, Twitter and LinkedIn, she shares practical advice and connects researchers so they can help each other. When she’s not busy writing blog posts, swapping stories on social media or training the latest tribe of NVivo users, she can be found wrestling four feisty offspring for control of the remote.