We often discuss different qualitative data analysis methodologies on Between the Data such as different approaches to sifting and understanding the data that research generates. What we don’t often talk about are the practical, nuts-and-bolts aspects of coding in qualitative research – especially when coding with a team.
Lindsay Giesen is a Principal Research Associate at Westat in Rockville, Maryland. She has more than 15 years of policy research and program evaluation experience, and her work focuses on child nutrition and food security. On a research study conducted on behalf of the United States Department of Agriculture (USDA), she developed an approach to manage a qualitative data coding team that she believes improved the quality of the analysis and final report. Giesen joined Dr. Stacy Penna on Between the Data: Episode 19 to discuss her method for structuring, training, and managing qualitative data coders.
In this case study, we'll cover the highlights from their conversation including Giesen's coding approach, analysis process, and use of qualitative data analysis software (QDA) NVivo.
Research Project Background and Reasoning for a Guide to Coding Qualitative Research
Giesen and her team were tasked with conducting a study for the USDA’s Food and Nutrition Services. The goal was to understand how schools, school districts, and states gather data for federally funded child nutrition programs, including the National School Lunch Program and School Breakfast Program. This project involved site visits in four states during the spring of 2018. The team conducted 154 interviews with school cafeteria workers, school district officials, and state education officials.
The team had a six-week turnaround time to complete the qualitative coding of all interview data. Giesen and her team understood that it was essential to take a structured approach to training and monitoring their coders in order to meet their deadline. She learned three key lessons during this project:
- Develop a management structure with layers of support for your team
- Take a gradual approach to training and build skills sequentially rather than trying to teach the team everything at once
- Create very detailed reference materials for guiding your team as they work
A Multi-Layered Management Structure for Qualitative Data Coding Teams: Constant Communication, Better Collaboration
First, Giesen distributed the work for each coder. Her team had conducted interviews in four different states, so coders were each assigned one state to work on. Giesen stresses that these coders weren’t coming to the transcripts cold.
“The coders were the more junior staff who had been the support staff on the site visits that we did,” Giesen explains. “So they knew the data, they knew the interviews and the respondents.”
Next, each coder was assigned to a senior reviewer. The reviewers had served as the leads for each site visit.
The senior reviewers reported to the lead analyst – that is, Giesen. An outside methodologist was also brought in to provide objective feedback on the team’s processes.
“We each had sort of our own support person, which was just so helpful,” Giesen explained. “It gives you someone to check your work and also someone to bounce ideas around with and questions, and it just created a really nice structure for us.”
This structure was collaborative, with constant communication among the different layers of the hierarchy. This flow of questions and feedback allowed issues to be flagged early in the process so that the coding scheme and other processes could be refined.
Coding Methods: Training in Stages for Higher-Quality Coding
Next, Giesen and the methodologist organized a training for the coders and reviewers.
“I'd worked on coding teams before . . . [where] I would get my instructions and the list of codes and then be left to my own devices," said Giesen.
Giesen didn’t want this to happen to her coders, and she also didn’t want to overwhelm them with information. She took a multi-step approach to training.
Step 1: In-Person Training for Coding Qualitative Data
First, her team met for a one-day, in-person training session that covered essential coding skills and learning to work with NVivo qualitative analysis software. The first part of the day involved manual coding with paper and highlighters – line-by-line coding.
“There's a lot of noise in qualitative data,” Giesen said, “And you need to teach [coders] just conceptually how to sift through the noise to find what you're looking for.”
She had the coders work with practice transcripts, circling parts of a transcript and noting which codes from the code database she had created might apply. This progressed to working with Microsoft Word documents to highlight specific sections of text to code and deciding when to apply more than one code. Using a predefined set of codes and then assigning them to data is called deductive coding. The alternative is inductive coding, which is deriving codes from qualitative data.
“I find [NVivo] invaluable, and I think the team picked it up really quickly . . . we wanted to make sure that everybody had a shared understanding of how to use it.”
Step 2: Practice and Virtual Review of Coded Transcripts
After the one-day training session, Giesen moved the team on to coding the simplest, shortest transcripts – the school-level interviews. This would allow the coders to work with a manageable number of the 200 or so codes she had created for the project.
Giesen held a shorter remote training session in which she shared her screen and walked teams through which codes she expected would apply to the data. These training meetings also included a short practice session and time for questions. Coders and reviewers would leave these meetings tasked with choosing an assigned transcript to talk through with their reviewer.
Step 3: Q&A Session and Timeline Review
A few days later, the team re-convened virtually for a check-in meeting. Everyone brought questions from their assigned transcripts.
“It brought up places where the coders weren't sure which code to apply to certain pieces of text. And we would take that feedback and refine the coding scheme during that call if we needed to,” Giesen explained.
It also offered her team a reality check about how long each transcript would take to code the data so that deadlines and timelines could be adjusted accordingly.
Train, Practice, Code
This train-practice-code process repeated itself throughout the six-week coding period, as the team moved from the school-level interviews to those conducted at the district and state levels.
Creating Detailed Reference Materials Showing Steps for Coding Qualitative Data
One of the challenges of coding in qualitative research is consistency among the coding team. Even experienced coders can struggle to assign codes accurately without adequate reference materials, and these inaccuracies can lead to problems when it comes time to analyze and dig deeper into the data through thematic analysis – identifying patterns and themes.
Giesen’s 2020 article for the Sage International Journal of Qualitative Methods, “Structuring a Team-Based Approach to Coding Qualitative Data”, describes the reference materials she created to help ensure consistent thematic analysis coding. These materials included a Microsoft Excel codebook that coders could filter depending on the type of transcript they were coding (e.g., only displaying codes for school-level interviews).
There was also a blank copy of the interview questions at each level with a list of the most applicable codes assigned to each question and a sample transcript with codes applied. The codebook was edited throughout the six-week coding period based on team feedback.
Giesen confessed that she was initially anxious about building all these different steps into the team coding process given how tight her deadline was. However, it wound up resulting in a stronger final product.
“Giving people endless opportunities to ask questions and continuously revising the coding scheme to better fit the data through our team meetings . . . made the whole process go really smoothly,” Giesen said. “Best of all, it made our analysis and reporting process so much better because the quality of the coding was strong.”
Learn More About Giesen’s Approaches to Qualitative Coding
Interested in more detail about managing teams when coding qualitative research? You can read the paper Lindsay Giesen co-authored in 2020 online at the Sage International Journal of Qualitative Methods.
You can also learn more about this research by listening to the full podcast episode here.
Code Qualitative Data with a NVivo Free Trial
Organize your data and improve your process of coding qualitative data by requesting a free 14-day trial of NVivo today!