Have you seen the advancements discussed in the new Sage Handbook of Mixed Methods Research Design (Poth, 2023)? This webinar will explore several emerging design advancements for the requisite integration of qualitative and quantitative data and findings within mixed methods research — including evolutions in typologies, use of technology, descriptions of culture, community-based approaches, and more. It will focus on how emerging practices can help researchers in their design and offer some practical guidance for designing quality mixed methods research.
The CATA and RATA tasks have extensive use in sensory research. While primarily intended for consumer evaluation, they also serve as alternatives to profiling tasks and are suitable for trained judges. In such instances, it is vital to analyze the panel's performance and conduct a multivariate analysis of sessions. This webinar demonstrates how to address these issues and enables you to take a deeper dive into the analysis of CATA and RATA data with XLSTAT.
Productive scholars establish a routine, a rhythm, that boosts productivity. Most preserve morning hours to tackle their most intellectually demanding tasks and push more routine tasks like meetings and teaching to the afternoon. Most work 40-50 hours per week, though some work more, with about half that time focused on research activities.
Productive scholars set goals, prioritize tasks, and orchestrate to-do list plans for reaching them. They are efficient. They fill large time blocks and small time pockets with scholarly work. They take breaks to keep fresh but don’t procrastinate. Productive scholars say “no” to invitations that interrupt priorities and to other time-killing tasks and distractions. Yet, they do more than research; they serve the field. Most seek and attain a healthy work-life balance that includes time for family, mental rejuvenation, and physical activity. Female scholars, perhaps because of societal norms, are particularly challenged in attaining a work-life balance.
Join this seminar for a simple introduction to the challenge of doing a grounded theory with online sources. Whether you are a first-time user of grounded theory or an experienced one, this seminar will inspire you to explore online data sources as a foundation for your grounded theory study. The joy of grounded theory is simple and well rewards any effort. Because grounded theory method has robust coding procedures, it enables detailed engagement with the data and invariably gives new insights. A significant number of social settings are now online, and it is important that we have effective ways of studying those settings.
Author: Christina Silver, Ph.D., Associate Professor (Teaching) & Director of the CAQDAS Networking Project
At the recent Lumivero conference, a few presentations discussed the role of artificial intelligence (AI) in qualitative, quantitative, and mixed-methods research. There were lots of fascinating and engaging presentations across all the conference days, and one that I found particularly interesting in the context of my current work at the CAQDAS Networking Project (CNP) that focuses on the relationship between qualitative software and analytic methods, was by Silvana di Gregorio Ph.D.
Di Gregorio's presentation was titled Disruption and the Rise of AI: Exploring the Role of Technology in Qualitative Research, and she shared the findings from her recent research into how qualitative researchers perceive AI and its role in qualitative analysis. Di Gregorio is a highly experienced qualitative researcher and methodologist who has been deeply involved in the use of technology for qualitative analysis for many years. She’s incredibly well-respected in the field and is someone I’ve known personally for many years, so when I saw she was going to be speaking about the rise of AI in qualitative research there was no way I was going to miss the presentation!
In this article, we’ll discuss di Gregorio’s presentation and dig into AI’s role in data analysis, content analysis, and qualitative research.
This sketchnote, created by blog author Christina Silver, summarizes the key points of di Gregorio’s presentation at the 2023 Lumivero Virtual Conference.
Artificial intelligence (AI) is a term that's been around a long time and is understood in a range of sometimes quite different and contested ways. With respect to qualitative analysis, I prefer a broad definition that encompasses the range of relevant technologies, of which there are many. This definition from Cornell University captures the breadth of technologies often contained within the term:
“Artificial intelligence or AI is the use of machine learning technology, software, automation, and algorithms (the automated computational application of rules) to perform tasks, to make rules and/or predictions based on existing datasets and instructions.” (Cornell Law School, Artificial Intelligence (AI))
Since generative-AI tools like ChatGPT and Google's Bard became widely available, there's been a lot of what I've elsewhere called "hoo-ha" about the impacts on qualitative data analysis and the use of digital tools, with disparate views ranging from horror to band-wagoning (see my series of posts on AI in qualitative research).
And that's why di Gregorio 's presentation that presented her research on perceptions amongst qualitative researchers really piqued my interest.
Di Gregorio framed her presentation around how the rise of generative-AI is causing disruption in the field of qualitative research, but she was quick to emphasise; that this form of AI is neither the first nor only form that has impacted how we go about doing qualitative analysis with the support of computers. I've also written about; that this form of AI is neither the first nor only form that has impacted how we go about doing qualitative data analysis with the support of computers.
To illustrate this, di Gregorio recounted a presentation she gave in 2020 discussing four types of AI that impact qualitative research: Natural Language Processing; Speech; Vision; and Admin Assistants.
Fig. 1 From di Gregorio, S. (2020) Can AI help you? Leveraging your human skills in a digital world, paper presented at QRCA Annual Conference: Keep Qual Human, 29-31 January, Austin, TX.
What's happening now though is causing much more disruption because of the ubiquitousness of generative-AI and how it’s being reported on and discussed – particularly in the media, but also in qualitative research circles.
"It's a disruption that no one can ignore because it's become sort of the rage. It's in the media all the time. There's a lot of fears about it. There's also a lot of excitement about it.” - Silvana di Gregorio
The AI tools that have been available in qualitative software for a while (and in some cases decades) didn't receive the level of attention that the generative-AI tools are receiving right now. And that's at least in part because they didn't infiltrate into our everyday lives in the same way.
Di Gregorio rightly reminded us that disruptions are common in the development of research practice, and that disruption of any kind actually forces us to rethink and take stock of what we do, and how we do it. This is of value in moving both methodology and technology forward; disruptions have their challenges, yes, but also often bring welcome advances when they're seen as opportunities.
Di Gregorio used Covid-19 as an example of a recent major disruption on how qualitative research practice happens, particularly in how data could be collected during lockdown. The effect this has had on methodological discussions about the role of digital tools in qualitative analysis and particularly online research methods more generally, I believe is a good thing. Online data collection and other research practices were well-established in some quarters pre-Covid (i.e., the first edition of Janet Salmons’ excellent textbook on Doing Qualitative Research Online was published in 2016, with the second edition coming out in 2021.)
However, many qualitative researchers perhaps fell into more customary, off-line, practices pre-Covid out of habit. There is no doubt that Covid forced everyone to consider online methods in previously unprecedented ways, and that is great for the development of methods. We also saw an upsurge in the availability of tools to support the full range of research needs, for example developments in automated transcription tools, data collection and communications tools like Zoom and Microsoft Teams, and collaboration tools like Google Drive and Google Docs.
In the first series of podcasts I've recorded on computer-assisted qualitative analysis (CAQDAS chat with Christina), several of my guests spoke about generative-AI and its methodological implications. Founders of the CAQDAS Networking Project, Professors Nigel Fielding and Ray Lee, recounted in my first podcast episode the disruption that the initial advent of qualitative software brought. They recall disruption being clearly evidenced at the first conference on the topic that they organised in 1989 with researchers' views on the opportunities and dangers being starkly divergent. The parallels with what's happening now with some qualitative software programs harnessing generative-AI in qualitative methods isn't lost on them, di Gregorio, myself, or many qualitative methodologists I've spoken to.
Di Gregorio recounted findings in her presentation from her research about what qualitative researchers think about generative-AI, and this formed the bulk of her presentation last month. During the summer of 2023, she gathered thoughts from researchers working in different sectors with varying amounts of qualitative research experience. In her first survey, she was keen to hear from researchers regardless of whether they’d used generative-AI or not.
To me, this was a great way to uncover perceptions because the full range of opinions, from the advocates of these new technologies to the sceptics, are relevant whether they're grounded in the experience of using these tools or not. This is important because perception of technology has an impact on the field of computer-assisted qualitative data analysis.
For example, when it comes to how we teach qualitative methods and tools at college and university, the perceptions of the faculty doing the teaching are of fundamental influence. Where courses are taught by faculty who do not use qualitative software like NVivo themselves, a complete absence of discussion or tutelage is not uncommon. The perceptions of those who are teaching current and future generations of qualitative researchers can shape the landscape. In the rapidly developing field of generative-AI and the potentially transformative effect it will have on analytic practice, we do our students even more of a disservice to ignore, sideline, or fear the developments. Therefore, knowing what researchers all the way along the "excited to sceptical" spectrum think is incredibly important.
"…if you don't get involved, you know, your students will get involved with it. It's something that we need to understand, and it will have an impact on our research." - Silvana di Gregorio
Di Gregorio followed up the survey with follow-up interviews with selected researchers to gain more in-depth understanding of perceptions – sharing with the conference audience the breadth of perceptions from these two sources. The topline findings from the survey responses and interviews can be seen in di Gregorio’s presentation.
Perhaps the most frequently mentioned potential benefit of generative-AI (and other AI-powered tools) in qualitative analysis I've heard is that it will save time. It seems finding short-cuts is high on the agenda for many. But coming out of di Gregorio 's research is also the potential for these tools to actually add something, nicely articulated in this quote from one of the respondents in her research:
“The way that the AI gave the answers was really interesting and thought provoking because it gave some perspectives which I didn't necessarily think of … So the AI went a step deeper and probably included some dimensions or perspectives which I wouldn't have done by myself.” - Doctoral student, some qual experience
This is something I was pleased to see; for me the role of any technology in the practice of qualitative analysis is about how to harness what computers can do to contribute to the quality and relevance of our analysis, not just being able to do it quicker. It is such principles that underlie many of the AI tools that have been incorporated into qualitative software before the rise of generative-AI, and it's important that they also drive how these newer computational capabilities are embraced by developers and researchers alike.
It's clear to me that as a community of practice we need to know more, and the respondents in di Gregorio 's research thought so too. Despite the length of time software like NVivo has been available, there's still a lack of understanding from some quarters about what the software can do and how to use them. As expressed by one of di Gregorio 's respondents, this is also the case with respect to generative-AI:
“I think the level of understanding of AI among qualitative researchers is currently quite low, so it is really important to explore further what can be done and have conversations about how it should or could be used.” - Respondent in di Gregorio's research
Di Gregorio's research will contribute to raise awareness about these technologies amongst the qualitative research community of practice, so I'm very much looking forward to seeing it formally published. There’s a massive appetite for research on this topic and it’s great to see that di Gregorio is amongst those at its forefront.
As Product Research Director, Di Gregorio has a direct line into the development of tools in NVivo, and her research findings around AI tools are actively being taken into consideration.
“… in doing this research, you know, Lumivero, we've been listening to you and we are committed to develop our software addressing your needs and concerns. And one thing is that I'd like to create a customer advisory group on generative AI. And the purpose would be get feedback from you from a diverse set of customers about how you'd like it in our products, you know, and also, feedback on prototypes we create, but also, you know, the ethical, privacy concerns and how they need to be addressed by us. So … if you want to continue the conversation … I look forward to hearing from your comments.” - Silvana di Gregorio
For qualitative researchers this is excellent to hear. If you’re an NVivo user, or a user of any of Lumivero’s products, I’d encourage you to get involved. Developers of software – in my 25+ years’ experience spanning the CAQDAS field – are demand-driven. They want to know what researchers need, and they try to develop tools to meet those needs. It’s our responsibility as researchers and users of their products, to let them know what those needs are.
At the CAQDAS Networking Project we're also hoping to contribute to the awareness-raising effort. One way we're doing so is by partnering with the Social Research Association (SRA) to organise a 2-part symposium on the topic later this year.
We're really excited that di Gregorio has agreed to be part of that event because it’s clear she will make a valuable contribution. She'll be participating in the "in conversation with…" session on the methodological implications of AI in qualitative analysis - and not just generative-AI, but AI in general. So, if you're interested in these topics, be sure to check out the program and register as soon as possible. It's a free event, and we hope to gather thoughts from qualitative researchers as well as to share some of what's occurring in this space.
Part 1: Nov. 24, 2023
Part 2: Dec. 1, 2023
We're at the early stages of integrating generative AI into NVivo. To ensure our decisions align with our community's needs and maintain our commitment to excellence, we're launching an AI Advisory Group. We invite researchers, tech experts, and enthusiasts to join us on this journey.
If you're passionate about qualitative research and excited about AI's potential, we'd love to hear from you. Apply today to be a part of this transformative journey. Together, we're shaping the future of NVivo with AI!
Gain deeper insight into your complex decision by visually mapping out all possible routes with probabilistic decision trees. By modeling the life of the project, you can account for the possible consequences of future decisions and gain realistic and useful decision modeling analytics.
Stop the guesswork and start identifying factors that make the largest impact on your output with probabilistic sensitivity analysis. By combining sensitivity analysis with Monte Carlo simulation, you can analyze the impact of different variables in your model by ranking inputs in order of importance and easily communicate results with tornado charts and spider graphs – leading to more informed decisions.
Improve your forecasting and decision-making by applying powerful statistical analysis to your data, directly in your Excel spreadsheet. With predictive modeling and forecasting techniques, you can gain the insight needed to make critical decisions with confidence for cases such as demand forecasting, pricing, portfolio allocation, load planning, sales forecasting, strategic planning, profit projections, and more.
Sophisticated, predictive neural networks imitate brain functions to identify patterns in historical or new, incomplete data sets – letting you intelligently predict the future and make decisions to meet your goals. Lumivero's powerful, robust neural networks software enables you to optimize results, even in the face of uncertainty.
Free Resources for Accelerating Your Literature Review
To help you get started with your literature review, we’ve collected some of our top resources showing how to write literature reviews with NVivo and Citavi.
The Monte Carlo method is a computerized mathematical technique that allows people to quantitatively account for risk in forecasting and decision-making. At its core, the Monte Carlo method is a way to use repeating random samples of parameters to explore the behavior of a complex system. A Monte Carlo simulation is used to handle an extensive range of problems in a variety of different fields to understand the impact of risk and uncertainty.