US Government strengthens international aid model with qualitative data analysis

USAID uses NVivo to improve learning agenda

Background

The United States Agency of International Development (USAID) is a U.S. government agency that betters the lives of millions across the globe through areas such as improving agricultural productivity, combating maternal and child mortality, providing immediate disaster relief, fostering private sector development, elevating the role of women and girls, and providing humanitarian support.[i] Since its inception during the Kennedy administration, USAID has made an impact in countries across the world.
Funded by the USAID Bureau for Policy, Planning and Learning (PPL), the USAID LEARN contract —implemented by Dexis Consulting Group—is designed to support strategic learning and knowledge management at USAID to improve the performance and effectiveness of its programs around the world.[ii]
USAID LEARN works with multiple teams and departments within USAID through an approach called, “collaborating, learning and adapting (CLA).” Part of this approach includes using Learning Agendas, which are a set of agency-specific questions directly related to its work. When these questions are answered, the Learning Agenda guides staff  through organizational learning in order to create an environment of growth and success.

The Challenge

USAID LEARN conducted 60 interviews and held two focus groups to properly analyze the Learning Agenda’s current landscape within federal agencies. They reviewed brochures and programs already informed by Learning Agendas across agencies, including The Department of Housing and Urban Development (HUD), The Department of Agriculture (USDA), The Department of Labor (DOL), The United Nations Research Institute for Social Development (UNRISD).[iii]
The analysis accumulated textual, video, audio, PDF or image and numerical data. However, traditional quantitative data analysis tools are only able to analyze the numerical data collected. And the traditional, pen/paper/highlighter/sticky note method of analysis would take too long. Individually reviewing each image, interview, and document would waste hours of time and risk the loss of important themes or ideas as time passes. USAID LEARN and Dexis Consulting Group needed a different solution.

Introducing NVivo

USAID LEARN conducted an assessment of the potential software that can analyze all of their qualitative (non-numerical) data in a quick and meaningful way. Because NVivo is often recognized as the industry standard for qualitative data analytics in education, many of the analysts had previous, positive experience working with the solution. NVivo’s functionality, accuracy and speed are exactly what the team was seeking, so it was the unanimous choice.

Results

The USAID LEARN team was impressed to learn from their NVivo research that the Learning Agenda, serving as an integral way to understand both how and why to grow, was in use far more widely than anticipated.

“We discovered that there are over 20 operating units within USAID using Learning Agendas. A variety of people, from HUD to the UN, in all different technical areas are also using them to learn and grow,” said Matt Baker, USAID LEARN’s Monitoring Evaluation, Research and Learning Specialist.

NVivo also helped the USAID LEARN team uncover the themes that make up their Learning Agenda questions.

“Because NVivo can interpret live language, it was able to read over 100 questions and show us how we could distill them down to the best five,” said Baker,

NVivo revealed the top reasons why Learning Agendas are useful by identifying three primary motivations for those who use them:

  1. Expectations of accountability, especially in response to leadership demands
  2. Leadership transitions and structural, strategic, or policy changes
  3. Responses to identified program-related needs

Many other discoveries were made in the landscape analysis that will lead the USAID LEARN contract as the team begins the one-to-two-year process of developing strong Learning Agendas for USAID. Because of this, USAID has the opportunity to improve the effectiveness of its important work, both in Washington and across the globe.

[i] https://www.usaid.gov/what-we-do
[ii] https://usaidlearninglab.org/learn-contract
[iii] https://usaidlearninglab.org/sites/default/files/resource/files/landscape_analysis_report_04_10_17.pdf
 

ABOUT THE AUTHOR

QSR International with USAID and Dexis Consulting Group

The United States Agency of International Development (USAID) is a U.S. government agency that betters the lives of millions across the globe through areas such as improving agricultural productivity, combating maternal and child mortality, providing immediate disaster relief, fostering private sector development, elevating the role of women and girls, and providing humanitarian support. Every day, QSR International helps 1.5 million researchers, marketers and others to utilize Qualitative Data Analysis (QDA) to uncover deeper insights contained within the “human data” collected via social media, consumer and community feedback and other means. We give people the power to make better decisions by uncovering more insights to advance their area of exploration.

Research in the field: Shooting Stars and the success of their community program

Australian not-for-profit are utilizing NVivo to evaluate the success of their community program.

Shooting Stars is an initiative of Netball WA and Glass Jar Australia, which uses netball and other incentives, as vehicles to encourage greater engagement and attendance at school of young Aboriginal and Torres Strait Islander girls living in Western Australia’s remote communities and regional towns. Through fostering collaborative relationships with the local community, Shooting Stars works with schools and service providers to implement a regular program of personal development activity and fun for young Aboriginal and Torres Strait Islander girls.

Participation in the program enables girls to have a taste of success to improve their self-image, develop hope and aspirations about their future, and provide motivation to attend school to achieve those aspirations.

Following a successful pilot in 2014 in Halls Creek, the program had its full-scale launch into five locations throughout the State in 2015. Funded by the Department of Prime Minister and Cabinet through to the end of 2017 across six locations, Shooting Stars plans to expand to 14 sites in 2018.

Participants undertake a program that involves netball and physical education activities, such as swimming when the weather is too hot for a game, over two sessions a week. There are also two health and wellbeing sessions covering topics including nutrition and healthy relationships aligned to the Australian Curriculum.

A unique research challenge

Shooting Stars was faced with a unique challenge when wanting to both gather and analyze data to provide feedback on the program. Firstly, staff were dealing with a group of children ranging in age from primary school through to teenagers, and found that the standard method of paper-based surveys was not engaging and did not help in keeping the program as culturally centered as they desired.

To address this issue, a new method of data collection was established through consultation with local cultural leaders. The traditional indigenous yarning circle was presented as an alternative to paper-based surveys, to gain feedback from the girls in the program, in a way that encouraged open discussion.

“The yarning circle is really the idea of sitting around and ‘having a yarn’,” explained Rose Whitau, Research Manager at Shooting Stars – Midwest Gascoyne.

“We’ve found this to be a very participant driven piece of research, through the yarning circles, which are driven by a localised steering committee, made up of our staff, school staff, local cultural leaders, and the remote school attendance strategy provider,” she said.

The next challenge was that yarning circles produced unstructured data, in the form of audio recordings of the sessions, and they required a tool to analyze this data.

“We wanted to turn messy qualitative data into something that was easy to digest, and NVivo has allowed our team to do this,” Whitau explained.

“The purpose of using NVivo is to be able to evaluate the program ongoing, as we want our participants to have an impact on how the program is structured and delivered, which is very empowering for them,” she said.

NVivo is now being used throughout Shooting Stars locations to analyze the results of the yarning circles, of which there have been 22 so far.

“NVivo is the perfect tool for our needs, especially its ability to analyze audio files. Coming from an academic research background, I was quite familiar with the methodology, but I have also been able to easily teach the software to our other staff who come from a range of different professional backgrounds,” Whitau said.

Shooting Stars continues to affect real change, and work to empower indigenous Australian girls to stay in school and engage in their education, as well as maintain a positive attitude toward their health and wellbeing. You can learn more about the great work they’re doing on their website.

ABOUT THE AUTHOR

QSR International with Shooting Stars

Shooting Stars is an initiative of Netball WA and Glass Jar Australia, which uses netball as a vehicle to encourage greater engagement and attendance at school of young Aboriginal girls living in WA’s remote communities and regional towns. Working collaboratively with the local community, schools and service providers to implement a regular program of personal development activity and fun for young Aboriginal girls. Every day, QSR International helps 1.5 million researchers, marketers and others to utilize Qualitative Data Analysis (QDA) to uncover deeper insights contained within the “human data” collected via social media, consumer and community feedback and other means. We give people the power to make better decisions by uncovering more insights to advance their area of exploration.

NVivo strengthens qualitative research into improving education for Indonesia’s 50 million-plus students

Dr Agustinus Bandur uses NVivo to improve the quality of education in Indonesia.

One of the driving influences behind improving the quality of education across Indonesia’s schools and higher education institutions is Dr Agustinus Bandur. A senior lecturer and strategic research & partnership leader at BINUS University in Jakarta, Dr Bandur has spent more than 15 years researching how to improve leadership and management in education across Indonesia. Dr Bandur also consults to several not-for-profit organizations, including the Florenza Children Resources Center, which he founded in 2008 to help children improve their learning.

Improving education in Indonesia

For Dr Bandur, the issue of improving Indonesia’s education system is compounded by its sheer size and diversity.

With more than 50 million students and 2.6 million teachers in more than 250,000 schools, it is the fourth largest education system in the world, behind only China, India and the United States.

While Indonesia has made major progress in improving its primary and secondary education, serious issues remain around the drop-out rates of students. According to the 2016 National Socio-Economic Survey, around one million children between 7 – 15 years old are not attending primary or junior secondary school. Meanwhile, another 3.6 million adolescents aged 16 to 18 are out of school. There are also concerns around the quality of learning at Indonesian schools. According to UNICEF, only 81% of primary school teachers hold the minimum qualifications required by government.

The role of research

Dr Bandur’s research has been central to discussions around school-based management (SBM) policy reform in Indonesia, which saw education decentralize and shift responsibility to schools in 2005 in an effort to improve management within the education sector. In response to higher education globalization in Indonesian universities, Dr Bandur has also focused his research on higher education internationalization.

While Dr Bandur has plenty of passion for improving Indonesia’s education management and leadership, what he doesn’t have is a lot of time. With so many projects, educational institutions and not-for-profit organizations are relying on his research insights, Dr Bandur realised early on that he needed a faster, more efficient way of collecting, managing and analyzing his research data.

In 2002, Dr Bandur began using NVivo, software designed to support qualitative data analysis. Since then, Dr Bandur relies on it almost every day for his research needs, from conducting literature reviews to content analysis of various sources, thematic and cross-case analysis, as well as mixed-methods research. He also uses NVivo to ensure his articles and books are based on authentic, evidence-based data.

How NVivo Helps

NVivo is the number one software chosen by academics around the world for presenting the most robust, defensible findings from qualitative research. According to Dr Bandur, NVivo saves him significant time and
effort, particularly with queries for content analysis, cluster analysis and visualizing themes, as well as with transcribing data.

NVivo saves time with transcribing

“For any qualitative research, transcribing is tiring,” said Dr Bandur. “Prior to learning about NVivo, I would have to listen to interviews in Windows Media Player and transcribe them into Microsoft Word. It meant having two programs opened at the same time, which took time to use and control. In NVivo, I can hear, see and manage my recorded data much more effectively and efficiently. It is also fascinating with NVivo to capture and transcribe secondary data from YouTube videos prior to conducting primary data collection in the field.”

NVivo identifies themes from hundreds of papers

“Another advantage of NVivo is that I do not have to read all the references to make sense of the data. Instead, with the query system in NVivo, I can search the main theme, word or topic that I am analyzing, saving me from having to skim through hundreds of papers and potentially miss important information. NVivo is also a perfect tool for me as it integrates my papers managed in Mendeley.”

NVivo adds credibility to research through the triangulation technique

“Finally, NVivo adds credibility to my research. It allows me to quickly create project mappings, analyze the attributes and perform cross-case analysis. The triangulation technique – combining data from different sources such as interviews, focus groups and photos – is something I use frequently to improve the strength of my findings. I also find team research helps. With NVivo, my research team can work on the same project, allowing us to conduct inter-rater analysis to measure reliability.”

Best practice research results in greater student achievements

According to Dr Bandur’s research, there is evidence that the implementation of the SBM policy has resulted in improving teaching learning environments and student achievements. He continues to study its effects and train others – along with advocating NVivo as a best practice way to collect, manage and analyze data. “The NGOs I work with, including Wahana Visi Indonesia, SMERU Research Institute and Perkumpulan Prakarsa are all now using NVivo. My Doctoral and Master’s students have also applied NVivo in their Doctoral dissertations and Master’s theses. I strongly recommend NVivo to students, scholars and research because it is a powerful tool for the purposes of conducting content, thematic, and cross-case analyses in qualitative research.”

ABOUT THE AUTHOR

QSR International with Dr Agustinus Bandur

Dr Agustinus Bandur PhD, is a senior lecturer and strategic research & partnership leader at BINUS University in Jakarta. Dr Bandur has spent more than 15 years researching how to improve leadership and management in education across Indonesia. Every day, QSR International helps 1.5 million researchers, marketers and others to utilize Qualitative Data Analysis (QDA) to uncover deeper insights contained within the “human data” collected via social media, consumer and community feedback and other means. We give people the power to make better decisions by uncovering more insights to advance their area of exploration.

Informing decision making in healthcare backed by a new standard of academic rigor

NVivo is helping the team at the Strategy Unit at NHS Midlands and Lancashire Commissioning Unit interrogate their data on a deeper level and make informed recommendations.

Background

The Strategy Unit hosted by NHS Midlands and Lancashire Commissioning Support Unit exists to improve outcomes through providing expert support and advice to public and third sector organizations. They work especially closely with Sustainability and Transformation Partnerships, emerging Integrated Care Systems and Clinical Commissioning Groups.  

Mahmoda Begum is a Senior Consultant within the unit. She works within a multidisciplinary team to support delivery of projects with a focus on high-quality research and evaluation. She contributes to generating, analyzing and interpreting evidence through mixed methods research approaches to evaluate change in health and social care.

While completing her Master’s degree in public health, Mahmoda had her first experience with NVivo. “Getting that experience allowed me to explore different research methods, and ways of critically looking at things. That’s now the approach I tend to take with qualitative research – that rigor and systematic academic way of doing things” she said.

Introducing academic rigor

The team at The Strategy Unit work with large amounts of data as part of their qualitative fieldwork, which includes transcriptions of interviews, focus groups and workshops, and researcher reflections. Access to data management and analyses tools in The Strategy Unit, such as NVivo, have allowed a new standard to be introduced to increase the quality of the outputs.

“Tools like NVivo have allowed us to support decision makers in the healthcare systems with the objective evidence of ‘what works’ and ‘what can be made to work’. We can be confident that it’s through a thorough analysis of all the data that has been collected in a timely and relevant way” she said.

The advantage of working to this standard is in being able to partner with academic organizations and publishing in peer-review journals. “Our approach wasn’t necessarily academically focused; it was more centered around meeting the needs of our health and social care leaders and practitioners that we were reporting to. Now we increasingly recognize the dual purpose of learning from and contributing to the evidence-base” said Mahmoda.

“It’s much more of a holistic approach than we had previously. There’s a few ideas that we’re actually working on now that could translate into academic papers, and NVivo has facilitated that” she said.

Showing NVivo’s benefits

Prior to Mahmoda joining the unit, the team had some initial exposure to NVivo, but she has been a driving force behind its adoption in the organization. For the past 18 months, the team have been working in NVivo, and utilizing its features beyond their standard data management activities. “We often try out new things that we think we might need for other projects, for example we’ve tried features like the visualizations, so we’ve tried to use it flexibly” said Mahmoda.

Showcasing the benefits of NVivo to her colleagues who have taken a more traditional approach to research has been a task Mahmoda has taken on with success.

“I’ve been able to show people in the organization who have come from a more traditional research background the benefits of NVivo by applying it to a dataset where they already had their own theories and notions around what the data was going to say” said Mahmoda.

“I was able to interrogate the data and say yes, these theories might match up to what you already think, but also there’s all these other ideas that you’ve missed out on. It’s been helpful to be able to give them the evidence to confirm their theories, but also demonstrate that some theories are simply their own preconceived ideas” she said.

NVivo has helped the team improve their processes for report writing methodology design and efficient analysis.  Looking to the future, the team want to be able to use it further in evidence synthesis, and to inform data triangulation. “The more people in The Strategy Unit who see the rigor and efficiency that NVivo provides, the more they value the benefit of using it” said Mahmoda.

NVivo in project work

The Strategy Unit were commissioned by NHS England to undertake a research project into appointment booking and other working arrangements covering the 10 High Impact Actions. The team interviewed practice managers of general practices, and looked at evidence that was already available with the intention to bring these pieces of information together in one place and explore the variations in access in general practice.

“We came up with a series of recommendations for the commissioners, national policy makers, and general practices on how general practices can improve appointment systems and other working arrangements for the access needs of their specific populations.  It was a substantial piece of work and hugely informative” said Mahmoda.

Overall, NVivo has been a welcome addition to the resources at The Strategy Unit, and has assisted in broadening the scope of what’s possible in their research and evaluation work. “Everything we do is evidence based, and NVivo really helps us practice what we preach” said Mahmoda.

“It’s something we want to use more widely, and we’ve got a lot of support internally to do that, we’re always looking for ways to improve and develop our skills” she said.

About the Strategy Unit at NHS Midlands and Lancashire Commissioning Unit and Mahmoda Begum

The Strategy Unit is a consultancy leading research, analysis and change from within the NHS.  They aim to support NHS, Public sector and third sector clients to understand the challenges they face, and make the best possible decisions, using evidence informed analysis and advice.

They work on a broad national level in the United Kingdom, and are made up of a team that have expertise and skills spanning from complex data analysis, decision support, research and development and strategic service transformation to executive-level strategic advice and evaluation.  They also work with a number of specialist associates and key partner organizations to deliver their services, including: ICF International, The Transformation Unit and the Health Services Management Center at the University of Birmingham. 

Mahmoda Begum is a consultant within The Strategy Unit, and has been a part of the team for just over 18 months. She comes from a research background and joined the team in a research and evaluation capacity and has helped drive the implementation of NVivo within the unit.  Mahmoda graduated university with a Bachelor’s Degree in Biomedical Science, but quickly decided a life of lab work wasn’t for her and moved to working in an applied social research setting, with a predominantly qualitative research focus. She has worked in the public sector for over 9 years, of which the last 4 years have been within the NHS.

ABOUT THE AUTHOR

QSR International with Mahmoda Begum, the Strategy Unit at NHS Midlands and Lancashire Commissioning Unit

The Strategy Unit is a consultancy leading research, analysis and change from within the NHS. They aim to support NHS, Public sector and third sector clients to understand the challenges they face, and make the best possible decisions, using evidence informed analysis and advice. Mahmoda Begum is a consultant within The Strategy unit, and has been a part of the team for just over 18 months. She comes from a research background and joined the team in and research and evaluation capacity and has helped drive the implementation of NVivo within the Unit.

Combining NVivo and Endnote for a quality assessed metasynthesis of literature

Ole Steen Kristensen’s research aimed to identify the difficulties of young people transitioning from foster care to independent adulthood from their perspective. In his study of how their present life is characterized by accumulated memories of the past and fragmented ideas about the future, he faced the challenge of synthesizing thousands of articles presenting diverse perspectives and methodologies. NVivo software, combined with Endnote, enabled him to effectively manage the sorting, sifting and analysis stages resulting in a quality assessed metasynthesis of the literature.

Background

Ole Steen Kristensen’s research aimed to identify the difficulties of young people transitioning from foster care to independent adulthood from their perspective. In his study of how their present life is characterized by accumulated memories of the past and fragmented ideas about the future, he faced the challenge of synthesizing thousands of articles presenting diverse perspectives and methodologies. NVivo software, combined with Endnote, enabled him to effectively manage the sorting, sifting and analysis stages resulting in a quality assessed metasynthesis of the literature.

The Challenge - to generalize from diverse research results

How Is it possible to generalize results from a large collection of qualitative research studies on a particular topic? The diversity of qualitative research with its range of concepts, methods and designs, makes such a task very complex.
Also, in reviewing the literature on this study, Ole Steen faced a number of further challenges:

Ole Steen decided to conduct a meta-synthesis of the qualitative studies in this area. A meta-synthesis is a systematic method to compare and generalize findings from several qualitative studies. It aims to portray an accurate interpretation of a phenomenon and to compare and contrast the constructs of individual studies to reach consensus on a new construction of that phenomenon. It involves: identifying findings, grouping findings into categories and grouping categories into synthesised findings (Kastner et al, 2012).

The Tools: The value of combining NVivo with EndNote

Given the diversity of qualitative research articles, Ole Steen began with a broad search of the online databases – PsychInfo and the Social Science Citation Index - through EndNote. The first search string contained twenty-one different terms. Criteria for article selection included that research samples be adolescents or young adults between 13 and 29 years old, in out-of-home placement, in the middle of transition in work or education. Preferred articles were in English. The breadth of the first search of sociological and psychological databases resulted in 1043 articles.
 
“EndNote was a relief -easy to work with and use in this kind of work"

For the initial work, EndNote simplified viewing and grouping articles and sifting out those covering irrelevant topics.
Exclusion criteria included articles on adoption, homelessness, youth crime, excess weight, sexual problems, psychiatric hospitalisation, elderly people, dementia, somatic problems (IPV, hepatitis, HIV) and out of home placements due to disabilities. Other exclusions were professionals’ perspective on transition, articles on supervision, therapy, grant proposals, policy analyses, technical papers on assessment, manuals and statistical indicators. Ole Steen also excluded research with focus groups as their research goals were not compatible with the project topic. This sorting and sifting resulted in 124 articles of interest.

Benefits of using NVivo for managing, querying and coding articles: Quickly import articles, abstracts and notes in one place

For the next step, NVivo enabled Ole Steen to import the 124 selected articles from EndNote into his NVivo project.  Although it was his first experience with NVivo he was able to do this easily and quickly.

After import, the articles attached to the original EndNote references were now all easy to access in NVivo as files ready for coding and querying. The import also automatically extracted the collection’s abstracts and notes into NVivo memos linked to the relevant articles.

With NVivo searches and text queries it was easy to find and remove from the 124 articles those exclusively on childhood trauma with no direct bearing on the study. This resulted in 70 articles ready for quality assessment.

Quality assessment of articles with NVivo

For appraising the 70 qualitative articles, Ole Steen drew on current frameworks of appraisal criteria. The most comprehensive, by Spencer et al (2003), focuses on 16 different appraisal questions for assessment of qualitative studies plus quality indicators for each question.

By developing a coding scheme Ole Steen was able to sift the qualitative studies according to criteria related to:

“It was my first experience of NVivo…but I had no problems [with NVivo]”

From this coding for the criteria and quality indicators, the appraisal process excluded more articles for the following reasons

  1. Inadequate sampling strategies
  2. Lack of systematic data analysis
  3. Quantification of qualitative information
  4. Results without documented support or citations

After appraisal there were 30 articles remaining for analysis in NVivo.

Ease of thematic analysis with NVivo

After more detailed, refined coding, Ole Steen worked with NVivo queries.  The highly heterogenous nature of the diverse studies meant that NVivo Word Frequency queries did not reveal sufficient key terms to suggest codes. However, Ole Steen found the coding queries very helpful for robust, in-depth exploration. For example, after coding for different family and social connections, the ease of running  NVivo Boolean coding queries across the different codes, enabled him to explore and expand more in-depth issues of social support networks of the transitioning youth.

“NVivo coding queries are very useful. Printing their results you can see how your codes change during your work”

Future plans for working with NVivo

From the ease with which NVivo enabled sound coding and querying in the metasynthesis, Ole Steen is now using the results to develop a hypothesis about transitioning to independent adulthood for a future publication. Based on his experience with NVivo in the metasynthesis, Ole Steen will continue to use NVivo in his research. His next study focuses on people with severe illness and how illness affects their time perspective.

With this study having benefited from NVivo’s robust coding and querying functions, in future studies Ole Steen plans to explore NVivo classifications in more detail. He will do this to compare useful article characteristics, including different research approaches within different disciplines in addition to dates of publication and countries of origin.

Furthermore, in his current teaching and supervision work, Ole Steen’s students are now using NVivo and EndNote to facilitate their research projects. They are also benefiting from the comprehensive and quality assessed usable database of literature in the foster care and transition study field that resulted from NVivo’s coding and querying functionality applied to EndNote data.

“I have discovered from the metasynthesis it is easy to generate a new hypothesis about foster youth”

For more information on the results of Ole Steen’s metasynthesis see: 
Kristensen, O. Attuning to the past while aging out of care – a Metasynthesis. Online: http://pure.au.dk/portal/files/121836846/final_ECQI_OSK_at_Leuven.pdf

Reference

Kastner M, Tricco AC, Soobiah C, et al. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Medical Research Methodology. 2012;12:114. doi:10.1186/1471-2288-12-114.
Spencer,L, Ritchie,J., Lewis, J and Dillon, L.  2003.  Quality in Qualitative Evaluation: A framework for assessing research evidence.  [Online: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/498322/a_quality_framework_tcm6-38740.pdf]

User information

https://help-nv.qsrinternational.com/12/win/v12.0.29-d3ea61/Content/files/reference-management.htm?Highlight=Endnote

About Ole Steen Kristensen

As professor in Educational Psychology in the Department of Psychology and Behavioural Sciences, at Aarhus University, Denmark, Ole Steen Kristensen divides his time between research and teaching and supervises postgraduate students.

The topic of his PhD (published 1989 in Danish) was "A social psychological study of adult education" and his fascination with teaching has led him to explore and develop teaching methods with computer technology. His research projects have focused on participatory research and developmental projects with current interests in:

Research with a conscience

In its purest form, data manipulation is the process of changing data in an effort to make it easier to read or be more organized. For example, a log of data could be organized in alphabetical order, making individual entries easier to locate. But what happens when data manipulation is not handled ethically? Controversies around Cambridge Analytica, Facebook, international fraudsters, and identity thieves have made us aware of how technology allows for our data to be manipulated.

As researchers, it’s vitally important that we’re aware of what it means to acquire and handle data ethically, especially in the face of constantly evolving technology. In this piece, we’ll look at how data can be manipulated, what it means to ethical, and how data manipulation can raise questions for researchers when it comes to technology solutions.

Data manipulation in the wild

No strangers to controversy, social media giant Facebook has walked a fine line of protecting and exploiting the data it’s the custodian of. In 2012, the organization came under fire when it was revealed that Facebook conducted a study of over 689,000 users without their knowledge or consent.

The study involved the manipulation of the users feed, to remove either positive or negative sentiment posts over the course of approximately a week, to observe how the user then posted as a result. One test decreased the users' exposure to their friends ‘positive emotional content’, which resulted in fewer positive posts of their own. Another test reduced their exposure to ‘negative emotional content’ and the opposite happened.

The study concluded:

"Emotions expressed by friends, via online social networks, influence our own moods, constituting, to our knowledge, the first experimental evidence for massive-scale emotional contagion via social networks."
 
The outrage from the academic community however, was centered on the conduct of the study, rather than the issue of data privacy alone.

It was not merely an observational study, which could be argued, since users consent in the acceptance of the Facebook terms of service. This particular study involved an intervention (i.e., the manipulation of the newsfeed), which lacked the element of informed consent for the participants.

This in itself is not necessarily unethical, as studies with interventions can be permitted on the grounds that such research aims could not be achieved any other way. However, there would be a number of standards to be met in order for such research to pass any kind of ethics test.

In the case of Facebook’s study, these guidelines were not followed or met, and it could be reasonably argued that the study was therefore unethical. 

The potential for further misuse of this kind of manipulation of data, beyond a study of its outcomes is cause for concern. When the story initially broke in 2014, Clay Johnson, the founder of Blue State Digital, the firm that managed Obama’s online campaign for the US Presidency in 2008 asked, “Could the CIA incite revolution in Sudan by pressuring Facebook to promote discontent? Should that be legal? Could Mark Zuckerberg swing an election by promoting ‘upworthy’ posts from two weeks beforehand? Should that be legal?”.

These are certainly all relevant questions which have come further into our consciousness and political discourse, given the somewhat turbulent and divided global political climate.

Data manipulation and research

What does this mean for researchers in academic institutions at all levels, particularly those who are interested in utilizing technology to further their outcomes? Data is often ‘manipulated’ (in the truest sense), to make it more usable with technology solutions that help researchers delve deeper into their sources.

Researchers understand the impetus of being ethical in all research, but when it comes to technology that is designed to make decisions on your behalf using algorithms and artificial intelligence, you could be forgiven for feeling like you’re taking a leap of faith into unknown territory.

The key to remaining on the right side of ethical standards, and being able to utilize technology as it becomes available to you, is transparency and control.

For example, the automation of transcription has long been on the wish list of many qualitative and mixed methods researchers, who have either spent many long hours of their own time, or struggled to find research assistants to transcribe interview data on their behalf. Advances in artificial intelligence (AI) and natural language processing technology have now made this a reality, and human powered transcription is no longer a researcher’s only option.

One of the advantages of utilizing transcription powered by AI and natural language processing, is the transparency in your final source. The transcription is verbatim, as opposed to an interpretation or summary of what was said from a human point of view. This means when it comes to the analysis of your data, you’re analyzing a verbatim written version of your recorded audio source.

Transparency and control are key

Taking an ethical approach to your research work, whilst also being able to take advantage of the technology offered to you is a matter of being able to maintain transparency and control over your sources.

In a digital climate that is plagued with data scandals, privacy issues and a district lack of transparency, it’s imperative the research community are not excluded from the use of new technologies, but that they are developed in a way that maintains the high standards expected by researchers.

Learn more about research transparency today in this free whitepaper Transparency in an Age of Mass Digitization and Algorithmic Analysis.

Natural Language Processing

When Joaquin Phoenix fell in love with ‘Samantha’, in the 2013 film ‘Her’, he sets about creating a meaningful relationship with an operating system that is artificially intelligent, and able to communicate with him in a language he can understand.

At the time the film was released, Apple’s Siri technology had been in the market, and in the hands of users for about two years, so the concept of speaking to a ‘smart’ device, and having it speak back to you wasn’t something entirely foreign to audiences. In the world Spike Jonze created in ‘Her’, this technology had evolved far enough that a human was able to develop a real emotional connection to it.

In reality, we’re not quite at the point where an exchange with your computer or smart device may lead you to romantic feelings, but it does make us consider where the technology is headed.

What is Natural Language Processing?

The technology that drives Siri, Alexa, the Google Assistant, Cortana, or any other ‘virtual assistant’ you might be used to speaking to, is powered by artificial intelligence and natural language processing. It’s the natural language processing (NLP) that has allowed humans to turn communication with computers on its head. For decades, we’ve needed to communicate with computers in their own language, but thanks to advances in artificial intelligence (AI) and NLP technology, we’ve taught computers to understand us.

In a technical sense, NLP is a form of artificial intelligence that helps machines “read” text by simulating the human ability to understand language.?NLP techniques incorporate a variety of methods to enable a machine to understand what’s being said or written in human communication—not just words individually—in a comprehensive way. This includes linguistics, semantics, statistics and machine learning to extract the meaning and decipher ambiguities in language.

How is it used?

Frequently used in online customer service and technical support, chatbots help customers speak to ‘someone’ without the wait on the telephone, answering their questions and directing them to relevant resources and products, 24 hours a day, seven days a week.

In order to be effective, chatbots must be fast, smart and easy to use, especially in the realm of customer service, where the user's expectation is high, and if they’re experiencing a technical issue, their patience may be low. To accomplish the expected level of service,?chatbots are created using NLP?to allow them to understand language, usually over text or voice-recognition interactions,?where users communicate in their own words, as if they were speaking (or typing) to a real human being. Integration with semantic and other cognitive technologies that enable a deeper understanding of human language allow chatbots to get even better at understanding and replying to more complex and longer-form requests.

In a research context, we’re now seeing NLP technology being used in the application of automated transcription services (link out NVivo transcription). Transcription is one of the most time-intensive tasks for qualitative, and mixed methods researchers, with many transcribing their interviews and focus group recordings themselves by hand. Unless you’re an incredibly fast and accurate typist, this is an incredibly laborious task, taking researcher’s time away from the actual analysis of their data.

Automated transcription tools utilize NLP technology to ‘listen’ to recordings of data such as focus groups, and interviews, and interpret them and produce them into a format and language that is useful for the researcher to go on and analyze, either manually, or using software.

Future uses of NLP

The NLP market size is estimated to grow to USD 16.07 Billion by 2021, globally, giving us a strong indication that NLP technology has huge growth opportunities across a number of sectors.

An understanding of human language can be especially powerful when applied to extract information and reveal meaning or sentiment in large amounts of text-based content?(or unstructured information), especially the types of content that has typically been manually examined by people.

Analysis that accurately understands the subtleties of language, for example, the choice of words, or the tone used, can provide useful knowledge and insight. NLP will play an important part in the continued development of tools that assist with the classification and analysis of data, with accuracy only improving as technology evolves.

Academics at the University of Bologna have applied NLP to the most used part of any academic article: the bibliography. A group of researchers are developing tools that can extract information on citations using natural language processing and common ontologies (representations of concepts and their relationships) that can be openly accessed and connected to other sources of information. The idea of the project is to enrich the bibliography in order to give the reader more comprehensive information about each single entry, instead of looking at the bibliography as one large piece of information.

In the commercial word, NLP analysis will have uses especially in the analysis of the typically carefully worded language of annual reports, call transcripts and other investor-sensitive communications, as well as legal and compliance documents. Effective analysis of sentiment in customer interactions will allow for organizations to make improvements in their product and service delivery outcomes.

NLP will be essential to the future of research

More effective and accurate understanding between humans and machines will only strengthen the efficiencies and outputs of those who need to understand and analyze unstructured data.

No matter where it is applied, NLP will be essential in understanding the true voice of the research participant, the customer, or the user and facilitating more seamless interaction and interpretation?on any platform where language and human communication are used.

To read more about automation, AI technology, and its effect on the research landscape, download this free whitepaper Transparency in an Age of Mass Digitization and Algorithmic Analysis.

Part Two: 14 Reasons why NVivo is the best qualitative data analytics software

On the first day of research, my data gave to me... (Part two)

Today we continue our 14 days of research journey with 7 more interesting elements of NVivo to discover. Click here if you’ve not read the first part of this series.

8 Interviews Auto-coded 

If you make one resolution for 2023, make it to be more efficient with Coding. Teach NVivo how you want your data coded, then let NVivo machine learning do the other 90% and present it to you for checking. Auto-coding that starts and ends with you! Learn more about auto-coding.

https://lumivero.com/gated-content/autocoding-with-nvivo-on-demand/

9 Free Webinars 

Our new year program of live webinars already has nine webinars available for you to register for. All webinars are recorded, with the recording sent to all registrants. Click here to see if there’s a webinar that interests you.

https://lumivero.com/resources/webinars-events/

10 Analyzed Sentiments 

Automated insights, including sentiment analysis, enables quick identification of themes and sentiment. This blog post shows how the tools, which are all part of NVivo as standard now, were used to analyse a large volume of student feedback. Read the blog now.

https://lumivero.com/resources/case-studies/analysing-survey-data-using-automated-insights/

11 Coding Stripes 

In NVivo 14, coding stripes will be making a big difference to how you code individually or as a team. Click here to learn how Coding Stripes will work better for you. https://lumivero.com/resources/applause-for-the-new-nvivo/

12 NVivo Academy Lessons 

Our NVivo Academy helps take your NVivo learning from beginner to expert – the on-demand courses focus on core NVivo skills, but don’t forget our focused learning modules on Coding and Queries. Click here to supercharge your NVivo learning.

https://lumivero.com/resources/academy/

13 Citavi References Exported 

Citavi is the only all-in-one reference management and knowledge organizer available, and with NVivo 14 it’s even more closely integrated to make light work of creating that literature review. Watch the webinar to see how Citavi can help you and then get a free 30-day trial. https://info.qsrinternational.com/FY23_CT_WBR-TL_12-01_Develop-Scientific-Arguments-with-Citavi_Registration.html

14 A new version of NVivo in the New Year 

Part One: 14 Reasons why NVivo is the best qualitative data analytics software

On the First day of research, my data gave to me...

Join us as we help you to (re)discover some of the key elements of NVivo that will help set you up for success in 2023. Our 14 days of research (in honour of NVivo 14, which is coming early 2023), selects some of the most useful elements to learn more about…

1 Upgrade to NVivo 14 

NVivo’s latest edition, with the reconceptualized NVivo Collaboration Cloud, introduces more powerful, real-time collaboration and insights for research, analysis, writing, publishing, and real-time coding across teams and operating systems. Experience the joy of a single project file, whether you work with Windows or Mac, with NVivo 14. Click here to learn more https://lumivero.com/nvivo-product-tour/

2 Codes Coded 

Coding is central to successfully getting richer insights from any qualitative research projects. Our blog on perfecting the art of qualitative coding makes a great Christmas read on things to consider before, during and after the coding stage. Read the blog here https://lumivero.com/resources/perfecting-the-art-of-qualitative-coding/

3 Memos Shared 

An invaluable tool for all qualitative researchers, memos can be used in a variety of ways from both a wider view, to a more detailed close-up on data excerpts. Watch this webinar to learn about different ways memos could be used to enhance your research. https://info.qsrinternational.com/Analytic_Memo_Strategies

4 Colleagues Collaborating 

We’re super excited to reveal that in NVivo 14, your research team using NVivo Collaboration Cloud will be able to collaborate in real-time, whether on Mac or Windows. Get ready for real-time collaboration with this topical blog. https://lumivero.com/resources/collaboration-in-qualitative-research/

5 Transcription Hours 

If you buy NVivo now, our fantastic end of year offer means not only will you also get NVivo 14, when it’s released in early 2023, but you’ll also get FIVE hours of NVivo Transcription absolutely free. And if that wasn’t enough, we’re giving you access to NVivo Core Skills course to get you up and running with NVivo in no time. Learn more about the complete offer here.

https://lumivero.com/nvivo-product-tour/

6 Fancy Word Clouds 

Your methodical review of the qualitative data and coding effort has resulted in a massive amount of interesting findings. Present your findings in easy-to-digest, visual formats quickly and easily with NVivo – the Word Cloud is perfect. Learn More

https://lumivero.com/resources/preparing-the-presentation-of-qualitative-findings/

7 Mind Maps Running 

If a picture is worth a thousand words, imagine how visualizing your data could enrich your research experience, creating yet another perspective from which to view your data. Yesterday Word Clouds, Today Mind Maps! Learn more about data visualizations. https://info.qsrinternational.com/visualize_data

Impatient to see the rest? CLICK HERE to visit part two of the 14 days of research.

Welcome to Academic Writing Month! (#AcWriMo) 

Academic Writing Month runs in November and challenges participants to set and meet a writing goal during the month.  

To help everyone participating, we’ve done a round of up our best scholarly writing resources and will also release great new writing content regularly during November.  

Let’s go! 

 

[Blog] What is #AcWriMo?  

Learn where #AcWriMo came from and how it helps scholarly writers  

Read Now  

 

[On Demand Webinars] Scholarly Writing  

Check out these webinars from our ongoing Scholarly Writing webinar series when it suits you 

 

[Live Virtual Learning] Writing Groups for Scholars 

Register for a free virtual writing group to ask questions and learn best practices from experts and peers on academic writing and publishing  

Register Now 

 

[On Demand Webinar Series] Summer Scholarly Writing Institute 

NVivo and Citavi partnered with Dissertation by Design to present the virtual Scholarly Writing Institute. These four sessions will help you enhance your writing skills and learn more about digital tools, like NVivo and Citavi, that can assist with the writing and publishing process.  

Watch Now 

 

[Blog] Extending Your Literature Review with NVivo 

Learn how NVivo adds new possibilities for analyzing literature, for example, exploring relationships between articles, authors and books.  

Read Now 

 

[Blog] Speed Up Publishing with NVivo and Citavi 

If you’re working with NVivo to help you analyze the results from your original research or supporting literature, Citavi can help you be more efficient in the other stages of the research process. 

Read Now 

 

[Blog] NVivo and Citavi Make it Easier than Ever to Write and Reference 

Learn how to speed up publishing by taking advantage of quick imports of literature into NVivo for coding, and exports to add citations through Citavi 

Read Now 

We hope you find these resources helpful in your writing journey! 

To make sure you don’t miss more great new scholarly writing content, sign up to receive blog updates using the blog subscribe box at the bottom right of this page, and follow us on
Facebook,
Twitter and
LinkedIn. 

Analyzing YouTube Comments with NVivo

Beauty has long been a topic of interest, from philosophers who struggled to define it to artists who sought to portray it. And now, with an entire industry devoted to products and procedures that claim to enhance it, it is a treasure trove of research with implications for everything from sociology to marketing.

Researchers also have a new avenue for collecting and analyzing data on beauty: social media. That’s where Jenna Koske went for her undergraduate research project for the College of Business at James Madison University. Specifically, she studied beauty YouTubers—individuals who do video makeup tutorials and product reviews on the platform—and how comments on their videos illuminated shifts in gender roles and identity, the effect of parasocial relationships, and the commercial implications of beauty influencers. Dr. Stacy Penna talked to
Ms. Koske about the project for the
46th episode of the Between the Data podcast.

“I knew I wanted to do something within the realm of digital marketing,” said Ms. Koske, explaining how her project aligned with her career goals. In addition, she was interested in exploring how social media impacts our daily lives. Combining the two posed an opportunity to contribute new findings to research.

Focus on Major Beauty Influencers

Ms. Koske chose to focus on three beauty YouTubers with significant followings:

Ms. Koske had a solid background in NVivo already, having helped out her advisor—who wanted to highlight NVivo’s usefulness for undergraduate research—by analyzing papers with the software for demonstration purposes and creating tutorials.  For her own project, she used an online bot to scrape the top comments from a selection of the influencers’ most popular videos, which she exported into XML files and then imported into NVivo.

From the top comments for the top 25 videos for each YouTuber, Ms. Koske collected a total of almost 2000 comments. The data included user names, the date it was posted, and the number of up votes and replies as well as the content of the comment. From this, she developed 13 themes and eight attributes.

To compare the prominence of the themes among the three influencers, she used the matrix coding query to count how many times each theme appeared as well as any overlaps and combinations. “This gave me a really good starting point to examine the comments in a more unstructured, qualitative way,” she says.

Key Findings

The themes of “laughter” and “amusement” were often signaled by commenters making jokes, which Ms. Koske calls “indicating that commenters “feel a personal connection with the YouTuber,” a critical component of parasocial relationships.

Tati’s commenters tended to fall into the themes of “desire,” “nostalgia,” and “suggestion.” “A lot of her users really wanted to provide ideas and what content she should post next,” Ms. Koske says.

Comments in her “disbelief” and “makeup compliment” themes were much higher for James and Nikki than for Tati, which led Ms. Koske to question whether their talent and creativity with makeup was more surprising to commenters because James is a man and more impressive because Nikki is transgender. James had the most negative comments, but Nikki had the most positive, which Ms. Koske surmised was related to the overwhelming amount of support Nikki received after revealing her transgender status.

Ms. Koske also noted the different ways the influencers developed trust and community with fans and commenters. Nikki “was all about portraying herself as very genuine and authentic,” she says. Disclosing that she is transgender actually brought her more support. James refers to his followers as “sisters,” which Ms. Koske says he uses in an inclusive sense despite the word’s gendered definition.  Tati developed credibility for her objective and straightforward product reviews, which Ms. Koske posits allows her “to walk this line between professionalism and being the big sister maternal figure.”

Implications for the Cosmetics Industry

Ms. Koske’s findings have significance for beauty and cosmetics companies and their marketing efforts. She points out that user-generated content like YouTube videos and comments are valuable sources for target market research on consumer behavior and characteristics as well as pain points. For instance, she noticed commenters on Tati’s videos complaining about a lack of affordable products and not enough diverse shades of foundation—which could help companies identify gaps in the market. James’ popularity could point to a potential customer base for products aimed at men, who she says are starting to care more about hygiene and appearance.

She also found that consumers of cosmetics and other beauty products often view influencers as more trustworthy than ads and other marketing materials. “Companies need to align themselves with someone that they think represents their brand well,” she says.

Ms. Koske’s advice to other undergraduates doing research projects: “Find something that you’re really passionate about.”  In this case, Ms. Koske’s interests in social media, gender, and beauty led to her receiving the honor of presenting her research at the National Conference for Undergraduate Research this past April.

Learn more about this research:
Listen to the full podcast episode here.

Using Qualitative Data Analysis to Study Perceptions of Breast Cancer Risk

NVivo was used to enable a recent study that identified family history as the most significant factor in assessing personal breast cancer risk, but health behaviors and environment were also influences.

October is Breast Cancer Awareness month—which makes it a good time for everyone to assess their risk of developing the disease. Advances in genetic testing and large-scale studies have given medical professionals the ability to calculate individual risk levels for breast cancer with greater accuracy. However, how women perceive that risk varies significantly from how clinicians view it. For many women, it’s about more than just statistics—which can have significant implications for their health and their clinical care.
 

Researching Qualitative Data on Risk Assessment

A recent study in the British Journal of Cancer used NVivo to analyze personal versus clinical perceptions of breast cancer risk. Researchers scoured databases for studies on women with a higher-than-average risk of breast cancer to gather qualitative data on how they appraised their own risk. They used NVivo to analyze that data with inductive line-by-line coding. Seven descriptive themes were then developed by merging codes based on their conceptual meanings and how they related to each other. Lastly, researchers considered the links between the descriptive themes to identify four analytical themes:

To complete the research, Dr. Corley’s team used NVivo to:

Better Communication = Better Healthcare

Psychology has shown that people don’t make decisions based on reason and facts alone, and that is certainly true for women assessing their personal risk of breast cancer. The researchers in this study note that additional studies are needed to dive deeper into the nuances of their findings, but their conclusions can help doctors have meaningful discussions with their patients about their clinical risk for breast cancer and their subjective understanding of that risk.

Ultimately, these discussions could lead patients to take steps to prevent breast cancer, such as changing their diet or exercising more. They might also encourage more women to monitor their breast health through regular self-exams and mammograms, enabling their doctors to detect cancer earlier, when it is more treatable and curable.

While science may not be able to prevent breast cancer, research using qualitative data analysis can help doctors serve their higher-risk patients more effectively and holistically, potentially leading to improved outcomes for both prevention and treatment—and more lives saved.

Learn more about how NVivo supports qualitative data analysis to help scholars uncover new insights.