Analyzing performance in a CATA experiment and further developing CATATIS with Givaudan

Introduction

Our R&D and Engineering teams are always working to bring new ergonomic functionalities and innovative statistical tools to XLSTAT. Many of these developed or improved features come from user requests or as a solution to a specific use case. The CATATIS feature is one of them.

Laure Bonnet works as a sensory project manager at Givaudan. The global industry leader in taste and wellbeing offers its food and beverage customers a wide range of solutions that meet consumer demand for clean label, organic and natural ingredients. Givaudan’s products contribute to happier and healthier lives and are manufactured in a way that respects natural resources and the environment.

This story highlights a collaboration between Givaudan (Laure Bonnet), Oniris (Pr. El Mostafa Qannari, Thibault Ferney, and Thibaut Riedel), and XLSTAT (Dr. Fabien Llobell) to further develop the CATATIS feature.

The challenge:

The limits of CATA analysis became apparent when traditional methods were applied to trained panel data. The Check-All-That-Apply (CATA) test is one of the so-called Rapid Tasks and is typically conducted with consumers to understand their perceptions of products. Some companies also use CATA with trained panels when samples within a set are very different.

However, there is an incompatibility between traditional CATA data analysis methods and the nature of the data when collected by means of a trained panel. On the one hand, judges have evaluated products more than once (different sessions). On the other hand, as with any analysis with trained panelists, one may question the accuracy of the agreement between repeatability and judges (homogeneity/consistency of the panel).

When Laure Bonnet was conducting research on six fish-flavored products using a CATA task, she found herself questioning some of the data.

The solution:

Improvement of the CATATIS feature to resolve incompatibility between method and data became the focus of a collaboration between Givaudan, Oniris, and XLSTAT.

Laure Bonnet and her team first conducted three sessions with a total of 12 subjects who evaluated 27 taste and smell attributes for each fish-flavored product. Once the study was completed, the team coded all the innovative methods used during the collaboration in the CATATIS feature of the XLSTAT software: possible sessions, repeatability study, consistency/homogeneity tests, and finally weight tests.

In order to consider the sessions and to study repeatability, the sessions were selected in the CATATIS function. In the outputs, the weight tests and consistency tests were selected. When combining CATATIS with significance, the data showed a very high similarity between the sessions, indicating that the repeatability of the subjects is strong.

Evaluating the sessions in the CATATIS method allowed the team to use all available characteristics to create a product/attribute map. The target product was clearly perceived as tuna, either by taste or smell.

With the CATATIS method, weights were assigned to each subject during map generation. These weights reflect the agreement of a subject with the rest of the panel. The team’s first development in CATATIS allowed them to check whether these weights were significant or not, and the results showed that all subjects were in agreement with the panel.

It is also important to determine whether the panel had greater homogeneity than that which would result by chance if the data structure remained the same. With the team’s second development, homogeneity was tested for the entire dataset as well as for individual attributes. The results showed that the panel is consistent as the homogeneity is significantly greater than the random dataset.

However, the team looked at each attribute individually, they saw that the subjects disagreed on the attribute amine. It is possible that the subjects did not fully understand this attribute.

The result:

Thanks to this collaboration, XLSTAT users can now:

• Perform a CATATIS analysis on data containing different sessions
• Determine if the assessors behaved similarly across sessions by using the inter-subject repeatability index (between 0 and 1)
• Find out whether a subject is significantly in agreement with other subjects by performing a permutation test
• Use a homogeneity test (or consistency test) on all attributes to check whether the data is exploitable
• Determine if any attributes were misunderstood by subjects or detect very high heterogeneity of responses to that attribute with an attribute-by-attribute homogeneity test (or consistency test)

This work was presented by Laure Bonnet in September 2022 at the Eurosense conference in Turku, Finland.