What is MaxDiff analysis?
MaxDiff (Maximum Difference Scaling) also called BWS (Best Worst Scaling), is a survey research technique used to measure preferences or the relative importance of multiple items. It helps to identify which options or attributes are most and least important to respondents by asking them to make choices between sets of items.
How Does MaxDiff Work?
Design the Survey:
- A list of items (e.g., attributes, products, features, etc.) is created.
- These items are grouped into sets, typically 3 to 5 items per set, though larger sets are also used.
Respondents' Task:
- In each set, respondents are asked to select the item they find most and least important or appealing.
- For example, if the items are A, B, C, D in one set, the respondent might choose A as the most important and C as the least important.
Analysis:
- The choices across all sets are analyzed to determine the relative importance of each item.
- The analysis typically involves statistical models that account for the frequency and pattern of selections.
Why Use MaxDiff Analysis?
- Discrimination Power: MaxDiff forces respondents to make trade-offs which often leads to clearer distinctions between items compared to traditional rating scales where people may rate everything similarly.
- Preference Insights: It provides detailed insights into how much more important an item is over another, which can be more actionable than just knowing that an item is "important."
- Simpler Task: Choosing the most and least important items from a set is cognitively simpler for respondents than ranking a long list of items or assigning scores to each one.
Where is MaxDiff Used?
- Product Development: Companies use MaxDiff to determine which product features are most valued by customers, guiding decisions on what to prioritize in development.
- Marketing Research: It’s used to understand consumer preferences for branding, packaging, advertising messages, or product attributes.
- Pricing Strategy: MaxDiff can help identify which product features customers are willing to pay more for, assisting in pricing and bundling decisions.
- Public Policy: Researchers and policymakers use MaxDiff to assess public preferences on different policy options or social issues.
MaxDiff Example Scenario
Imagine a company wants to understand which smartphone features are most important to users. They list features like battery life, camera quality, screen size, and storage capacity. In the MaxDiff survey, respondents are shown sets of these features and asked to choose which feature is the most and least important in each set. The results help the company prioritize which features to focus on in their next product launch.
Key Advantages of MaxDiff
- Clear Prioritization: It gives a clear picture of which items are more preferred or important than others.
- Efficient Data Collection: Compared to full ranking methods, MaxDiff requires fewer comparisons to get reliable data.
- Avoids Rating Bias: Since respondents can’t rate everything equally important, it reduces biases like the tendency to avoid extreme ratings.
Using the ErrVarNorm Index for Improved Accuracy
As a reminder, MaxDiff consists of giving each respondent combinations of attributes and then asking them to select for each combination the Best and Worst attribute. But what happens when two respondents give the exact opposite answers of Best and Worst attributes?
Let's go back to the Smartphones example and explain it in more detail. The 10 attributes are:
- Battery Life
- Camera Quality
- Storage Capacity
- Price
- 5G Connectivity
- Processor Speed
- Screen Size
- Water Resistance
- Fast Charging
- Build Quality
The goal is to measure the importance of each of the terms to make decisions about the next product and marketing.
So, each respondent will see several combinations of attributes, like this:
Now imagine if:
- In Set 1: The respondent said that Battery Life is the Most important, and Camera Quality the Worst
- In Set 3: The respondent said that Camera Quality is the Most important, and Battery Life the Worst
How do you draw conclusions based on that?
That’s why we need to consider only consistent respondents!
Using XLSTAT to Solve for Respondent Consistency Issues
After research, our team led by Dr. Fabien Llobell has detected that the classical index for this is RLH (Root LikeliHood) index. It’s a value between 0 and 1 and increases with the consistency of the respondent. But we found a lot of issues with this index:
- Really long computing time (in our previous example, compute RLH means compute the Hierarchical Bayes approach so 19 minutes!)
- A perfect consistent respondent can have a RLH lower than 1 (the max)
- 2 equally consistent respondent can have different RLH index
- The RLH index depends totally on the number of attributes in each set
Therefore, this index is very hard to use in real life situations and users struggle to make decisions based on it.
That's where the ErrVarNorm index comes in! The ErrVarNorm index is:
- Extremely fast to compute!
- A perfect consistent respondent has ErrVarNorm= 1 (the max)
- 2 equally consistent respondent have same ErrVarNorm index
Our ErrVarNorm index has minor dependency on the number of attributes, making it more robust when dealing with larger datasets
We have therefore a homemade index to answer an important need!
Dr. Fabien Llobell presented this new index at the 2024 Sensometrics Conference, 2024 Eurosense Conference, and the 2024 Lumivero Conference in partnership with Professor Sara Jaeger.
Start Using MaxDiff Analysis in XLSTAT
As you can see, MaxDiff is a powerful tool in market research and decision-making, helping to reveal true preferences and priorities among a set of options.
Try it out for yourself! Download your free 14-day trial of XLSTAT.