Bayesian Analysis of Preference Tests
Combining historical (or expert opinion) data with preference test data to improve precision and enhance decision making
The issue: Preference tests are simple to carry out and routinely done, but standard analysis using a “frequentist” approach, means the decisions are based only on data collected for today’s study, whereas the Bayesian approach offers the chance to combine historical data with current data, providing a formal (rather than subjective) way of incorporating prior knowledge into decision making. In addition, the frequentist approach depends on the concept of repeated sampling and is heavily reliant on statistical significance making the chosen sample size crucial. The Bayesian approach eliminates the reliance on p-values and instead quantifies decision risk in an intuitive and business-relevant manner and provides a basis for reduction in sample size given prior knowledge and agreed risk tolerance.
The problem: Bayesian analysis is new to the sensory and consumer world and is not currently available in many stats software package and requires coding or specialist software.
The solution: The Qi Bayesian app allows the user to perform a simple Bayesian analysis of a preference test with minimal data input and clear graphical outputs, which can be compared to desired target performance goals and business decisions.
Bayes Preference Test Application
The user first sets a prior belief of how the data should look (a prior distribution for p) based on historical data or expert opinion, in the app. Then the results of the actual preference test are input. These 2 items are then combined to come up with a Beta posterior distribution graph
The app gives the user the parameters and characteristics of the posterior distribution and compares these to a user-defined target performance and action standards.
The ‘app’ works extremely quickly, and so the user can change the basic parameters and see instantly how changing the shapes of the held beliefs or previous results (prior distributions) will affect the resulting posterior distribution.
In addition performance against different business targets can also be assessed interactively. The output is shown clearly with colour coding for ‘good and ‘bad’ results compared to target performance.
Once the analyses are finalised the results can be exported.
- Web based
- Simple to use and input data
- Complex stats method made easy
- Easy to calculate many scenarios and try different priors
- Clear visualisations with interactive functions for comparing to ‘target’ deliverables
- Results displayed and exported simply
- Colour coding to clearly identify ‘good’ and ‘bad’ results compared to target performance of the product
$392.34 – $980.85 (Excluding any applicable taxes)