Consumer Insights Tracker Technical Report: Data Analysis

Data analysis 

The data analysis process used focuses on univariate and bivariate quantitative analysis. Analysis for each question is conducted at the overall sample level, as well as across key demographic groups, using the weighted data. Analysis of changes across waves, as well as differences between demographic groups, are only reported when these differences are statistically significant (discussed further below).

Index of Multiple Deprivation (IMD) is one of the demographic factors calculated and reported for England, Wales and Northern Ireland. IMD is calculated using country-specific metrics, and then combined to give an overall measure of relative deprivation within a respondent’s specific region. 

• England: The IMD ranks each English LSOA from 1 (most deprived) to 32,844 (least deprived).
• Wales: The Welsh equivalent (WIMD) ranks each Welsh LSOA from 1 (most deprived) to 1,909 (least deprived).
• Northern Ireland: The NI equivalent ranks each OA from 1 (most deprived) to 5,022 (least deprived).

Statistical reliability and confidence intervals

As only a sample of the population, rather than the entire population, is surveyed each month, the results are subject to sampling tolerances, which vary with the size of the sample and the percentage figure concerned. For example, for a question where 50% of participants in a (weighted) sample of 2,000 respond with a particular answer, the chances are 95 in 100 that this result would not vary more than 2.2 percentage points, plus or minus, from the result that would have been obtained from a census of the entire population (using the same procedures). 

Table 3 provides a useful rule of thumb when judging the statistical significance of the figures contained in the final dataset.

Table 3:  Confidence intervals for a sample of n=2,000 for various results 

Reporting 

The FSA project team work closely with YouGov each wave to identify results of strategic importance for the FSA. Toplines typically focus on new/emerging issues, significant changes in findings over time, an overview of each topic from the Consumer Insights Tracker (e.g. food availability, food insecurity, consumer concerns) and key demographic differences of interest. Findings also consider key demographic groups of interest to the FSA, particularly when they indicate a significant difference to the total survey population. Changes over time, or between demographic groups, are only reported when they are statistically significant using t-tests ^{(footnote 1)}, at p<0.05. This helps to ensure that only ‘true’ shifts in behaviour and attitudes are focused on in the report (i.e. they are unlikely, 5% or less, to be a result of chance).

The data for this project and other research that the FSA undertakes can be found on the FSA’s website, and in particular via the FSA data catalogue.