Skip to main content
English Cymraeg
Developing rapid and effective communications testing

Developing rapid and effective communications testing: Appendices

Appendices A, B and C for the developing rapid and effective communications testing report.

Last updated: 3 February 2023
Last updated: 3 February 2023

A. Methodology

Ipsos piloted the following  tools:

  • i:Omnibus: Ipsos’ standard surveying approach using our online panel to access of a representative sample of 2,000 adults aged 16-75 in the UK. Fieldwork took place across two waves, the first being 4-7 March 2022 and the second 14-16 March 2022
  • fastFacts: This tool is very similar to the i:Omnibus approach, using the same questionnaire. However, it is more flexible, as it is a stand-alone survey as opposed to an Omnibus and provides quicker result to test during a crisis comms, for example. Fieldwork took place on 21 March 2022
  • Ipsos DUEL: A standardised survey tool which aims at measuring front of mind preferences between pieces of material and infer what specific words/ messages/ images land best. Participants quickly pick between different stimuli and were asked to select the ones that they found most helpful in knowing when milk is safe to drink. Fieldwork took place 25-26 March 2022. 

B. Communication pieces tested

Through the Creative Testing platform (i:Omnibus and FastFacts) we tested 27 pieces featuring a mix of characteristics, as the table below summarises. 

These pieces of communication can be found by following the URL links in the table below. Where some materials were still only drafts, the exact text tested has been provided instead.

List of communication titles and links to the communication:

Table 2.1 Pieces of communication tested through Ipsos Creative testing tool

Features Count (amongst 27 pieces tested)
Topics Food safety: 9
Food sustainability: 6 
Foodborne illnesses: 2
Novel food: 5
Food allergies: 5
Objectives Increasing reputation of the FSA: 4
Raising awareness: 12
Increasing understanding and knowledge: 12
Changing behaviours: 3
Live pieces versus draft messages Live pieces: 19 
Draft messages: 8
Formats Social media posts: 6
Articles: 3
Blogs: 3
Messages: 8
Videos: 7
Channels Twitter: 6
FSA website: 1
Gov.UK: 4
ITV News: 1
YouTube: 5
Facebook: 1
TikTok: 1
N/A (draft message): 8
Voices FSA: 16
External: 3
N/A (draft message): 8
Use of expert voices

Y: 15

No: 4

N/A (draft messages): 8

Use of supporting visuals

Yes: 18 (10 videos, 8 images)

No: 9 (including draft messages)

Use of case studies/everyday stories

Y: 4

No: 23

Practical tips/advice provided

Y: 14

No: 13

Use of statistics/scientific terms

Y :6

No: 21

Signposting provided

Y: 12

No: 15

Through DUEL we tested 16 combinations of the same message (around milk best before and use-by dates). 

Table 2.2: Different executions of the message tested (16 in totals) 

Messages No image Picture of milk Picture of milk and a child Graphic style image
Message 1: Use by + sniff test (but NO explanation of use by/best before) Stimulus 1 Stimulus 2 Stimulus 3 Stimulus 4
Message 2: Use by + explanation of use by/best before (but NO sniff test)  Stimulus 5 Stimulus 6 Stimulus 7 Stimulus 8
Message 1: Best before + sniff test (but NO explanation of use by/best before) Stimulus 9 Stimulus 10 Stimulus 11 Stimulus 12
Message 2: Best before + explanation of use by/best before (but NO sniff test) Stimulus 13 Stimulus 14 Stimulus 15 Stimulus 16

C. Data tables

Table 1: Overview of how different engagement indicators vary across each other, showing highest, lowest and average.

Piece of communication tested Lowest score Average Highest score
Carries an important message 28% 60% 80%
Is relevant 22% 50% 73%
Clear 30% 58% 80%
Is trustworthy 23% 52% 72%
Told me something new 21% 42% 62%
Is for someone like me 16% 36% 57%
Eye-catching 12% 30% 50%
Interesting 22% 42% 60%
Memorable 17% 34% 54%
Made me emotional 8% 14% 24%

E7. How well does each of the following statements describe the information we have shown you? Showing highest, lowest, average NET scores (average of those selecting 7-10 on a scale of 0-10 where 0 is not at all and 10 completely), across all 27 pieces of content.

Table 2: Overview of awareness and familiarity for the top/ bottom performing pieces of content (ranked by familiarity shift)

Pieces of communication tested Awareness shift Awareness baseline Familiarity shift Familiarity baseline
Average across all pieces of content +1 84% +8 54%
Video, FHRS explained +4 86% +20 61%
Draft message Milk 3 +8 86% +19 54%
Article, cooking frozen raw chicken +6 77% +18 47%
Tweet reheating rice safely +2 82% +17 50%
Draft message Milk 4 +6 83% +15 54%
Video, use by vs best before +5 88% +15 64%

A1/C1. How much, if anything, do you know about the Food Standards Agency, also known as the FSA? Combined figures for awareness are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does, I’ve heard of the FSA but know nothing about it. Combined figures for familiarity are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does. 

Table 3: Overview of Awareness and Familiarity for the bottom performing pieces of content (ranked by familiarity shift)

Pieces of communication tested Awareness shift Awareness baseline Familiarity shift Familiarity baseline
Average across all pieces of content +1 84% +8 54%
Video, PPDS explained -5 92% 0 60%
Draft message CBD 3 -6 86% -1 52%
Article, consuming insects +2 86 +-2 52
Tweet, wrinkly apples 0 89% -2

64%

Draft message CBD 4 -1 75% -4 45%
Draft message CBD 1 -9 82% -5 46%

Table 4: Overview of favourability for the top/bottom performing pieces of content (ranked by highest shift in favourability)

Pieces of communication tested Favourability shift Familiarity baseline
Average across all pieces of content +4 62%
Video, FHRS explained +30 53%
Article, cooking frozen raw chicken +25 57%
Video, Tyler on TikTok +15 59%
Tweet, reheating rice safely +14

64%

Draft message Milk 3 +11 63%

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 5: Overview of favourability for the bottom performing pieces of content (ranked by shift in favourability)

Pieces of communication tested Favourability shift Favourability baseline
Average across all pieces of content +4 62%
Video, Jack on allergies -3 68%
Draft message: Milk 4 -4 76%
Article, environmental labelling rules -5 60%
Draft message: CBD 4 -6 59%
Blog, climate change and diet -12 69%

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 6: Overview of trust for the top/bottom performing pieces of content (ranked by shifts in trust) 

Pieces of communication tested Trust shift Trust baseline
Average across all pieces of content +3 69%
Article, cooking frozen raw chicken  +24 64%
Video, use by vs best before  +18 63%
Video, FHRS explained +14 65%
Draft message: Milk 3 +13 63%
Video, Tyler on TikTok +9 69%

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it. 

Table 7: Overview of trust for the bottom five performing pieces of content (rank by shift in trust)

Pieces of communication tested Trust shift Trust baseline
Average across all pieces of content +3 69%
Draft message: CBD 1 -5 61%
Article, consuming insects -6 71%
Video, PPDS explained -6 74%
Article, environmental labelling rules -8 75%
Blog, climate change and diet -9 72%

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it.

Table 8: Pieces of communication showing greatest/lowest levels of increase in familiarity with of topic (ranked by shifts in familiarity)

Piece of communication tested Topic Awareness shift Awareness baseline Familiarity shift Familiarity baseline
Average across all pieces of content - +2 89% +3 54%
Video, PPDS explained Rules and information available about allergens +7 90% +27 40%
Video, New allergen labelling laws Rules and information available about allergens +6 92% +22 39%
Article, cooking frozen raw chicken Food hygiene at home +2 92% +19 72%
Article, consuming insects Novel food (e.g., CBD/ cannabidiol, genetically modified food) +4 98% -12 86%
Video, FHRS explained Food hygiene when eating out +3 98% +16 63%
Draft message: Milk 3 Date label, such as "best before" and "use by" labels -4 98% -12 86%
Draft message: Milk 4 Date label, such as "best before" and "use by" labels 0 95% -12 86%
Tweet, raw eggs Food hygiene at home -1 97% -10 80%
Draft message: Milk 2 Date label, such as "best before" and "use by" labels 1 97% -9 83%

QC5. How much do you feel you know about the following topic? Combined figures for awareness are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it, I’ve heard of it but don’t know much about it, I’ve heard of it but don’t know anything about it. Combined figures for familiarity are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it.

Table 9: Top/bottom performing pieces of content for increasing levels of concern (ranked by shift in concern)

Pieces of communication tested Topic Concern shift Concern baseline
Average across all pieces of content - +7 49%
Video, use by vs best before Date label, such as "best before" and "use by" labels +20 44%
Draft message: Milk 3 Date label, such as "best before" and "use by" labels +18 36%
Draft message: CBD 3 Novel food (e.g., CBD/cannabidiol, genetically modified food) +17 36%
Draft message: Milk2  Date label, such as "best before" and "use by" labels +17 39%
Article, environmental labelling rules Making diets more sustainable +16 51%

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 10: Pieces of content showing lowest levels of increase in concern (ranked by shift in concern)

Pieces of communication tested Topic Concern shift Concern baseline
Average across all pieces of content - +7 49%
Tweet, freezer-friendly food Food waste -1 78%
Video, FHRS explained Food hygiene when eating out -3 73%
Draft message: 
CBD 4
Novel food (e.g., CBD/ cannabidiol, genetically modified food) -5 42%
Tweet, check FHRS Food hygiene when eating out -5 67%
Draft message: 
CBD 1
Novel food (e.g., CBD/ cannabidiol, genetically modified food) -5 35%

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 11: Increases in levels of intended behaviour – top/bottom performing content (ranked by intended behaviour shift)

Pieces of communication tested Target behaviour Intended behaviour shift Intended behaviour baseline
Average across pieces - +2 49%
Tweet, Reheating rice safely Storing cooked food (eg rice, meat etc) in the fridge, as opposed to leaving it out overnight +14 65%
Video, Jack on allergies Checking information about food allergies before buying/ consuming food +11 29%
Video, PPDS explained Checking information about food allergies before buying/ consuming food +8 34%
Draft message: Milk 3 Throwing away food which has passed its use-by date +7 40%

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. Source: Ipsos’ online panel, first wave fieldwork: 04/03/2022-07/03/2022, second wave fieldwork: 14/03/2022-16/03/2022, FastFacts fieldwork: 21/03/2022. Bases: All UK adults shown one piece of comms (between first wave, n = 2,243 and second wave, n = 2,246; n = 500 for “Pork scratchings recall”).

Table 5.10: Increases in levels of intended behaviour – lowest performing content, (ranked by likely behaviour shift)

Pieces of communication tested Target behaviour Likely behaviour shift Likely behaviour baseline Past behaviour baseline
Average across pieces - +2 49% 38%
Video, Tyler on TikTok Checking information about food allergies before buying/ consuming food 0 29% 19%
Draft message: CBD 2 Consuming CBD in certain doses/ not consuming it at all -1 6% 5%
Article, cooking frozen raw chicken Being careful when handling and cooking raw meat products -2 82% 77%
Video, Emily on TikTok Checking information about food allergies before buying/ consuming food -3 37% 28%
Draft message: CBD 3 Consuming CBD in certain doses/ not consuming it at all -5 15% 12%

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. 

Table 6.2: Preference, punch and promise scores of each piece of stimulus from Duel

Piece of content Preference scored Punch score Promise score
Stimulus 12 (Message C, Image 3) 129 110 134
Stimulus 4 (Message A, Image 3) 123 109 127
Stimulus 11 (Message C, Image 2) 106 110 113
Stimulus 7 (Message B, Image 2) 104 108 110
Stimulus 6 (Message B, Image 1) 99 114 110
Stimulus 9 (Message C, No image) 117 95 109
Stimulus 5 (Message B, No image) 110 99 106
Stimulus 3 (Message A, Image 2) 101 107 106
Stimulus 8 (Message B, Image 3) 100 106 104
Stimulus 16 (Message D, Image 3) 108 94 100
Stimulus 15 (Message D, Image 2) 88 107 94
Stimulus 10 (Message C, Image 1) 104 83 89
Stimulus 14 (Message D, Image 1) 98 88 88
Stimulus 12 (Message A, Image 1) 95 84 83
Stimulus 13 (Message D, No image) 74 92 74
Stimulus 1 (Message A, No image) 43 96 52