Developing rapid and effective communications testing: Appendices

A. Methodology

Ipsos piloted the following  tools:

• i:Omnibus: Ipsos’ standard surveying approach using our online panel to access of a representative sample of 2,000 adults aged 16-75 in the UK. Fieldwork took place across two waves, the first being 4-7 March 2022 and the second 14-16 March 2022
• fastFacts: This tool is very similar to the i:Omnibus approach, using the same questionnaire. However, it is more flexible, as it is a stand-alone survey as opposed to an Omnibus and provides quicker result to test during a crisis comms, for example. Fieldwork took place on 21 March 2022
• Ipsos DUEL: A standardised survey tool which aims at measuring front of mind preferences between pieces of material and infer what specific words/ messages/ images land best. Participants quickly pick between different stimuli and were asked to select the ones that they found most helpful in knowing when milk is safe to drink. Fieldwork took place 25-26 March 2022. 

B. Communication pieces tested

Through the Creative Testing platform (i:Omnibus and FastFacts) we tested 27 pieces featuring a mix of characteristics, as the table below summarises. 

These pieces of communication can be found by following the URL links in the table below. Where some materials were still only drafts, the exact text tested has been provided instead.

List of communication titles and links to the communication:

• Tweet, Reheating rice safely
• Article, cooking frozen raw chicken (food.gov)
• Tweet, raw eggs
• Tweet, freezer friendly food
• Tweet, wrinkly apples
• Blog, climate change and diet (food.blog.gov)
• Blog, sustainable diets (food.blog.gov)
• Blog, eco-labelling (food.blog.gov)
• Draft message: CBD 1 - "CBD is a novel food which means CBD businesses must apply for authorisation for individual products before they can be authorised. There are currently no CBD products on the market which are authorised novel foods. We have received hundreds of applications and are working to progress these applications through the process."
• Draft message: CBD 2 - "People should think carefully before taking CBD and follow the FSA’s advice about CBD products. Scientific studies suggest CBD can affect the liver if taken at higher doses, but there have been very few studies. As a precaution we recommend healthy adults do not take more than 70mg a day, unless a doctor agrees to more. This is about 28 drops of 5% CBD. FSA will not hestitate to take action if evidence emerges that products are unsafe and put  consumers at great risk."
• Draft message: CBD 3 - "People should think carefully before taking CBD and follow the FSA’s advice about CBD products. As a precaution we do not recommend CBD for people in vulnerable groups, unless under medical direction. These include: pregnant and breastfeeding women and people taking any medication."
• Draft message: CBD 4 - "As THC occurs naturally in CBD and it is difficult to extract completely, it is possible that there could be small amounts of THC in CBD products"
• Article, environmental labelling rules (ITV news)
• Tweet, check FHRS
• Article consuming insects (food.gov.uk)
• Draft message: Milk 1 - “Milk can have either a use-by date or best-before date. You should not use the ‘sniff test’ on milk that has a use-by date. Food can look and smell fine even after the use-by date has passed. You can’t see or smell the bugs that can cause food poisoning.” 
• Draft message: Milk 2 - “Milk can have a use-by date or best-before date. A use-by date is about safety. If the milk has a use-by date, it can be used until the end of this date but not after.”
• Draft message: Milk 3 - “Milk can have a use-by date or best-before date. For milk with a best-before date, you can check the look and smell to decide whether it’s appropriate to drink.”
• Draft message: Milk 4 - “Milk can have either a use-by date or best-before date. A best-before date is about quality, and it is guidance for when the product should be consumed to get the best quality, taste and texture.”
• Video, FHRS explained (food.gov.uk)
• Video, PPDS explained (food.gov.uk)
• Video, New allergen labelling laws
• Video, Tyler on TikTok
• Video, Emily on TikTok
• Video, Jack on allergens
• Video, use by vs best before
• Tweet, pork scratchings recall

Table 2.1 Pieces of communication tested through Ipsos Creative testing tool

Through DUEL we tested 16 combinations of the same message (around milk best before and use-by dates). 

Table 2.2: Different executions of the message tested (16 in totals) 

C. Data tables

Table 1: Overview of how different engagement indicators vary across each other, showing highest, lowest and average.

E7. How well does each of the following statements describe the information we have shown you? Showing highest, lowest, average NET scores (average of those selecting 7-10 on a scale of 0-10 where 0 is not at all and 10 completely), across all 27 pieces of content.

Table 2: Overview of awareness and familiarity for the top/ bottom performing pieces of content (ranked by familiarity shift)

A1/C1. How much, if anything, do you know about the Food Standards Agency, also known as the FSA? Combined figures for awareness are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does, I’ve heard of the FSA but know nothing about it. Combined figures for familiarity are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does. 

Table 3: Overview of Awareness and Familiarity for the bottom performing pieces of content (ranked by familiarity shift)

Table 4: Overview of favourability for the top/bottom performing pieces of content (ranked by highest shift in favourability)

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 5: Overview of favourability for the bottom performing pieces of content (ranked by shift in favourability)

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 6: Overview of trust for the top/bottom performing pieces of content (ranked by shifts in trust) 

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it. 

Table 7: Overview of trust for the bottom five performing pieces of content (rank by shift in trust)

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it.

Table 8: Pieces of communication showing greatest/lowest levels of increase in familiarity with of topic (ranked by shifts in familiarity)

QC5. How much do you feel you know about the following topic? Combined figures for awareness are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it, I’ve heard of it but don’t know much about it, I’ve heard of it but don’t know anything about it. Combined figures for familiarity are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it.

Table 9: Top/bottom performing pieces of content for increasing levels of concern (ranked by shift in concern)

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 10: Pieces of content showing lowest levels of increase in concern (ranked by shift in concern)

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 11: Increases in levels of intended behaviour – top/bottom performing content (ranked by intended behaviour shift)

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. Source: Ipsos’ online panel, first wave fieldwork: 04/03/2022-07/03/2022, second wave fieldwork: 14/03/2022-16/03/2022, FastFacts fieldwork: 21/03/2022. Bases: All UK adults shown one piece of comms (between first wave, n = 2,243 and second wave, n = 2,246; n = 500 for “Pork scratchings recall”).

Table 5.10: Increases in levels of intended behaviour – lowest performing content, (ranked by likely behaviour shift)

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. 

Table 6.2: Preference, punch and promise scores of each piece of stimulus from Duel