Developing rapid and effective communications testing

Background 

The FSA wanted to identify features that make their communications most effective. This report presents the learnings from the pilot, which can be used to aid the development of future communications. 

This report is split into five sections: 

1. Executive summary
2. How the pieces of communication landed: including initial reactions and engagement.
3. Reputation indicators: exploring how exposure to materials impacts awareness, familiarity, favourability, and trust in the FSA.
4. Topic specific indicators: exploring the impact exposure to materials has on people’s familiarity and attitudes towards the specific topics covered by them.
5. Testing different best before/use by dates messages: findings from the AB testing using Ipsos DUEL. 

Methodology 

A total of 43 pieces of communications were tested; 27 existing FSA messages using Ipsos Creative Testing (26 through i:Omnibus and one through FastFacts); and 16 different executions of the same message using Ipsos DUEL. The communications tested covered a mix of topics, formats, objectives, channels and stages of message development (see this report’s annex for exact details). 

Ipsos piloted the following tools: 

• i:Omnibus: Ipsos’ standard surveying approach using our online panel to access of a representative sample of 2,000 adults aged 16-75 in the UK. 
• FastFacts: This tool is very similar to the i:Omnibus approach, using the same questionnaire. However, it is more flexible, as it is a stand-alone survey as opposed to an Omnibus and provides quicker result to test during a crisis comms, for example. 
• Ipsos DUEL: A standardised survey tool which aims at measuring front of mind preferences between pieces of material and infer what specific words/ messages/images land best. Participants quickly pick between different stimuli and were asked to select the ones that they found most helpful in knowing when milk is safe to drink.  

Sample

• testing through i:Omnibus was split between two waves of fieldwork. The first wave achieved a sample of 2,243 adults aged 16-75 in the UK, the second wave achieved a sample of 2,246. These samples were weighted to be representative of the UK population by key demographics. Each piece of material was shown to approximately 170 adults. 
• one piece of content was tested through FastFacts. This survey went to 500 adults in the UK aged 18-65, representative by age, gender and region. 
• 16 pieces of content were tested through Ipsos DUEL. This survey went to 150 in the UK aged 18-65.

Note on interpreting the data 

Points to consider when reading this report:

• all pieces of content were shown to the general public as a whole (as opposed to targeting different demographic groups).
• as this testing took a split sample approach, the sample sizes are small (outlined above). 
• averages presented from the i:Omnibus and FastFacts testing are based across all 27 pieces tested through these methods unless stated otherwise.
• shifts shown through the i:Omnibus and FastFacts testing are calculated by subtracting the pre percentages from the post percentages (for example, percentage points). Participants were asked key metrics before being exposed to the pieces of communication (i.e. “pre measures”) and then again afterwards to measure its impact (“post measures”).
• where differences are highlighted, they have been statistically tested. Where we comment on “shifts” in opinion before and after being shown materials, we have focused on the pieces which saw the largest/smallest shifts for the relevant measure that is being reported. 
• data tables are provided in the appendix. These show the largest/smallest shift for each measure.

Key findings 

Factors most likely to increase likeability and engagement of communication pieces:

1. Focusing on topics on which the FSA is considered a leader (for example, food hygiene, food safety, food borne illnesses as opposed to emerging topics of interest such as novel foods or sustainability), so that the content is more likely to be trusted. 
2. Providing tailored/ targeted messaging to different audiences, so that they feel relevant and important to people.
3. Ensuring an engaging execution, through the use of supporting visuals, images, music, colours so that the material is eye-catching and memorable. 
4. Using supporting information such as signposting to further information, using case studies, or real-life scenarios. 
5. Providing clear messages in plain English and avoiding the use of jargon/ acronyms which are not explained. 
6. Providing a clear call to action and practical tips and advice on what to do.

Key considerations around outcomes: 

1. The role of branding and positioning of the FSA is key (for example, clear referencing through name and logo), as an increase in awareness of the FSA leads to an increase in trust, which is a key factor driving engagement with messages.
2. More research is needed to understand the most effective use of communications to build reputation as a trusted “voice” in topics which are not currently associated with the FSA’s work (such as food sustainability). 
3. The key factors in building awareness and familiarity are clarity of messaging, use of statistics and signposting, and the use of an expert (and trusted) voice, such as from senior staff within the FSA.
4. Pre-existing familiarity with specific topics are the main drivers in shaping attitudes towards that issue. For instance, when addressing topics which people are familiar with but which are often misunderstood (for example, date labels), it is good to ensure the information provided is as clear as possible, and comes across as relevant and memorable in order to effectively raise levels of concern. When designing pieces of communications around topics the public might not be familiar with (for example, novel food), it is important that content is clear, supported by statistics and uses easy-to-follow tips to aid engagement with the issue. 
5. Shifts in familiarity, awareness, attitudes, and behaviours widely depend on how much the public believe they know about the topic in the first place. Communications on topics the public might think they are already familiar with may see drops in familiarity scores once they have seen the piece. This is a positive finding if coupled with an increase in “concern” as people have realised the information provided does not meet their preconceived ideas. Content on topics people don’t know much about initially are likely to achieve the highest increase in familiarity/ awareness, which needs to be coupled with intention to follow the advice provided to translate into a positive outcome. It is therefore important to understand people’s current beliefs around issues in order to address them via communications.
6. People’s behaviours will be impacted by factors beyond levels of engagement with the content. Stated likelihood to engage in behaviours is affected by factors other than the engagement scores for different pieces of content – for instance, an engaging piece of communication does not necessarily lead to a positive shift in intended behaviour. This is key to bear in mind when designing pieces of communications and setting up their objectives – clarity of understanding around the intended behaviour shift is critical. Initial data suggests that when designing pieces of communications aimed at shifting behaviours, it is key to avoid confusion and include a clear call to action, supported by practical tips where appropriate. 

This section focuses on two areas of performance: 

1. Initial reactions to the material shown. After being shown the material, participants were asked a simple and short question on whether they liked the piece or not. 
2. Engagement with the material shown. Participants were asked the extent to which they thought the piece shown could be described as one of the features of “engagement” listed: being eye catching, clear, memorable, interesting, relevant, trustworthy, being for someone like me, telling people something new, carrying an important message. 

Key findings 

Communication which performed well on whether the public liked it and found it engaging tended to have these features: 

• focus on topics which the FSA is considered a leader on: Communications about topics typically linked to the work of the FSA (for example, food safety) performed best on likeability and engagement (on metrics such as relevance, importance); whilst pieces about topics less strongly linked to the FSA performed less well (for example, novel food and sustainable food)
• tailored/ targeted messaging: Communications which felt relevant to people and carrying an important message also tended to score higher on likeability. These features were above average amongst the top engaging pieces. Whereas, being boring and perceived as irrelevant were mentioned as features in the least liked pieces tested
• engaging execution: The use of supporting visuals, and the fact pieces were considered eye-catching and memorable were among the most liked and engaging. Music and colour were also mentioned as something participants liked, as these were attention grabbing and encouraged engagement
• use of supporting information: Providing signposting to further information had a positive bearing on likeability, as well as the use of case studies and real-life scenarios
• clear messages: Content delivered in plain English with an easy-to-understand message landed well generally and rated highly on clarity 
• clear call to action: Providing practical advice and a clear call to action were key to likeability and engagement. This is a key consideration for pieces which participants described in their owns words as "overwhelming" and "scary" (the CBD draft messages) as they would have appreciated knowing what they were supposed to do about the issue presented
• use of a positive tone: Using an upbeat, friendly, and light-hearted tone was seen positively. Whereas pieces with overwhelming information were seen as off-putting.  

Initial reactions to the pieces tested 

This section looks at factors affecting likeability from three different sources:

1. The features of the different communications (format, topic, use of supportive images/ stats/ case studies etc).
2. Engagement indicators (were they considered eye catching, trustworthy, relevant etc).
3. Spontaneous comments from the open-ended questions. The figure below provides an overview of the most and least liked pieces of communication tested. 

Figure: Overview of likeability for the top and bottom performing pieces of communication (%)  

E5 Overall, to what extent do you like or dislike this information? Show combined percentages of those who select 1-2 (“Liked it very much” and “Liked it somewhat”) on a scale of 1-5 where 1 is liked it very much and 5 disliked it very much. Please see methodology chapter for fieldwork dates and sample sizes. 

Most liked pieces of communication 

The top liked pieces of communication were the:

• video on use by vs best before
• tweet on reheating rice safely
• tweet on wrinkly apples
• article on cooking frozen raw chicken
• tweet on checking FHRS
• video on FHRS

They all had the following features in common: 

• they were about safe eating (foodborne illnesses, eating safely at home; with the wrinkly apples tweet being about a food waste), one of the topics the FSA is best known for.
• they made use of supporting visuals (either images or videos). 
• they all provided practical tips and advice (with the exception of the video on FHRS) and had signposting to further information. 
• they were mostly delivered in plain English. 
• the engagement scores consistently featured amongst the most liked pieces included feeling relevant, being eye-catching and being perceived as trustworthy. Clarity also featured higher than average.

Looking at the attributes which participants spontaneously identified as features they liked (through the open-ended questions) ^{(footnote 1)},  some key themes emerged: 

• being clear, simple, easy to understand and well-paced: Participants liked this as it allowed them to digest and understand the information.
• being upbeat, friendly, and light hearted: This made information seem more engaging and accessible, without scaring participants by being too serious.
• using music and colour: This was seen to be engaging and helped capture people’s attention.
• being seen as important, informative, and telling something people didn’t know: Participants liked instructions which might change their behaviour as it gave the communication purpose.

Below are some examples of what participants said they liked about the highest performing pieces of content in their own words:

“I liked how it was easy to follow and understand. It explained the difference between best before and use by which people get confused by. Very informative.”
Video, Use by vs. best before

“It was something important that I didn’t know about. It offered solutions. I liked the graphics. The information was very clearly presented with good use of colour.”
Tweet, Reheating rice safely

“It isn’t scolding or emotionally blackmailing people to address an issue. It offers advice on how we can take specific action to minimise waste.” 
Tweet, Wrinkly apples

“I liked the way it was explained clearly, precisely and efficiently without being long and boring with too much unnecessary detail.”
Article, Cooking frozen chicken 

Least liked pieces of communication 

Most of the least liked materials were draft messages relating to CBD (scoring between 18%-34%). In comparison, the draft messages relating to milk received between 45%-53% of participants saying they liked the content, suggesting their “draft format” was not the only reason they scored low. The article on consuming insects and the blog on sustainable diets were the least liked pieces of published (for example, not draft) content.

Common features amongst the bottom performing pieces included: 

• topics currently perceived outside the FSA’s main remit for example, novel foods and food sustainability
• they did not use case studies or practical tips. The published pieces were reporting findings from surveys conducted by the FSA
• With the exception of the blog on sustainable diets, the other pieces did not have any supporting visuals (and on the blog this was a generic image of food)
• with the exception of the blog on sustainable diets, the other pieces of content were not written in plain English (for example, the acronym CBD was not explained) 
• all of these pieces scored lower than average on engagement scores, especially on being perceived as relevant/ for people like me, eye-catching and trustworthy.

When prompted to say what they did not like about these pieces, participants spontaneously mentioned the following ^{(footnote 2)}:  

• being boring, long and feeling irrelevant: The published pieces were articles from the FSA’s website providing survey findings and participants questioned their relevance to them.
• not trusting the accuracy of the information: Participants questioned whether the information provided was correct as it was not in line with their personal experiences. For example, some participants questioned the validity of the milk messages, claiming they have consumed milk past its use-by date in the past without issue. 
• the information not been presented in an engaging and eye-catching way: Some participants critiqued the way the information was presented, for example the use of certain images and/or presenters, and this was not only the draft pieces of content which had not been properly designed but also finalised pieces which had been published. 
• lack of clarity: Participants disliked acronyms which were not explained (for example, CBD and THC) as it was difficult to understand. The better performing CBD Draft 3 message had a clear warning and remained concise ^{(footnote 3)}.   
• overwhelming information: CBD Draft 2 differed in style by including statistics. It received better scores for being clear and telling people something new; however, some felt that the information was ‘scary’ and ‘too much’.

Below are some examples of what participants said they disliked about the lowest performing pieces of content in their own words: 

“There’s too much jargon. It needs to be presented in plain English. Too many acronyms. Hard to understand. Not helpful.”
Draft message: CBD 4

“The subject matter itself is very important but the presentation is dry – wordy and inaccessible to probably the majority of the public through the way it has been written.”
Blog, Climate change and diet 

“I always give milk an extra few days if it looks and smells ok. It has never done me any harm. Seems wasteful to tell people this.”
Draft message: Milk 2 

How the pieces tested engaged the audience 

Across all tested materials, on average participants gave the highest scores for content carrying an important message (60%) and being clear (58%). Being seen as trustworthy (52%) and relevant (50%) also scored higher than average. Whereas the tested materials had the lowest scores for making people emotional (14%), being eye catching (30%) and memorable (34%). Being eye-catching and being perceived as relevant were key factors driving likeability on both published and draft pieces, so focusing on making the content more targeted and eye-catching could be a consideration for the FSA comms team moving forward when testing near finalised content.

Looking at characteristics that featured in the most engaging pieces of content and bottom engaging pieces4,  similar patterns as observed with “likeability” emerged. The top performing pieces tending to focus on the topic of food safety, being written in plain English and providing practical advice.

These measures covered key factors to track the FSA’s reputation, which we explore in this chapter. They were: 

1. awareness
2. familiarity
3. favourability 
4. trust 

Key findings 

• overall, there were no clear patterns revealing key features which improve reputation. This could be because only a few pieces tested were designed to increase the reputation of the FSA. These four pieces were about food sustainability – an emerging topic of interest for the organisation.
• there was a link between all the reputation scores (Awareness, Familiarity, Favourability and Trust). The role of branding and positioning here is key, as familiarity with the FSA positively impacted Favourability and Trust. Further research would be needed to understand the right balance between “too much branding” and “too little branding” when designing communications. 
• communication about topics typically associated with the FSA’s work were more likely to positively shift reputation scores; whereas those around topics not strongly associated with the FSA (for example, food sustainability) were less likely to build reputation. More research is needed to understand how the FSA can most effectively use communications to position itself as a trusted “voice” in emerging debates.  

Awareness and Familiarity 

Before being shown any communication material, the majority (84%) of respondents said they had heard of the Food Standards Agency ^{(footnote 1)}.  However, the proportion who were fairly or very familiar with the FSA (saying they know at least a little about what it does) was significantly smaller (54%). This meant that there was greater scope to increase levels of familiarity than to increase levels of awareness given its already high baseline level: on average, there was just a +1-percentage point shift in awareness, compared to a +8-point shift in familiarity. 

Content most likely to improve familiarity with the FSA included: the video explaining FHRS, draft messages around milk (use by dates / sniff test), the article providing precautionary advice on cooking frozen raw breaded chicken products, the tweet around reheating rice safely, and the video outlining differences between best before and use by dates. In terms of content scoring particularly well or poorly, some patterns were observed: 

• topics typically associated with the FSA (for example, food safety) positively increased Familiarity and Awareness scores. This is likely due to the strong association between the concepts covered in these materials and the role of the FSA.
• broadly speaking, the most liked pieces of communication were also the most effective at increasing Familiarity and Awareness. Although, draft milk messages (use by dates/ sniff test) were not amongst the most liked pieces, but still effective in shifting reputation scores. 
• there was some correlation with increasing Familiarity and Awareness and specific engagement scores, particularly clarity, carrying an important messaging and being for somebody like me. However, this was not consistent (for example, the Milk Draft messages did not score well on engagement). 

In contrast, three of the CBD draft messages, the tweet around wrinkly apples, article on consuming insects and the video exploring what it means for food to be pre-packed for direct sale (PPDS) scored poorly on Awareness and Familiarity metrics:

• in the case of the PPDS video, this likely reflects the businesses-focussed messaging as it was tested with the public instead of food industry stakeholders. 
• the messages around CBD did not score well on engagement metrics, perhaps reflecting lack of awareness and interest in the topic. 

Favourability 

Before being shown any material, three in five (62%) were favourable towards the FSA (amongst those who have at least heard of the organisation). There was a +4-point average uplift in levels of favourability towards the FSA once exposed to the communication material.

There was variation in the levels of change pre-post exposure of different content, ranging from +30 to -12. The top performing pieces of content increasing favourability towards the FSA were:

• the video explaining FHRS
• the article on cooking frozen raw chicken 
• the tweet on reheating rice safely
• the Tyler West Tik Tok video on allergy awareness

Again, these appeared to perform well on the broader reputational measures through a combination of engaging content and focus on topics which people are likely to associate with the FSA.

The pieces of material tested that were less likely to improve Favourability (in fact, they slightly decreased it but all not significantly) were:

• the blog on climate change and diet
• CBD draft message 4
• the article on environmental labelling rules
• one of the draft messages on milk labelling message 4
• the Jack video about allergies

There were no consistent features that seemed to drive favourability down. However, most of these pieces (with the exception of the video explaining FHRS) did not use plain English and they scored lower than average on clarity, trustworthiness and memorability. 

Notably, the stakeholder update ‘How climate change will impact on diet and what is the regulatory responsibility?’ had the highest negative impact on favourability towards the FSA (and also on trust, as described in the next section). This is reflected in the low clarity and memorability engagement scores, and also potentially lack of strong association between FSA and sustainability.  

Trust

The proportion who said they trust the FSA to do its job rose 3-points from 69% to 72% upon exposure to the communications within the survey ^{(footnote 2)}. This average uplift masks significant variations across different materials tested, which ranged from +24 to -9.

The materials which scored highest and lowest in terms trust tended to be the same as those in similar positions within metrics on familiarity and/or favourability: the article on cooking frozen raw chicken and the video on use by vs best before dates were particularly successful in instilling trust in the FSA. 

In contrast, the stakeholder update “How climate change will impact on diet and what is the regulatory responsibility?” and the article on environmental labelling rules - "Enormous sense of urgency: FSA calls for new environmental labelling rules”, appeared to slightly reduce levels of trust amongst those exposed to these materials. Both pieces aimed at increasing awareness of the topic of sustainable food, corroborating the earlier finding that suggests it could be challenging for the FSA to create a reputational impact when addressing new topics it is not known for.

This included measures on:

1. awareness and familiarity with topics
2. concern about the topics
3. stated and likely future behaviours in relation to the topics

A summary of the range of topics addressed by the tested materials is in the appendix, alongside a table indicating which pieces included an element of encouraging behaviour change. 

Key findings

• it is important to be clear about the objectives the piece of communication was designed to address. For instance, a decline in familiarity can be a good thing if it means the piece of communication has made people realise that they didn’t know as much as they previously thought, which is a positive outcome if accompanied by an increase in “concern” in the issue. Similarly, a decrease in levels of concern might be positive, if the messaging is aimed at providing reassurance about something that may be concerning the public. 

• key factors in increasing familiarity with a topic are the use of plain English, the support of videos and the use of signposting. The use of expert voices, making sure the content is not too short and that it is perceived as relevant are also factors to consider. 
• pre-existing levels of familiarity and concern about specific topics are the main drivers in shaping attitudes towards an issue. It is important to consider different strategies to “raise concern” based on different topics:  (1) When designing communications about topics widely talked about, but which are likely to be tainted by misconceptions (for example, use-by vs best before dates), it is important to ensure the information provided is as clear as possible, and it comes across as relevant and memorable; (2) When designing pieces of communications around topics the public might not be too familiar with (for example, novel food), messages should be clear, supported by statistics and use easy-to-follow advice. 
• people’s behaviours will be influenced by a broad range of factors depending on the individual, broader contextual considerations and the behaviour in question. Stated likelihood to engage in behaviours may be influenced by communications, though will be affected by factors beyond the engagement scores for different pieces of content – an engaging piece of communication will not necessarily lead to a positive shift in intended behaviour. There is merit in considering the COM-B framework when designing pieces of communications, to set clear expectations around what behavioural changes to expect. Initial data suggests that when designing pieces of communications aimed at shifting behaviours, it is key to avoid confusion and include a clear call to action, supported by practical tips where appropriate. 

Awareness of and Familiarity with topics 

The top performing pieces in shifting familiarity with their specific topics were the Pre-Packed for Direct Sale (PPDS) explained video, the new allergen labelling laws video, the article on frozen chicken, the article about consuming insects and the FHRS explained video, which had features in common ^{(footnote 1)}:  

• some of them were videos posted on YouTube as part of the “FSA Explains” series, which were created the purpose of using plain English to raise awareness of topics people might initially find confusing. 
• all included the voice of an FSA expert suggesting they are important in distilling complex information, and they provided signposting for further information about the topic.
• most scored high on “carries an important message” (ranging from 71% to 78% vs. an average of 60%) and trustworthiness (ranging from 67% to 71% versus average of 52%). 

The bottom performing pieces when it came to shifting familiarity with certain topics were three out of the four draft pieces on Milk messages and the Tweet on raw eggs, which have key features in common: 

• they all focused on broad topics with which the public were likely to be familiar: date labelling, food waste and food hygiene. A reduction in familiarity may reflect realisation amongst some respondents that they had not known as much as they initially thought about this topic
• they were short messages, which might not be enough space to fully explain detail around a topic, especially as these messages tended to challenge behaviours consumers might already be doing (sniff testing milk, cooking with raw eggs, freezing food)
• furthermore, none of these pieces used experts and only one provided signposting. This in comparison to the better performing pieces which took a more considered approach in video format with FSA experts
• all performed below average on the engagement metric measuring whether it “is for someone like me” (ranging from 28% on the raw eggs tweet to 44% vs. an average of 50%), suggesting targeting audiences is important.  

Levels of concern around topics 

The biggest increases in concern were around the broader topics of food hygiene at home (+10-ppt increase in personal concern) and date labels (+12-ppt). In line with this, the top performing pieces on shifting concern about the topic addressed were the video on use by vs best before, and two of the draft Milk messages (Milk 3 and 2). 

Pre-existing levels of concerns with the topic and familiarity with it seemed to be the main drivers in making people more/less concerned about the issues after seeing the material. 

• date labels is a topic participants thought they were familiar with (84% vs an average of 54%); and materials addressing this topic, on average, decreased familiarity (-6ppt). However, they all pieces on this topic performed high on the concern indicators. This increase in concern strengthens the suggestion that people engaged effectively with the messaging, as it has prompted them to re-think how much they knew about date labels – highlighting the need to be cautious in interpreting decreases in perceived familiarity, and to look at shifts in combination with other measures. They all had in common the fact that they used plain English, that they were slightly more likely than average to be considered “memorable” and to be “for somebody like me”. 
• the draft message CBD 3 and the article on environmental labelling rules also performed well in increasing levels of concerns – even if they were about relatively new topics the public are not too familiar with (levels of knowledge before we showed the material were 30% and 52% respectively vs an average across pieces of 54%). They both had in common the use of statistics. 

Several pieces led to very slight decreases in levels of concern. These were the tweet on freezer-friendly food; the video explaining the FHRS; Draft message CBD 4 and 1; the tweet on checking FHRS. 

Two of the lowest performing pieces in shifting levels of concern were draft CBD messages and linked to the novel foods topic. The lack of clarity and potential confusion around these messages (which included acronyms many may not be familiar with) likely explains why levels of concern decreased. Furthermore, the main difference between CBD draft messages 1 and 4 which did not increase levels of concern; and CBD draft message 2 which was successful at doing so was that the latter provided tips/advice on what to do next. An easy-to-follow call to action is key to “balance” feelings of concerns and translate them into positive engagement with the piece of communication. 

Stated and likely behaviours

The communications focusing on encouraging behaviour change achieved a small uplift of +2ppts in intended behaviour from the baseline measures. The pieces achieving the highest uplifts were the tweet around reheating rice safely, the video featuring Jack with an allergy to eggs which encouraged people to speak up for allergies when ordering a takeaway. Both included a short video element (max 1 minute), used plain English and provided a clear call to action and practical tips. 

With regards to the pieces which had the least impact on intended behaviours, there were a few which led to a very slight decrease in intended action: two of the four draft messages on CBD (2 and 3), the Emily Video on TikTok (focusing on food allergies), the articles on frozen raw chicken, the Tyler video on TikTok (also about food allergies). They had certain elements in common: 

• two of the draft CBD messages scored amongst the lowest in changing behaviour, again likely reflecting the general confusion and lack of positive engagement with these draft messages
• the Emily on TikTok video also scored lower, contrasting against the Jack on TikTok video covering the same topic and scored highly on changing behaviours. The former lacks the same levels of clarity around the call to action, with the Jack video benefiting by having greater focus on a specific, relatable scenario (ordering a takeaway), whilst Emily’s video focused on a viral social media game not everyone might be familiar with. 

We tested 16 different executions of messaging around the difference between a use-by date and best-before date on milk, using Ipsos DUEL. Early iterations of four messages were created explaining the difference between a use-by-date versus a best-before date. Messages each used a different image, and one did not have one. Messages were pitted against each other, and participants were asked to select which one they preferred, and the selected iteration progressed to the next round. Full details of these messages can be found in the appendix to this report, alongside a table showing full results from the testing. 

Key findings

• the use of images makes pieces of communications more likeable
• the real-life picture of milk was liked the most, reinforcing the hypothesis emerging from the literature review that “real-life” situations are a key factor for successful communication
• top performing stimuli provided an instruction for the public to act upon, or a call to action (reflecting previous findings presented in this report)
• it is difficult to distinguish between whether focusing on the Best-before or Use-by date makes a difference, although on the whole messages referencing use-by date performed slightly better. The message on Use-by date (without the sniff test) was more likely to be considered by “relevant” than others, perhaps because consumers are instinctively more concerned about “safety” than “quality”. 

Findings

The pieces of content with the highest which performed best were Stimulus 12 and Stimulus 4:

• both pieces of content use a real-life image of milk (Image 3)
• stimulus 12, which wins overall, focuses on the best-before date (Message C) and Stimulus 4 focuses on the use-by date (Message A). Both messages are instructional, providing the reader with an action (i.e., Stimulus 12 says “you can check the look and smell”, Stimulus 4 says “you should not use the smell test”). This is in comparison to the other two messages which are more descriptive. 

When looking specifically at how different variations of messaging performed, we could observe some trends: 

• overall, messages focusing solely on the Use-by date (Messages C and B) tended to perform better
• whilst Message A with the real-life image of milk was among the top two (message focusing on use-by date, with advice on the “sniff test”), when accompanied with other images or no images at all it did less well. This suggest more unpicking might be needed to understand the relationship between this specific message and the different pictures presented in the iterations tested. 

When looking at the use of images specifically, we observed the following patterns: 

• the bottom two performing Stimuli (13, 1) both used no images, highlighting the importance of a visual aid
• stimulus 13, a bottom performing iteration, also uses Message D, which focused on the Best-before date and does perform well across the board (the highest score it achieves is 100, and this is with the top performing image)
• similarly, iterations which used the cartoon image of milk (Image 1) tended to perform less, suggesting it is important to use real life scenarios. 

In the survey we also asked about specific relevance and differentiation. On average, two in five (44%) thought the stimuli shown were relevant. “Differentiation” is not a key engagement indicator for the FSA, and this is reflected by the relatively lower average scores for differentiation (32% average). 

• when it comes to the relevance score, there were no clear patters amongst the bottom scoring stimuli. However, Messages B (around use-by date, without sniff test) seemed to perform better than average (at 53% for stimulus 6, 52% for stimulus 8 and 49% for stimulus 7 vs an average of 44%). This reinforces the findings above, which is messages focusing on use-by dates might resonate better with the public
• when it comes to differentiation score, there were no clear patterns. However, three out of the three stimuli without pictures (Stimuli 9, 5, 13) performed slightly worse, reinforcing the point that imaged help materials stand out. 

A. Methodology

Ipsos piloted the following  tools:

• i:Omnibus: Ipsos’ standard surveying approach using our online panel to access of a representative sample of 2,000 adults aged 16-75 in the UK. Fieldwork took place across two waves, the first being 4-7 March 2022 and the second 14-16 March 2022
• fastFacts: This tool is very similar to the i:Omnibus approach, using the same questionnaire. However, it is more flexible, as it is a stand-alone survey as opposed to an Omnibus and provides quicker result to test during a crisis comms, for example. Fieldwork took place on 21 March 2022
• Ipsos DUEL: A standardised survey tool which aims at measuring front of mind preferences between pieces of material and infer what specific words/ messages/ images land best. Participants quickly pick between different stimuli and were asked to select the ones that they found most helpful in knowing when milk is safe to drink. Fieldwork took place 25-26 March 2022. 

B. Communication pieces tested

Through the Creative Testing platform (i:Omnibus and FastFacts) we tested 27 pieces featuring a mix of characteristics, as the table below summarises. 

These pieces of communication can be found by following the URL links in the table below. Where some materials were still only drafts, the exact text tested has been provided instead.

List of communication titles and links to the communication:

• Tweet, Reheating rice safely
• Article, cooking frozen raw chicken (food.gov)
• Tweet, raw eggs
• Tweet, freezer friendly food
• Tweet, wrinkly apples
• Blog, climate change and diet (food.blog.gov)
• Blog, sustainable diets (food.blog.gov)
• Blog, eco-labelling (food.blog.gov)
• Draft message: CBD 1 - "CBD is a novel food which means CBD businesses must apply for authorisation for individual products before they can be authorised. There are currently no CBD products on the market which are authorised novel foods. We have received hundreds of applications and are working to progress these applications through the process."
• Draft message: CBD 2 - "People should think carefully before taking CBD and follow the FSA’s advice about CBD products. Scientific studies suggest CBD can affect the liver if taken at higher doses, but there have been very few studies. As a precaution we recommend healthy adults do not take more than 70mg a day, unless a doctor agrees to more. This is about 28 drops of 5% CBD. FSA will not hestitate to take action if evidence emerges that products are unsafe and put  consumers at great risk."
• Draft message: CBD 3 - "People should think carefully before taking CBD and follow the FSA’s advice about CBD products. As a precaution we do not recommend CBD for people in vulnerable groups, unless under medical direction. These include: pregnant and breastfeeding women and people taking any medication."
• Draft message: CBD 4 - "As THC occurs naturally in CBD and it is difficult to extract completely, it is possible that there could be small amounts of THC in CBD products"
• Article, environmental labelling rules (ITV news)
• Tweet, check FHRS
• Article consuming insects (food.gov.uk)
• Draft message: Milk 1 - “Milk can have either a use-by date or best-before date. You should not use the ‘sniff test’ on milk that has a use-by date. Food can look and smell fine even after the use-by date has passed. You can’t see or smell the bugs that can cause food poisoning.” 
• Draft message: Milk 2 - “Milk can have a use-by date or best-before date. A use-by date is about safety. If the milk has a use-by date, it can be used until the end of this date but not after.”
• Draft message: Milk 3 - “Milk can have a use-by date or best-before date. For milk with a best-before date, you can check the look and smell to decide whether it’s appropriate to drink.”
• Draft message: Milk 4 - “Milk can have either a use-by date or best-before date. A best-before date is about quality, and it is guidance for when the product should be consumed to get the best quality, taste and texture.”
• Video, FHRS explained (food.gov.uk)
• Video, PPDS explained (food.gov.uk)
• Video, New allergen labelling laws
• Video, Tyler on TikTok
• Video, Emily on TikTok
• Video, Jack on allergens
• Video, use by vs best before
• Tweet, pork scratchings recall

Table 2.1 Pieces of communication tested through Ipsos Creative testing tool

Through DUEL we tested 16 combinations of the same message (around milk best before and use-by dates). 

Table 2.2: Different executions of the message tested (16 in totals) 

C. Data tables

Table 1: Overview of how different engagement indicators vary across each other, showing highest, lowest and average.

E7. How well does each of the following statements describe the information we have shown you? Showing highest, lowest, average NET scores (average of those selecting 7-10 on a scale of 0-10 where 0 is not at all and 10 completely), across all 27 pieces of content.

Table 2: Overview of awareness and familiarity for the top/ bottom performing pieces of content (ranked by familiarity shift)

A1/C1. How much, if anything, do you know about the Food Standards Agency, also known as the FSA? Combined figures for awareness are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does, I’ve heard of the FSA but know nothing about it. Combined figures for familiarity are: I know a lot about the FSA and what it does, I know a little about the FSA and what it does. 

Table 3: Overview of Awareness and Familiarity for the bottom performing pieces of content (ranked by familiarity shift)

Table 4: Overview of favourability for the top/bottom performing pieces of content (ranked by highest shift in favourability)

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 5: Overview of favourability for the bottom performing pieces of content (ranked by shift in favourability)

A2/C2 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How favourable or unfavourable do you feel towards the Food Standards Agency (FSA)? Combined figures for favourability are: very favourable + somewhat favourable. 

Table 6: Overview of trust for the top/bottom performing pieces of content (ranked by shifts in trust) 

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it. 

Table 7: Overview of trust for the bottom five performing pieces of content (rank by shift in trust)

A3/C3 The Food Standards Agency (FSA) is the Government Agency responsible for food safety in England, Wales and Northern Ireland. How much do you trust or distrust the Food Standards Agency to do its job? That is to make sure that food is safe and what it says it is. Combined figures for trust are: I trust it a lot + I trust it.

Table 8: Pieces of communication showing greatest/lowest levels of increase in familiarity with of topic (ranked by shifts in familiarity)

QC5. How much do you feel you know about the following topic? Combined figures for awareness are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it, I’ve heard of it but don’t know much about it, I’ve heard of it but don’t know anything about it. Combined figures for familiarity are: I’ve heard of it and know quite a lot about it, I’ve heard of it and know a bit about it.

Table 9: Top/bottom performing pieces of content for increasing levels of concern (ranked by shift in concern)

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 10: Pieces of content showing lowest levels of increase in concern (ranked by shift in concern)

C6. To what extent, if at all, are you personally concerned about the following issue? Combined figures for concern are: Very concerned, fairly concerned. 

Table 11: Increases in levels of intended behaviour – top/bottom performing content (ranked by intended behaviour shift)

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. Source: Ipsos’ online panel, first wave fieldwork: 04/03/2022-07/03/2022, second wave fieldwork: 14/03/2022-16/03/2022, FastFacts fieldwork: 21/03/2022. Bases: All UK adults shown one piece of comms (between first wave, n = 2,243 and second wave, n = 2,246; n = 500 for “Pork scratchings recall”).

Table 5.10: Increases in levels of intended behaviour – lowest performing content, (ranked by likely behaviour shift)

QC7. How often, if at all, do you do the following…? QC8. And thinking about the next few months or so, how likely or unlikely are you to do the following…? Combined figures for likely behaviour are: Very likely, Somewhat likely. Combined figures for past behaviour are: Always, Most of the time. 

Table 6.2: Preference, punch and promise scores of each piece of stimulus from Duel