Developing rapid and effective communications testing: topic specific indicators

This included measures on:

1. awareness and familiarity with topics
2. concern about the topics
3. stated and likely future behaviours in relation to the topics

A summary of the range of topics addressed by the tested materials is in the appendix, alongside a table indicating which pieces included an element of encouraging behaviour change. 

Key findings

• it is important to be clear about the objectives the piece of communication was designed to address. For instance, a decline in familiarity can be a good thing if it means the piece of communication has made people realise that they didn’t know as much as they previously thought, which is a positive outcome if accompanied by an increase in “concern” in the issue. Similarly, a decrease in levels of concern might be positive, if the messaging is aimed at providing reassurance about something that may be concerning the public. 

• key factors in increasing familiarity with a topic are the use of plain English, the support of videos and the use of signposting. The use of expert voices, making sure the content is not too short and that it is perceived as relevant are also factors to consider. 
• pre-existing levels of familiarity and concern about specific topics are the main drivers in shaping attitudes towards an issue. It is important to consider different strategies to “raise concern” based on different topics:  (1) When designing communications about topics widely talked about, but which are likely to be tainted by misconceptions (for example, use-by vs best before dates), it is important to ensure the information provided is as clear as possible, and it comes across as relevant and memorable; (2) When designing pieces of communications around topics the public might not be too familiar with (for example, novel food), messages should be clear, supported by statistics and use easy-to-follow advice. 
• people’s behaviours will be influenced by a broad range of factors depending on the individual, broader contextual considerations and the behaviour in question. Stated likelihood to engage in behaviours may be influenced by communications, though will be affected by factors beyond the engagement scores for different pieces of content – an engaging piece of communication will not necessarily lead to a positive shift in intended behaviour. There is merit in considering the COM-B framework when designing pieces of communications, to set clear expectations around what behavioural changes to expect. Initial data suggests that when designing pieces of communications aimed at shifting behaviours, it is key to avoid confusion and include a clear call to action, supported by practical tips where appropriate. 

Awareness of and Familiarity with topics 

The top performing pieces in shifting familiarity with their specific topics were the Pre-Packed for Direct Sale (PPDS) explained video, the new allergen labelling laws video, the article on frozen chicken, the article about consuming insects and the FHRS explained video, which had features in common ^{(footnote 1)}:  

• some of them were videos posted on YouTube as part of the “FSA Explains” series, which were created the purpose of using plain English to raise awareness of topics people might initially find confusing. 
• all included the voice of an FSA expert suggesting they are important in distilling complex information, and they provided signposting for further information about the topic.
• most scored high on “carries an important message” (ranging from 71% to 78% vs. an average of 60%) and trustworthiness (ranging from 67% to 71% versus average of 52%). 

The bottom performing pieces when it came to shifting familiarity with certain topics were three out of the four draft pieces on Milk messages and the Tweet on raw eggs, which have key features in common: 

• they all focused on broad topics with which the public were likely to be familiar: date labelling, food waste and food hygiene. A reduction in familiarity may reflect realisation amongst some respondents that they had not known as much as they initially thought about this topic
• they were short messages, which might not be enough space to fully explain detail around a topic, especially as these messages tended to challenge behaviours consumers might already be doing (sniff testing milk, cooking with raw eggs, freezing food)
• furthermore, none of these pieces used experts and only one provided signposting. This in comparison to the better performing pieces which took a more considered approach in video format with FSA experts
• all performed below average on the engagement metric measuring whether it “is for someone like me” (ranging from 28% on the raw eggs tweet to 44% vs. an average of 50%), suggesting targeting audiences is important.  

Levels of concern around topics 

The biggest increases in concern were around the broader topics of food hygiene at home (+10-ppt increase in personal concern) and date labels (+12-ppt). In line with this, the top performing pieces on shifting concern about the topic addressed were the video on use by vs best before, and two of the draft Milk messages (Milk 3 and 2). 

Pre-existing levels of concerns with the topic and familiarity with it seemed to be the main drivers in making people more/less concerned about the issues after seeing the material. 

• date labels is a topic participants thought they were familiar with (84% vs an average of 54%); and materials addressing this topic, on average, decreased familiarity (-6ppt). However, they all pieces on this topic performed high on the concern indicators. This increase in concern strengthens the suggestion that people engaged effectively with the messaging, as it has prompted them to re-think how much they knew about date labels – highlighting the need to be cautious in interpreting decreases in perceived familiarity, and to look at shifts in combination with other measures. They all had in common the fact that they used plain English, that they were slightly more likely than average to be considered “memorable” and to be “for somebody like me”. 
• the draft message CBD 3 and the article on environmental labelling rules also performed well in increasing levels of concerns – even if they were about relatively new topics the public are not too familiar with (levels of knowledge before we showed the material were 30% and 52% respectively vs an average across pieces of 54%). They both had in common the use of statistics. 

Several pieces led to very slight decreases in levels of concern. These were the tweet on freezer-friendly food; the video explaining the FHRS; Draft message CBD 4 and 1; the tweet on checking FHRS. 

Two of the lowest performing pieces in shifting levels of concern were draft CBD messages and linked to the novel foods topic. The lack of clarity and potential confusion around these messages (which included acronyms many may not be familiar with) likely explains why levels of concern decreased. Furthermore, the main difference between CBD draft messages 1 and 4 which did not increase levels of concern; and CBD draft message 2 which was successful at doing so was that the latter provided tips/advice on what to do next. An easy-to-follow call to action is key to “balance” feelings of concerns and translate them into positive engagement with the piece of communication. 

Stated and likely behaviours

The communications focusing on encouraging behaviour change achieved a small uplift of +2ppts in intended behaviour from the baseline measures. The pieces achieving the highest uplifts were the tweet around reheating rice safely, the video featuring Jack with an allergy to eggs which encouraged people to speak up for allergies when ordering a takeaway. Both included a short video element (max 1 minute), used plain English and provided a clear call to action and practical tips. 

With regards to the pieces which had the least impact on intended behaviours, there were a few which led to a very slight decrease in intended action: two of the four draft messages on CBD (2 and 3), the Emily Video on TikTok (focusing on food allergies), the articles on frozen raw chicken, the Tyler video on TikTok (also about food allergies). They had certain elements in common: 

• two of the draft CBD messages scored amongst the lowest in changing behaviour, again likely reflecting the general confusion and lack of positive engagement with these draft messages
• the Emily on TikTok video also scored lower, contrasting against the Jack on TikTok video covering the same topic and scored highly on changing behaviours. The former lacks the same levels of clarity around the call to action, with the Jack video benefiting by having greater focus on a specific, relatable scenario (ordering a takeaway), whilst Emily’s video focused on a viral social media game not everyone might be familiar with.