Developing rapid and effective communications testing: reactions to communications tested

This section focuses on two areas of performance: 

1. Initial reactions to the material shown. After being shown the material, participants were asked a simple and short question on whether they liked the piece or not. 
2. Engagement with the material shown. Participants were asked the extent to which they thought the piece shown could be described as one of the features of “engagement” listed: being eye catching, clear, memorable, interesting, relevant, trustworthy, being for someone like me, telling people something new, carrying an important message. 

Key findings 

Communication which performed well on whether the public liked it and found it engaging tended to have these features: 

• focus on topics which the FSA is considered a leader on: Communications about topics typically linked to the work of the FSA (for example, food safety) performed best on likeability and engagement (on metrics such as relevance, importance); whilst pieces about topics less strongly linked to the FSA performed less well (for example, novel food and sustainable food)
• tailored/ targeted messaging: Communications which felt relevant to people and carrying an important message also tended to score higher on likeability. These features were above average amongst the top engaging pieces. Whereas, being boring and perceived as irrelevant were mentioned as features in the least liked pieces tested
• engaging execution: The use of supporting visuals, and the fact pieces were considered eye-catching and memorable were among the most liked and engaging. Music and colour were also mentioned as something participants liked, as these were attention grabbing and encouraged engagement
• use of supporting information: Providing signposting to further information had a positive bearing on likeability, as well as the use of case studies and real-life scenarios
• clear messages: Content delivered in plain English with an easy-to-understand message landed well generally and rated highly on clarity 
• clear call to action: Providing practical advice and a clear call to action were key to likeability and engagement. This is a key consideration for pieces which participants described in their owns words as "overwhelming" and "scary" (the CBD draft messages) as they would have appreciated knowing what they were supposed to do about the issue presented
• use of a positive tone: Using an upbeat, friendly, and light-hearted tone was seen positively. Whereas pieces with overwhelming information were seen as off-putting.  

Initial reactions to the pieces tested 

This section looks at factors affecting likeability from three different sources:

1. The features of the different communications (format, topic, use of supportive images/ stats/ case studies etc).
2. Engagement indicators (were they considered eye catching, trustworthy, relevant etc).
3. Spontaneous comments from the open-ended questions. The figure below provides an overview of the most and least liked pieces of communication tested. 

Figure: Overview of likeability for the top and bottom performing pieces of communication (%)  

E5 Overall, to what extent do you like or dislike this information? Show combined percentages of those who select 1-2 (“Liked it very much” and “Liked it somewhat”) on a scale of 1-5 where 1 is liked it very much and 5 disliked it very much. Please see methodology chapter for fieldwork dates and sample sizes. 

Most liked pieces of communication 

The top liked pieces of communication were the:

• video on use by vs best before
• tweet on reheating rice safely
• tweet on wrinkly apples
• article on cooking frozen raw chicken
• tweet on checking FHRS
• video on FHRS

They all had the following features in common: 

• they were about safe eating (foodborne illnesses, eating safely at home; with the wrinkly apples tweet being about a food waste), one of the topics the FSA is best known for.
• they made use of supporting visuals (either images or videos). 
• they all provided practical tips and advice (with the exception of the video on FHRS) and had signposting to further information. 
• they were mostly delivered in plain English. 
• the engagement scores consistently featured amongst the most liked pieces included feeling relevant, being eye-catching and being perceived as trustworthy. Clarity also featured higher than average.

Looking at the attributes which participants spontaneously identified as features they liked (through the open-ended questions) ^{(footnote 1)},  some key themes emerged: 

• being clear, simple, easy to understand and well-paced: Participants liked this as it allowed them to digest and understand the information.
• being upbeat, friendly, and light hearted: This made information seem more engaging and accessible, without scaring participants by being too serious.
• using music and colour: This was seen to be engaging and helped capture people’s attention.
• being seen as important, informative, and telling something people didn’t know: Participants liked instructions which might change their behaviour as it gave the communication purpose.

Below are some examples of what participants said they liked about the highest performing pieces of content in their own words:

“I liked how it was easy to follow and understand. It explained the difference between best before and use by which people get confused by. Very informative.”
Video, Use by vs. best before

“It was something important that I didn’t know about. It offered solutions. I liked the graphics. The information was very clearly presented with good use of colour.”
Tweet, Reheating rice safely

“It isn’t scolding or emotionally blackmailing people to address an issue. It offers advice on how we can take specific action to minimise waste.” 
Tweet, Wrinkly apples

“I liked the way it was explained clearly, precisely and efficiently without being long and boring with too much unnecessary detail.”
Article, Cooking frozen chicken 

Least liked pieces of communication 

Most of the least liked materials were draft messages relating to CBD (scoring between 18%-34%). In comparison, the draft messages relating to milk received between 45%-53% of participants saying they liked the content, suggesting their “draft format” was not the only reason they scored low. The article on consuming insects and the blog on sustainable diets were the least liked pieces of published (for example, not draft) content.

Common features amongst the bottom performing pieces included: 

• topics currently perceived outside the FSA’s main remit for example, novel foods and food sustainability
• they did not use case studies or practical tips. The published pieces were reporting findings from surveys conducted by the FSA
• With the exception of the blog on sustainable diets, the other pieces did not have any supporting visuals (and on the blog this was a generic image of food)
• with the exception of the blog on sustainable diets, the other pieces of content were not written in plain English (for example, the acronym CBD was not explained) 
• all of these pieces scored lower than average on engagement scores, especially on being perceived as relevant/ for people like me, eye-catching and trustworthy.

When prompted to say what they did not like about these pieces, participants spontaneously mentioned the following ^{(footnote 2)}:  

• being boring, long and feeling irrelevant: The published pieces were articles from the FSA’s website providing survey findings and participants questioned their relevance to them.
• not trusting the accuracy of the information: Participants questioned whether the information provided was correct as it was not in line with their personal experiences. For example, some participants questioned the validity of the milk messages, claiming they have consumed milk past its use-by date in the past without issue. 
• the information not been presented in an engaging and eye-catching way: Some participants critiqued the way the information was presented, for example the use of certain images and/or presenters, and this was not only the draft pieces of content which had not been properly designed but also finalised pieces which had been published. 
• lack of clarity: Participants disliked acronyms which were not explained (for example, CBD and THC) as it was difficult to understand. The better performing CBD Draft 3 message had a clear warning and remained concise ^{(footnote 3)}.   
• overwhelming information: CBD Draft 2 differed in style by including statistics. It received better scores for being clear and telling people something new; however, some felt that the information was ‘scary’ and ‘too much’.

Below are some examples of what participants said they disliked about the lowest performing pieces of content in their own words: 

“There’s too much jargon. It needs to be presented in plain English. Too many acronyms. Hard to understand. Not helpful.”
Draft message: CBD 4

“The subject matter itself is very important but the presentation is dry – wordy and inaccessible to probably the majority of the public through the way it has been written.”
Blog, Climate change and diet 

“I always give milk an extra few days if it looks and smells ok. It has never done me any harm. Seems wasteful to tell people this.”
Draft message: Milk 2 

How the pieces tested engaged the audience 

Across all tested materials, on average participants gave the highest scores for content carrying an important message (60%) and being clear (58%). Being seen as trustworthy (52%) and relevant (50%) also scored higher than average. Whereas the tested materials had the lowest scores for making people emotional (14%), being eye catching (30%) and memorable (34%). Being eye-catching and being perceived as relevant were key factors driving likeability on both published and draft pieces, so focusing on making the content more targeted and eye-catching could be a consideration for the FSA comms team moving forward when testing near finalised content.

Looking at characteristics that featured in the most engaging pieces of content and bottom engaging pieces4,  similar patterns as observed with “likeability” emerged. The top performing pieces tending to focus on the topic of food safety, being written in plain English and providing practical advice.