Evaluation of Food Standards pilot: Findings

It provides an overview of the baseline findings, focusing on the key factors that influenced the progress of the pilot project, and answers the evaluation questions through analysis of the data and evidence gathered from interviews.

2.1 Baseline

A baseline round of LA interviews was conducted in early 2021 after the Pilot had begun. The purpose of baseline interviews was to compare the changes between the baseline and the end of the pilot, as well as to observe whether the control LAs evolved in a different manner to the pilot LAs. 

Normally, baseline interviews would be conducted before the pilot project starts. However, the study team was not able to interview the LAs prior to the start date due to COVID-19 restrictions and busy LA workloads prior to the Christmas period. The evaluation team, together with FSA, decided to carry out the baseline interviews two weeks after the pilot project had started. It was understood that, at that point in time, officers would not have received any training and would just be familiarising themselves with the supporting documentation provided by FSA. During the interviews, the evaluation team made clear the purpose of the interview and explained what it aimed to capture. 

Each LA was different in terms of levels of available resource, the systems and processes used to deliver food standards controls, their geographical location (e.g. rural or urban) and the overall business profile registered (e.g. manufacturing companies, restaurants or supermarkets). The baseline highlighted how these different attributes may impact the LA’s implementation of the proposed model. 

2.1.1 LA resources

Participating LAs varied in relation to their available staffing resource. Key variations were identified regarding the following:

• number of authorised food standards officers employed;
• experience of officers within teams;
• team capacity;
• officer absence, either planned (maternity, retirement) or unplanned.

LAs stated that staff were typically experienced and enthusiastic about their job. However, staff resource was a commonly reported issue as well as the lack of new professionals entering the workspace. 

LAs mentioned that the higher the number of dedicated staff, the easier it would be for them to carry out their food standards work. In addition, some LAs reported experiencing a high turnover of staff. Prior to the start of the pilot, it was expected that those LAs with more experienced team members and greater available resources would be able to adapt and use the proposed model more successfully.

2.1.2 Geographical location 

The location of a LA can lead to operational differences. LAs highlighted the following considerations: 

• the size of the LA area can affect the amount of travel time it takes for officers to conduct physical inspections. For example, officers in rural areas may be required to spend more time travelling to deliver interventions compared to urban areas. 
• LAs with a particular local interest in the Geographical Indication  scheme or products claiming to be ‘locally produced’ can experience a higher risk of food fraud compared to other areas.
• the type/s of business registered with the LA can pose different risks, with some being more complex than others. For example, some areas have a greater proportion of large manufacturers registered than others, which can have an impact on regulatory delivery. 

The implications of this in relation to the proposed model could include a greater need for resource in some LAs compared to others, and the need to ensure the new risk scheme can identify all types of risk (including those specific to local areas). These variations could also mean that the use of Targeted Remote Interventions (TRIs) gives greater benefit to LAs covering large geographical areas, compared to LAs covering smaller, densely populated areas. 

2.1.3 Type of local authority

As highlighted in section 1.3 above, the pilot included LAs of different types to ensure suitable representation.

In the pilot there were: 

• 4 Unitary Authorities implementing the pilot and 3 as control;
• 3 County Councils implementing the pilot and 1 as control.

The variation in LA type can affect different elements of the proposed model. For example, some of the LAs in the pilot reported that they did not have access to the Trading Standards Intelligence Database (IDB). This could have an impact on the LAs ability to share intelligence with FSA. IDB is a database where LA officers can upload intelligence and access intelligence submitted by others. FSA has access to the database, providing oversight of intelligence recorded in the system at a national level.

The type of authority will also have geographical implications, as discussed above.

2.1.4 Management Information system (MIS)

Participating LAs used different Management Information Systems (MIS) to record their activity, and capture and record data in different ways. Some LAs reported concerns around data quality and accuracy in relation to registered food business data held on their MIS. For example, some food establishments had duplicate records, or showed as active despite having ceased to trade. This posed further challenges during COVID-19, as LAs experienced a significant increase in the number of food businesses registering. 

2.1.5 Use of intelligence

LAs were already using intelligence to guide their work to different extents. In terms of intelligence gathering and sharing, some LAs reported strong relationships with regional liaison groups, FSA, and strong relationships across environmental health and trading standards teams, whilst others did not have such relationships in place. As mentioned above, some LAs had access to IDB while others did not. Prior to the pilot, participating LAs had different ways of recording and sharing intelligence, information, and data. 

Existing practices for sharing intelligence by LAs

LAs used a range of networks to share or receive intelligence, including other LAs, regional/in-house intelligence officers, internal colleagues, FSA, other agencies, sampling data, regional and national food standard groups, HMRC, the police, public analysts, and the Knowledge Hub (footnote 1).

Food standards intelligence has been inconsistently uploaded on the National Trading Standards IDB database. LAs had different perceptions on uploading information to IDB, with some LAs not uploading information at all, and others uploading frequently. 

The type of information that was uploaded to IDB also differed among LAs, with some only uploading verified information that they perceived to be a national consideration. Other LAs uploaded information that they perceived could contribute to developing a national picture of an emerging risk, and recorded intelligence more frequently. 

Due to the differing levels of maturity in terms of operating an intelligence-led approach at a local level within LA regulatory services, during the pilot some LAs were better able to adapt and use the intelligence function of the proposed model than others.

2.1.6 Ways of working

Most of the LAs interviewed were not following the Food Law Code of Practice (FLCoP) in a consistent manner to guide their work. The LAs explained that they were already working in similar ways to the changes suggested by the proposed model and had already adapted their approach to delivery to be able to target their resources more effectively. For example, some LAs stated that they would prioritise interventions at all high-risk establishments identified under the FLCoP intervention rating scheme (or equivalent), and target remaining resources based on a consideration of national, regional and/or local intelligence. 

As discussed in section 1.2.2.5, LAs were already taking different approaches to food standards sampling activity. Some LAs were participating in sampling programmes set by regional groups, or used intelligence to guide their local sampling, while others undertook sampling on a reactive basis as a result of information received, such as consumer complaints. 

2.2 External factors

Several external factors impacted the pilot, with COVID-19 being the most significant. LA resources were affected differently by the impacts of COVID-19, however most LAs experienced staff reallocation due to the need to carry out COVID-19 related tasks. 

Some LAs reported a large increase in new food businesses registering during COVID-19, with some of these businesses being based in domestic settings. LAs struggled to keep their databases up to date during this time and struggled to assess the risk profile of some of these new businesses in a timely manner.  

The various impacts of COVID-19 hampered the ability of pilot LAs to fully embrace the proposed model and had an adverse effect on FSA’s ability to collect evaluation data due to the enforced changes in LA workload during this time. To mitigate these impacts, FSA extended the pilot for 3 months to enable additional evaluation data to be collected, with the pilot ending in March 2022 rather than the original date of December 2021.  

In terms of the directed sampling programme, the pilot allowed LAs to choose whether to get involved in the three phases. Some LAs struggled to participate, particularly during the first phase, for several reasons. These included the impacts of COVID-19 (the redeployment of LA food officers to public health work) or, in some cases, due to officers not being able to visit their offices, meaning they were not able to access internal systems necessary to support the sampling (for example, to make online purchases or to take receipt of samples). 

2.3 How did the proposed model perform compared to the current framework – what worked well and less well?

Overall, the model operated well, with evaluation of the pilot indicating that: 

• the proposed model was appropriate and fit for purpose. The proposed risk scheme was suitable for assessing and identifying risk and prioritising LA resources. 
• the proposed model was easy to use and could be easily integrated into existing working practices.
• the proposed model introduced a single, uniform risk rating scheme to assess the risk profile of FBOs more accurately. It was dynamic and integrated intelligence for a more targeted and efficient response, enabling LAs to focus their resources on food businesses presenting the greatest risk. 
• LAs viewed the directed sampling approach as being effective at identifying non-compliance and providing useful intelligence.

On the other hand, the intelligence element of the proposed model requires further consideration:                         

The intelligence element was not fully developed by the end of the pilot. 

• at the end of the pilot, there was no consistent approach to what information FSA and LAs considered to be intelligence; the type of information which should be shared between LAs and FSA; the frequency of intelligence sharing and the most suitable mechanism to share intelligence. 
• LAs found it challenging to carry out all the directed sampling activities as they felt these were not planned far enough in advance. 

In terms of the implementation of the pilot: 

• the support provided by FSA throughout the pilot was well received and crucial to the success of the pilot. the main challenges to the first steps during the implementation of the pilot were the lack of clarity with some terms used in the risk scheme, the lack of consistency of data held by LAs (both issues addressed during the pilot), and the lack of compatibility of the LAs MIS for the purposes of the pilot. It should be noted that LAs’ MIS integration was not part of the pilot to using the proposed model.     

This section presents the findings relevant to how well the proposed operating model worked. It assesses whether the model was fit for purpose, its ease of use and any challenges identified in using the proposed model. This section also analyses the context in which the pilot was delivered, assessing the enabling factors and barriers faced during the implementation of the pilot project itself. The effects of the proposed model, namely whether it achieved the outputs and outcomes as set out by the intervention logic (see Annex 2) are discussed in section 2.5 of this report. 

2.3.1 Appropriateness of the proposed model (fit for purpose)

The proposed model tested during the pilot sought to address the shortcomings identified with the current FSDM.

LAs participating in the pilot agreed that the new risk scheme was fit for purpose and more appropriate at identifying risk than the previous approach they were using. LAs felt the three elements of the proposed model worked well, providing a more realistic and up to date assessment of the risk posed by businesses and the associated frequency of official controls. This provided greater flexibility, allowing LAs to further target their resources more effectively. As such the proposed model addressed the challenges identified in Table 1.1 Each of the three elements of the proposed model (see section 1.2.2) are assessed below:

2.3.1.1 The proposed risk scheme

The proposed risk scheme was very well received by LAs overall. A positive change when compared to the existing model was the fact that LAs had experienced difficulties in effectively assessing all business types under the current framework. For example, LAs explained that there had been an increase in home catering businesses, but that these did not easily fit in with the existing scoring system, whereas the proposed risk scheme allowed LAs to score them more accurately. 

Also, previously large manufacturers were identified as high risk, regardless of levels of previous compliance due to the categorisations in the FLCoP. However, these types of businesses (where premises are often part of a larger company who conduct internal audits and have specific processes to follow) were identified as being less of a concern for officers.  Officers highlighted the benefit of using their knowledge of a manufacturers’ previous performance and management to assess the risk posed by these businesses, and for this to inform the intervention frequency. As such, under the proposed model this type of business would now be more likely to be considered lower risk if it can demonstrate high levels of compliance with suitable management systems in place. LAs appreciated being able to use their judgement to reflect the risks and prioritise resources, as this allowed them to target their resources towards businesses presenting a higher risk, rather than focusing on the same businesses for annual interventions.

LAs reported that the proposed model allows for more flexibility than the existing one, enabling them to effectively target resources to higher risk establishments. This is because the proposed model combines the assessment of risk and compliance of FBOs, giving increased emphasis to the level of compliance within a business rather than focusing on inherent risk, allowing for more granularity in the risk assessment process. 

In terms of implementation of the pilot, LAs identified initial issues relating to the conversion of their food business risk data to the new risk scheme. This was because not all LA databases were up to date, and the resulting lack of data quality negatively impacted the conversion process in some cases. These issues were addressed together with FSA during the pilot.

LAs mentioned that the identification of non-compliances linked to allergens had been difficult for them using the proposed model. By the end of the pilot, LA officers became more familiar with scoring allergens under the proposed risk scheme. 

“I think part of the issue [with allergens] was that we were not scoring them quite correctly. So like allergens for us, because it’s a safety issue, we’d always revisit and I think some officers were sort of scoring that as a three, and then when we went through it with FSA it was actually a one because its safety. So I think that was maybe a bit of an issue with officers not scoring them harshly enough.” – Unitary Authority

LAs believed businesses with allergen risks should be targeted with a physical intervention, as these are more effective at identifying non-compliance, particularly in takeaways or restaurants. These types of businesses were perceived as being less suitable for a TRI. One LA suggested that the new risk scheme could be improved in relation to allergens by developing the descriptors included the risk rating scheme. Another LA suggested that the proposed risk scheme identified establishments that are more likely to have issues across their business, rather than identifying and giving a suitable level of emphasis to specific allergen issues.

2.3.1.2 The decision matrix

LAs highlighted that the decision matrix allowed risks to be assessed and balanced across different types of businesses. Further, it included multiple criteria (the inherent risk profile and the compliance assessment risk sub-categories), allowing for a more accurate assessment of the risk posed by an FBO and a more appropriate frequency of official controls based on the associated level of risk. 

LAs agreed that the decision matrix was easy to understand. They liked the ability to rescore premises after a re-visit, allowing a change in the risk rating. For example, a LA mentioned that previously, if a non-compliance was identified at a Category A establishment, continued re-visits were required to the premises until compliance was achieved, but the officers were not able to change the risk score to reflect this. The proposed model enables rescoring to take place in such instances. If the officer is satisfied that the non-compliance/s have been resolved and the officer has confidence in the FBO, premises can more easily switch risk ratings, thereby impacting the date of next intervention.  

As previously mentioned, several LAs highlighted that a key benefit of the proposed risk scheme and decision matrix was that manufacturers were no longer considered high risk by default. This was perceived as a benefit by LAs who felt that TRIs could be an appropriate approach for some manufacturers, as they typically have strict systems and procedures in place and they could usually supply copies of relevant documentation, internal procedures, and product labelling etc. to enable their levels of risk and compliance to be assessed remotely. 

2.3.1.3 Intelligence-led approach and directed sampling

The proposed model encourages LAs to collect, share and action intelligence to inform the delivery of official controls. It defines intelligence as an all-encompassing term and aims to encourage intelligence-led working. LAs recognise that intelligence is important and has the potential to help identify risks and target work. This flexibility in working and conducting interventions is a key feature of the proposed model, and LAs reported that officers positively perceived using intelligence to guide their work as the pilot continued. At the end of the pilot, LAs reported becoming more familiar with using intelligence in accordance with the proposed model, and as a result were uploading information to IDB more frequently. 

Prior to the pilot, most LAs reported using intelligence to guide their work to some extent but noted that intelligence was infrequently and inconsistently received. Challenges relating to the use and receipt of intelligence persisted throughout the pilot, but a general positive trend was reported through increased recognition of the role of intelligence. 

LAs noted that, during the pilot, they had received more intelligence from FSA, and they recognised the potential benefits of working in a more intelligence-led way, rather than organising their work based on a list of programmed interventions identified by the FLCoP. 

LAs reported that officers were increasingly considering intelligence received to determine the relevant priority of their workload. One LA suggested that intelligence recorded by officers and shared with FSA should be prioritised and assessed against multiple sources of data to ensure that officer resource is targeted at areas that present the greatest risk. They also noted that acting on early intelligence received by FSA could limit the risk posed to consumers. LAs mentioned further that, if no risk was found after an initial investigation, LAs could communicate their findings to FSA to gather insight from other authorities. 

However, as discussed later in (see section 2.3.3), LAs still had an inconsistent approach to using intelligence by the end of the pilot. Challenges identified during the pilot, regarding inputting information onto IDB and strengthening relationships with external agencies, persisted throughout the pilot, and were still present at the end of the pilot. This was explained partially by the fact that some LA officers were more familiar with the type of information to share, and the mechanisms to do so than others. 

The pilot addressed these challenges to some extent, and there was an improvement in the way LAs using the proposed model were using intelligence and facilitating communication channels from LAs to FSA particularly. Once the pilot concluded, FSA recognised the challenges associated with embedding this way of working across all LAs and established a team to lead these changes and address the challenges.

Part of the pilot was to test the proposed directed sampling programme, to ensure that sampling was co-ordinated and effective, and resulted in measurable corrective action by LAs where appropriate. This intelligence-led programme introduced a more targeted approach to sampling, supported by the identification of risk-based national food standards sampling priorities based on an assessment of available intelligence and other data.

Overall, LAs viewed sampling as being effective at identifying non-compliance and gathering intelligence. One LA highlighted that they had a large sampling programme and that participating in FSA’s directed sampling did not cause them significant issues.

“FSA directed some of the samples that we were routinely going to take anyway so we took less of our own samples and substituted those with samples that FSA would take” – Unitary Authority

LAs did not identify testing the directed sampling approach as challenging. However, they reported practical challenges related to planning the implementation of the sampling (for example, locating all necessary food products) and the availability and capacity of the Public Analysts (as discussed in the intelligence section).

2.3.2 Ease of use of the proposed model

LAs participating in the pilot perceived the new risk scheme to be easy to understand and felt that it allowed them to prioritise interventions in a more effective way than before. LAs reported that the flexibility of the new model enabled them to effectively target resources to higher risk FBOs. However, there were mixed perspectives regarding the role of allergens and the use of TRIs as part of the new model. 

Prior to the pilot, LAs had used the FLCoP to prioritise their resources on an annual basis. In some cases, LAs used local, regional and/or national intelligence to help target their resource. However, in baseline interviews, some LAs reported that they were not able to complete all scheduled interventions required under the FLCoP due to a lack of capacity. These LAs noted that before the pilot, they would prioritise completing high risk (Category A) and medium risk (Category B) interventions. 

The flexibility in the proposed risk scheme allowed officers to make an informed decision on the appropriate intervention frequency for an establishment based on their informed assessment of risk, which considered the internal processes and management of the business. This was identified as being a positive feature because under the current FLCoP, officers highlighted that if they acted on intelligence, it would not be recognised as part of their annual report on official control delivery.

Overall, the proposed model was viewed as being compatible with the way that LAs perceived food standards should operate, as it identified the major risks, and for some the proposed model fitted with how they were already working. The ability for LAs to act on intelligence received was viewed as a key strength of the proposed model, because it enabled them to justify the redirection of resources as necessary, rather than strictly following the FLCoP. 

2.3.2.1 Targeted Remote Interventions

LAs had mixed perceptions on the benefit of TRIs. Typically, TRIs were viewed as being appropriate for certain business types, such as manufacturers or microbreweries, where it was typically felt that the FBO would be able to provide the required information to an officer quickly and easily. Some LAs highlighted that smaller businesses may not always have the information to hand, and that it can take a lot of time for officers to receive and assess the information. 

For establishments that were viewed as presenting a higher risk for allergens, such as takeaways and restaurants, a TRI was typically not viewed as suitable. LAs identified that a physical inspection is more appropriate when conducting a proper allergen assessment because officers can verify information that is provided by the FBO. The verification of information was identified as a weakness of TRIs, and some LAs highlighted that FBOs could provide information that would likely negate the need for a physical visit to intentionally mislead officers. Officers also typically provide advice and assistance to FBOs when they conduct a physical intervention, which is viewed as being a valuable means to increase compliance.

“[The FBO] is going to send you data or information or specifications but then they might have something to hide. They might still hide it when you’re there for an intervention but at least you can open cupboards and have a good look round and see their menus.” – Unitary Authority

“[TRIs] work well for manufacturers, especially manufacturers you’ve been to before where you don’t necessarily need to do a site visit. So, if you can look at product specs and labelling and stuff, the TRIs work well for that because you can get them to email it to you and have a chat… it doesn’t work for pubs, shops, that sort of thing because you need to go and see them.” – County Authority 

Some LAs also reported a difference in officers’ commitment to TRIs. Some officers were engaged with the TRI approach and reported positive experiences in conducting them, whereas other officers reportedly were not as engaged. 

One LA highlighted they were not considering TRIs because they had a significant number of establishments that are categorised as needing an annual intervention. This LA also highlighted that, for weights and measure inspections, a TRI would be unsuitable, so they would carry out a physical intervention whilst doing the weights and measures inspection. 

One LA highlighted that they had historically sent out questionnaires for their low-risk establishments and conducted a physical intervention on a 10% sample of these responses to identify how truthful FBOs were in their answers. This LA highlighted that they would decide to conduct a physical intervention at an establishment that was identified as being suitable for a TRI if they received intelligence. 

2.3.3 Use of intelligence

The main challenge identified with the proposed model related to the intelligence element. While improvements were made through the duration of the pilot, there remain considerable challenges. 

The proposed intelligence-led approach features a feedback loop whereby LAs share data or information with FSA, who then collect and analyse the information before disseminating relevant intelligence back to LAs so it can drive their work (see section 1.2.2.3). The proposed model aims to produce strategic assessments informed by intelligence, information and data shared by LAs and other sources. This will allow FSA to set priorities and intelligence requirements for LAs to assist in directing their interventions and intelligence gathering activities. 

Prior to the pilot, engagement indicated that most LAs were already using intelligence to inform their delivery of food standards official controls. It was anticipated that the implementation of the intelligence-led approach to food standards delivery would be less complex than it became.

During the pilot, FSA team identified that each pilot LA understood and worked with intelligence very differently, meaning the assumed baseline understanding was not consistent. The team also recognised a need for FSA to review and adapt its internal processes regarding sharing and receiving intelligence to properly support the new model. These were unexpected barriers that led to a delay in integrating an intelligence-led approach to the delivery of food standards. This work is still ongoing and essential to supporting LAs with the integration of intelligence-led working. Further work is required to continue to understand the range of processes and knowledge of intelligence across England, Wales, and Northern Ireland. 

2.3.3.1 Challenges on what type of intelligence is shared

LAs interpreted intelligence as being verified information of local, regional or national relevance that has an evidence base and includes specific details. In terms of definitions of intelligence, some LAs mentioned specific features of what could be classed as intelligence including: it should be corroborated somehow, it should lead them to carry out some sort of action or to target resources better, it relates to something that is potentially illegal or fraudulent. Some LAs perceived a difference between "intelligence" and "intelligence that would be put on IDB" and would only share information to IDB if they perceived it to be relevant beyond their local area and serious enough to warrant notification, such as cases involving fraud or risk of harm. 

LAs reported being familiar with receiving intelligence and working with other agencies, and they welcomed the intelligence-led approach under the proposed model. Typically, LAs use formal and informal sources of data to guide their work, though ‘formal’ intelligence received through IDB or trading standards groups was reported as being more reliable than complaints (considered ‘informal’). LAs noted that, before and during the pilot, they did not receive intelligence frequently from external agencies. Most of their daily work was guided based on local intelligence, such as officer knowledge or consumer complaints, but not on national or regional intelligence. This was also reflected by a control LA, who reported that intelligence received from other enforcement agencies can be vague, which is unhelpful in guiding their work. 

The ground level intelligence gathered by officers and received through public complaints was identified as being essential in identifying local issues. Some LAs suggested that it would be detrimental to uncovering food standards issues if there was an overreliance on intelligence that is communicated through formal agencies. 
LAs also mentioned they were familiar with working with regional groups, FSA (including the NFCU), and wider agencies such as Port Health Authorities and the Police. LAs often only share data they consider ‘exceptional’ information. One pilot LA highlighted that they would only share ‘important’ data with other bodies such as the police and HMRC, and that most of the intelligence that guided their work would be considered ‘mundane’ by other agencies. Usually, routine intelligence provided in a complaint would not be communicated to other agencies or uploaded to the IDB database. The type of information that LAs would consider intelligence includes regional or national issues, such as counterfeit spirits or product labelling issues. LAs highlighted that if they identified a high-risk issue that could harm consumers, they would communicate this information straight to FSA. 

This way of working appeared to be a relatively standard practice, however LAs were not sure whether, due to the proposed model, they should now report more intelligence through IDB, or whether local intelligence should be shared with FSA (via IDB or some other channel). 

Further, LAs suggested that the information received via other mechanisms (not collected by the LA itself) often was not enough to guide their food standards work. This could be because other agencies (e.g. NFCU) collect very specific types of intelligence, which LAs don’t always find useful. During the pilot, LAs reported sharing intelligence more frequently and reliably than before the pilot, particularly with FSA. However, LAs mentioned that consumer complaints remained the largest source of intelligence they use, and that they did not really receive much information from external agencies. There seemed to be disconnect between LAs and FSA regarding the interpretation of intelligence, which persisted in all rounds of interviews. 

At the end of the pilot, one LA reported that their officers were considering information that they would have previously regarded as being a local isolated issue as potentially providing intelligence to more LAs and informing FSA. However, the findings suggest that LAs could benefit from intelligence shared by other LAs or agencies. There was not a consistent approach agreed by the end of the pilot on the type of data that needed to be captured and shared. 

2.3.3.2 Challenges on the frequency that intelligence is shared

The other difficulty was the lack of agreement on how often data should be shared. 

LAs record complaints and share data in different ways, with some of them recording every type of information as soon as they receive it and sharing it with others, while other LAs do not always consider the information they gather as intelligence, leading to data only being shared with FSA sporadically. One LA highlighted they would not record every complaint received on IDB but they would only do so if multiple complaints were received. One LA highlighted that there was no available resource to dedicate officer time to uploading information on to IDB so they would not share data, with two other LAs reporting that officers inconsistently upload information. Across LAs, not every complaint is actioned, and officers use their professional judgement to decide whether further investigation is needed. Allergens were a priority for all LAs and non-compliance would always lead to further investigation. 

Some LAs agreed that receiving intelligence about an issue at an early stage could lead to risks being identified and addressed earlier on a national scale. LAs also suggested that intelligence sharing would be more effective if there were routine collaborative efforts made to coordinate the intelligence collected by LAs and other partners. 

2.3.3.3 Challenges with the mechanisms to share intelligence 

There was no standard method for gathering and sharing intelligence among the LAs in the pilot project. At the same time, FSA was yet to develop an effective mechanism to receive intelligence from LAs in a way that would facilitate rapid processing and analysis to provide timely feedback to the LAs. 

Most LAs viewed regional food liaison groups as being a useful forum to share intelligence, and it was noted that Trading Standards are a well-connected professional community. Groups that include wider agencies were identified as being a less useful source of intelligence. LAs perceived that some agencies were reluctant to share information with them. LAs mentioned using regional groups to share data. This could mean that LAs and FSA could strengthen intelligence networks to enable and facilitate communication between partners.

In addition, some LAs use databases to share intelligence. One control LA reported that all complaints and intelligence are recorded on their MIS, whereas a LA testing the proposed model reported scoring intelligence, received in the form of complaints, based on their own internal intelligence matrix, which provided criteria that could trigger an intervention if needed.

All LAs with Trading Standards functions and with access to IDB shared and recorded intelligence using the database. However, not all LAs use it in the same manner, leading to potential inconsistencies in the way that LAs share intelligence with other agencies. During the pilot, LAs were encouraged to use IDB more frequently for food intelligence and to input more information than they did prior to the pilot. Consistent challenges were reported throughout the pilot related to IDB. Two LAs identified that IDB does not provide any feedback to officers or provide any information as to how their intelligence is used. These LAs suggested that if this additional feature was included, officers may be more proactive in uploading information to IDB.

Challenges with using IDB and inputting information remained at the end of the pilot. Officers have a limited amount of time available to input information and there were different perspectives regarding the usefulness of IDB, with LAs typically reporting that it is less appropriate for recording food standards information. However, throughout the pilot, in general LAs have increased how frequently they upload information to IDB. It was suggested that if LAs received recognition for contributing intelligence to the database, it may encourage officers to upload information more consistently. 

“[IDB] is based on the police model, so it’s not really ideal for food, which is the first problem with it. Its more based on people whereas food is more based on products, so the system is about identifying people and activity and linking people together, which is great if you’re looking at car crime but with food it’s much more about the product, so it doesn’t lend itself to the other issues, so we don’t tend to put food information on because its data entry twice” – County Council

“we’ve increased our usage [of IDB] a bit but I think officers still struggle with the idea that IDB is an appropriate place for food” – Unitary Authority

Finally, some LAs mentioned using the online platform Knowledge Hub and local complaints as informal channels of communication to guide their work. LAs also reported having communication with the police, other local authority departments and the public on an informal frequent basis. 

2.3.3.4 Challenges of communicating outside FSA

Food is not one of the National Trading Standards (NTS) priorities. The NTS priorities include fair trading, illegal tobacco, intellectual property, and marketing fraud. As food issues are not considered in the decision making that informs these priorities, food standards teams reported feeling overlooked in this process. This can be demotivating for officers who upload food intelligence frequently and consistently to IDB because they do not receive recognition from other agencies for their contributions. LAs suggested that promoting their food work and improving relationships with other agencies, could lead to a mutually beneficial relationship, where food standards concerns would be more respected. 

The challenges related to the sharing and use of intelligence across organisations are beyond the scope of the proposed model but are key considerations in the practicalities of the effectiveness of intelligence. 

2.3.4 Implementation of the pilot

2.3.4.1 Enabling elements

FSA support was the key enabling element in the success of the pilot. The support can be divided into informal and formal support, as discussed below:  

The support received from FSA before and during the pilot was reportedly very useful to LAs. The pilot opened a line of informal communication between FSA and the LAs, where they could collaborate to solve issues together. 

FSA were identified as being responsive to communication. LAs felt they had a ‘point of contact’ where their queries would be addressed or signposted within FSA to the relevant department if the query was not related to the pilot. Most LAs highlighted that they always had a positive relationship with FSA but FSA’s receptiveness during the pilot had helped to strengthen relationships. 

‘I’ve certainly got to know a few more officers over there [at FSA] and I feel like they’ve always been very helpful when I’ve had a query and very approachable and so hopefully that will continue following the pilot’ – Unitary Authority 

FSA helped build confidence across LAs and helped embed the proposed model into the LAs’ working practices, which is a positive indication on the success of the pilot. Overall, LAs perceived FSA as being proactive and receptive to their ideas throughout the pilot, and they hoped that the communication and engagement would continue. 

In addition, the collaborative process adopted to develop the risk assessment scheme and the decision matrix was appreciated by LAs and FSA. 

LAs also reported sharing learning with their teams. In particular some of the LAs participating in the pilot highlighted they were sharing learning as part of the regional groups they participate in. Some LAs reported conducting their own in-house consistency exercises with officers to help provide further reassurance. 

A key enabling element for the success of the pilot was effective regular communication between LAs and FSA, as there needed to be a common interpretation of the risk scheme and decision matrix. Prior to the pilot, FSA carried out a series of training activities to introduce the aspects of the proposed model. FSA provided specific training on intelligence and on the proposed risk assessment model. LAs appreciated the training provided and found it valuable in preparing them for the pilot. 

Some LAs which were already familiar with using intelligence reported that the training on the use of the risk assessment scheme and decision matrix was more useful than intelligence training, as intelligence was already well understood by them. 

In addition, FSA’s National Food Crime Unit (NFCU) delivered intelligence training to the pilot LAs as part of a wider training programme to aid consistency. Feedback from this training indicated that officers would appreciate a greater focus on applying intelligence in a food-related context, with specific examples related to their areas of work, as this would help officers apply the training in their day-to-day activities. 

FSA delivered two consistency exercises in relation to the new risk assessment scheme. These helped reassure officers that they were scoring correctly by comparing food businesses that presented different risks and explaining the scoring criteria. The first session was delivered in the early stages of the pilot (January 2021) to refine some of the scoring criteria and provide officers with a greater understanding of each element of the risk factor scoring criteria. This clarity was very useful to officers. LAs also reported carrying out this type of consistency exercise with officers in their internal team meetings, where they would discuss scoring and share ideas with each other, helping to build on the consistency exercises hosted by FSA. LAs perceived that these consistency exercises would be useful in the early stages of a potential national roll out of the programme. 

“Each officer who has gone through that consistency exercise comes out with roughly the same scoring and so I think the information that’s supplied or been amended throughout the pilot has helped enormously with the consistency of the scheme and the scoring system” – County Council

Pilot LAs mentioned, however, that the second session (‘Risk Assessment Scheme Consistency exercise’ February 2022) was delivered slightly late in the pilot, and that it would have been more beneficial to have had this session earlier in the pilot. 

2.3.4.2 Barriers to implementation 

Most of the challenges identified during the interviews were related to practical issues experienced by LAs in integrating the proposed way of working with their current practices for the purposes of the pilot. The main challenges were: 

Clarity on terminology

LAs identified that the description of some terms in the proposed model were not always clear. Two LAs identified that the risk scheme could be improved by increasing the clarity of definitions. A specific example was provided in relation to the definition of a ‘single local supplier’ within the ‘Complexity of Supply Chain’ risk factor, and where they procure their stock. The LA stated that the risk scheme considers where an FBO purchases their food. Whilst many

FBOs will buy from one supplier local to their business, that supplier could be a multinational business that imports foods globally from a number of distributors. However, suppliers that import food do not have a consistent supply chain because they will buy products from the cheapest source. As such, one officer could identify an FBO as having a single local supplier and being low risk, whereas another could view the FBO as being more complex because they were sourcing products from a larger, multinational business. Another LA highlighted that the proposed model could be improved by including the descriptors from the FLCoP as this would provide clarity to officers, helping to ensure consistency, particularly for definitions of single local supplier.

''The only hiccup was really regarding some of the definitions within the risk assessment and officers perhaps having different views but I mean we're only talking one or two, on the whole it [the pilot] worked fine for us'' - Country Authority

Data consistency

Not all LAs had their databases up to date. As such, there were difficulties when incorporating the food businesses into the proposed model. For example, there was out of date information, duplicate entries, and uncertainty over whether businesses were still in operation. One LA experienced this more acutely than others. Nevertheless, LAs noted that FSA officers were very helpful in providing support to ensure data consistency. These data issues led to challenges when the LA food business risk data was converted to the proposed risk scheme. 

Data issues were reportedly compounded by COVID-19, as LAs experienced a significant increase in the number of FBOs registering at a time when footfall in businesses was being discouraged due to the pandemic. It was reported that LAs were not aware of FBOs that may have registered and then not opened. 

“During the pandemic, people were looking to keep themselves busy… we’ve had a national explosion in the amount of food registrations that are received for new food businesses… most are home factories” – Unitary LA

“We had about 1,000 new food registrations in about 3 or 4 months” – County Council

They highlighted that this could be a wider issue, and it will likely be experienced by other LAs during wider implementation. Ensuring consistency in LA data prior to national roll out will be important.

Compatibility of MIS 

LAs use MIS to record and store information across their authority’s service delivery. MIS databases are used by authorities to plan work and record premises-level activity and enforcement data and other information about the businesses they regulate in several different service areas. LAs usually have longstanding contractual commitments to their MIS software provider and the databases are often used by different teams and services within an authority to store a range of information. 

MIS are used by food standards teams to generate inspection dates, plan interventions, record their activities and to maintain premises records. It is difficult for LAs to change software provider and the function of the MIS database is established centrally, by the software provider. The proposed model was not integrated into the LAs MIS for the purposes of the pilot as this was not practicable. This presented challenges related to the recording of premises information for most LAs for the duration of the pilot. 

The most significant challenge identified related to the operational delivery of the proposed model and its compatibility with LA MIS. At the time of the pilot, across LAs there were several MIS providers, including Civica, IDOX and Tascomi. LAs highlighted that it will be important for FSA to ensure the proposed new risk model is compatible with all MIS providers prior to rollout. 

While the pilot did not expect LAs to reconcile MIS databases, the issue caused operational difficulty for pilot LAs and led to initial differences between the progress made by LAs participating in the pilot. Some LAs were not able to use their MIS during the pilot and had to input data manually. They identified this as a major challenge of pilot participation and reported that this affected their resources, as officers were required to duplicate information, recording it on both the excel spreadsheet provided by FSA for evaluation and on their MIS. 

One LA did manage to configure their MIS to be compatible with the new risk scheme and was the only LA that did not report challenges related to the MIS integration with the proposed model. The ability of this LA to resolve this compatibility issue is an important consideration for FSA, as it suggests that the issue can be resolved. 

Given the lack of consistency in MIS across LAs, it was recommended that FSA communicates with all providers to ensure that systems would be compatible with the proposed model.

Use of intelligence

As indicated above, Pilot LAs were at different stages of understanding and use of intelligence at the start of the Pilot. This led to intelligence-led activities, such as directed sampling, being adapted, or delayed.  

FSA, with input from the NFCU, delivered intelligence training for the pilot LAs. This included what it is, how it can be used and the appropriate mechanisms for gathering and sharing it. Intelligence training, guidance and support is an ongoing piece of work that FSA is seeking to develop and continue. It is recognised that there is no consistent approach to how LAs understand and use intelligence and something that will incrementally develop.  

Directed sampling 

Practical challenges were reported by LAs related to the directed sampling programme. These included locating the specified food samples and the timeframes for receiving results from the public analyst (PA) laboratories. 

Typically, LAs plan their sampling programme for the year ahead and inform their PA, which allows the PA to plan their resources accordingly. The intelligence-led approach to directed sampling meant that the timeframes for notification of sampling criteria and the submission of samples to the PA were shorter than usual. The proposed approach requires LAs to be more dynamic and responsive in the management of resources to address emerging risks, but this was identified as a difficulty by some LAs. 

“The sampling caused us problems we really need to plan at the beginning of the year… having to get samples back in the next 4 weeks did not work for us or the analyst… we still haven’t got the samples back that we took in February because the analyst needs to plan throughout the year.” – Unitary Authority

“It’s managing the [sampling] process both here and at the analyst and there’s no guarantee what samples the analyst is going to get next year so it puts an enormous amount of pressure on a system which is already straining.” – Unitary Authority

The lack of planning in advance also affected LA officer resources and work scheduling. It was identified that the directed sampling programme did not always correlate to officer availability and one LA experienced difficulties in locating the samples required, which was identified as a further strain on resources. 

​There were three distinct phases within the directed sampling programme. In phases 1 and 2, FSA issued sampling criteria to LAs in a bundle, with a long period of time assigned to officers to procure samples. In the third phase, following feedback from pilot LAs, sampling criteria were issued to LAs as multiple, staggered priorities with shorter sample procurement windows. Feedback from pilot LAs on the different approaches was then considered by FSA during the evaluation period. 

2.4 What has been the experience of each of the stakeholders with respect to the specific elements of the proposed model and the proposed model changes as a whole?

 The experience of each stakeholder (LAs and FSA) was overall very positive. 

• LAs decided to join the pilot to co-create the program with FSA, influence the change, and to be able to adapt early to it.  
• LAs identified that the main challenge was related to staff capacity and some technological challenges. 
• FSA mentioned that regular communication with LAs had been the key to the success of the project. They perceived LAs as being very open and honest, which allowed FSA to adapt the tools and guidance developed in a way that could best serve LAs and the objectives of the proposed new model. 
• a significant challenge raised was linked to the intelligence function

This section analyses the experience of the different stakeholders engaged in the pilot and their perspective on the new proposed model. The evaluation team was not able to interview any FBOs during the evaluation period, as such, this section focuses on the experience of LAs and FSA. The section addresses the reasons for LAs to join the project, considers LAs’ attitudes towards the pilot and finally summarises FSA experience of the pilot. 

2.4.1 Reasons for LAs to join the pilot programme

LAs decided to join the pilot programme for a number of reasons. All LAs noted that the previous model was not working well for them and that there was a need to update it, with many having already departed from the approach to some extent. Several LAs had been part of the existing food standards working group and were engaged in the topic already, contributing to the development of the proposed model. 

All LAs stated that the decisive factors for joining were their willingness to influence the model to ensure it fits their needs, and the capacity to anticipate and adapt to the changes as soon as possible. LAs appreciated having the opportunity to start early and co-create the new model with FSA. This shows that all LAs participating in the pilot either testing the proposed model or in the control group, had already identified flaws with the prior model, were willing to try the proposed one, and to collaborate with FSA to ensure a smooth transition. They were all self-selected and knowledgeable. 

It may be that the wider population of LAs – those who did not offer to participate in the pilot – are materially different from those who did in terms of their willingness to embrace change or their ability to do so (e.g., in terms of their resources or knowledge base). This could mean that during the planned national roll-out challenges not encountered in the context of the pilot may be experienced. Lessons learned (discussed in Section 3) highlight how showcasing the pilot LAs experience with the proposed model could be a good way of engaging other LAs. 

2.4.2 LA experience - implementing the pilot and working with the proposed model

Section 2.3.4 discusses the barriers and enablers to the success of the pilot project. When discussing LAs’ experience, they highlighted again the communications both with FSA and with the other LAs, where they were able to share progress, challenges experienced and ways forward. 

FSA provided inspection programmes based on the new model (conversion spreadsheets and reporting templates) to support implementation of the new model within the pilot, and these were well received. LAs mentioned that increased pilot paperwork constrained their resources, however, officers were aware that it was specifically due to the pilot and would not be replicated as part of the proposed model itself. 

“In officers’ minds they are looking at it and going ‘this is really good actually because the work that we are doing is being recognised and also what we think is important is being deemed by the risk scheme as being important” – County Council

“For the officers it was just getting used to the new system and once they got used to it, they carried on and I don’t think there’ll be problems because compliance and risk should always be looked at together and that is a positive of the pilot” – Unitary Authority

“The model formalised what we were already doing” – Unitary Authority

LAs were working in similar ways prior to the pilot to deliver food standards controls, however these practices were not harmonised across LAs. LAs perceived that the guidance provided by FSA to deliver the proposed model was what was needed to address the flaws identified with the existing approach. A control LA also reported being guided by intelligence in their work, which could suggest that if the model was rolled out nationally, some LAs could adapt to the proposed model faster than others. 

Some of the LAs participating in the pilot had been engaging with FSA to develop some aspects of the proposed model, so they were not surprised by the changes implemented. Six out of the seven LAs testing the proposed model had embedded the new practices by the end of the pilot and were committed to continue working in line with the proposed approach. The area that was more challenging was related to using intelligence to drive the risk assessment process. 

2.4.3 FSA experience - implementing the pilot and working with the proposed model

FSA reflections on the implementation of the pilot were very positive. The staff involved in the implementation and working regularly with the LAs shared that the regular communication with LAs had been the key to the success of the project. As discussed, FSA also perceived open communication with LAs as a key enabler for the implementation of the pilot. 

The main challenge raised was the effect that COVID-19 had on LAs, and how the first phases of the project were slower than anticipated. The second challenge was linked to the intelligence function. The intelligence team was set up to deliver the new function, which meant they worked in parallel with the pilot. While progress was made, they are aware that there is still work to be done before intelligence is embedded into the new proposed model. 

2.5 What has been the effect on resources for each of the stakeholders because of the proposed model?

LAs discussed the changes in resources in terms of:

• staff requirements to deliver the pilot, adapt to the proposed model, and continue the implementation of the proposed model. Overall, they perceive the number of staff required to be the same. 
• regarding technical skills and tools required to implement the proposed model, some LAs perceived a system adapted to the proposed model would make the work more effective.

In terms of effectiveness and efficiency of LA resources, it was identified that the proposed model allowed LAs to target resources more effectively. 

The answer to this question focuses on the changes made by the stakeholders to adapt the proposed model. This includes changes linked to the implementation of the pilot, and changes to resources linked to the delivery of the proposed model. It then assesses the efficiency of the resources used during the implementation of the pilot, using the quantitative data gathered by FSA analytics team.  

2.5.1 How LAs adapted to the proposed model 

In terms of resources, LAs mentioned two main types of change. They required additional staff resource to deliver the pilot (supported by FSA), and to adapt to the proposed model. However, once adapted to, they perceived the number of staff required was the same compared to the current framework. LAs also adapted some of their skills and tools to implement the new proposed model in a more effective manner.

Staff redeployments due to COVID-19 impacted on the capacity of LAs to adapt to and implement the proposed model in a timely manner. FSA mitigated this by extending the pilot by three months so that additional data could be gathered. FSA explained to LAs the requirements and anticipated burdens before starting the pilot. Some LAs mentioned that a lot of staff time was required for the monthly data reporting requirements, ongoing meetings, and new proposed sampling approaches, however, these aspects were only linked the pilot of the model. 

Finally, in terms of the resources required for the implementation after the pilot project, LAs highlighted that the proposed model allowed them to target resources at areas that pose the greatest risk, making them more efficient.

One of the LAs from the control group stated that they would welcome the introduction of a risk model that prioritises risk and targets inspections in a more efficient way. LAs also perceived that TRIs, where feasible, would use resources more efficiently. 

When discussing the resources required, LAs explained that they valued better targeting of their resources thanks to the proposed model. However, they perceived that the same number of staff would be required to deliver the new model. The main change would be that the proposed model would enable LAs to be more effective at prioritising food standards work and identifying and resolving non-compliances with food law. 

The second area discussed by LAs was related to the skills and tools required to follow the proposed model.

Typically, LAs had different experiences of resource needs throughout the pilot, due to their organisational differences and their perceptions of the usefulness of TRIs. These differences are important to consider for national roll out and TRIs could utilise resources more effectively, however mixed perspectives were identified throughout all three rounds of interviews.

LAs perceived that they required improved mechanisms to gather and share intelligence data. Some LAs already had an integrated intelligence function as part of their work planning and used multiple channels of communication whereas others were aware of intelligence but were not guided by it. LAs that were driven by intelligence mentioned that they had adapted their MIS, so they are fit-for purpose and up to date. They also mentioned they had trained their officers so they would be familiar with the systems and the intelligence function. FSA did not require LAs to do this, however, those that adapted their systems found it easier to work with the new proposed model. 

2.6 What has been the overall effect of the proposed model? Did it deliver its objectives? Were there unintended consequences?

 The pilot had a positive effect on: 

• improved identification of food businesses that present the greatest risk. The LAs using the proposed model identified a higher number food businesses that were not broadly compliant with food law (16%) when compared to the control group (4%) during the pilot period.
• targeting interventions better. The proportion of proactive (programmed) interventions that led to a follow up corrective action to prevent or address an issue was higher for the pilot group (57%) than the control group (37%). 
• increase in the proportion of LA resources driven by intelligence. The number of reactive interventions (intelligence led) is much higher in the pilot group, accounting for 20% of the total number of interventions, compared to 1% in the control group. 

The evaluation team did not identify any unintended consequences.

This section summarises the effects of the proposed model, namely whether it achieved the outputs and outcomes as set out by the intervention logic (see Annex 2): 

• whether the proposed model is effective at targeting higher risk FBOs. 
• whether LAs changed the way they were working. 
• whether there has been an increase in the proportion of LA resources driven by intelligence rather than establishment-based risk assessment. 

The evaluation team also assessed whether there had been any unintended consequences. The potential for improvement is discussed later, in section 3.) 

This section draws on data supplied to the FSA by LAs engaged in the pilot. As mentioned in section 1.4.3, the quantitative data for one of the control LAs distorted the overall analysis results and has been excluded from this chapter. An analysis of the full data set is included in Annex 3, which also includes a note on data quality. 

2.6.1 Identifying FBOs that present the greatest risk

The new risk model was identified as offering a more accurate approach to identifying risk, as it considers the inherent risk of a business and their previous compliance record. This was identified as a positive element because businesses are not rigidly limited to always being in the same risk category.

LAs highlighted that the new risk model also allows for officers to be more reactive to emerging risks and current trends. For example, LA officers identified that during the COVID-19 lockdowns, there was an increase in new businesses opening from domestic settings (e.g., people baking at home and selling their produce). Under the current FLCoP model these businesses would not fit into a category that would be deemed as high risk, however, LA officers considered them high risk due to levels of awareness of food risk and applicable legislative requirements, in addition to the risks introduced when producing food in a domestic setting. This makes the proposed model more effective at identifying higher risk establishments.

2.6.1.1 New types of interventions: Targeted Remote Intervention (TRI) and No Actionable Risk (NAR)

The proposed decision matrix introduced two new types of interventions, TRIs and NARs. 

LAs reported having different experiences with TRIs. Some LAs, including a control LA, had carried out some type of remote interventions outside of the pilot due to COVID-19 restrictions. Some LAs highlighted that TRIs were useful because they can provide an intervention for FBOs who would not have previously had any contact with the LA. One LA reported that TRIs did not work for their geographical area partly because there were a lot of small businesses who were not equipped for remote inspections. Typically, TRIs were viewed as being appropriate for some types of businesses, where the information exchange would be easier to facilitate than for other businesses. For example, manufacturers or microbreweries would often be able to send the required information to an officer faster than other types of businesses, allowing LAs to complete the TRI in a more efficient manner. Smaller business did not always have the information to hand, leading to long waiting times for officers to receive the required information, delaying the outcome of the intervention, resulting in a less efficient TRI. 

There were mixed responses regarding the effectiveness of the NAR category. The NAR category was added as part of the proposed food standards risk model, and applied to compliant low risk premises who do not require a regular intervention but would instead be subject to ongoing monitoring by the LA through other means. One LA highlighted that some premises that were identified as NAR perhaps presented a higher risk, because the business activities had changed. An example was provided for an FBO that was previously NAR but had started to sell sandwiches with labels on. The pre-packed for direct sale (PPDS) legislation change had meant that this FBO was no longer suitable for the NAR category. Therefore, it was suggested that the NAR category may require a verification of FBO activity to ensure they are consistent with the NAR classification.  On the other hand, other LAs suggested that the NAR category did not have a significant impact on them because they did not have many FBOs in the category.

2.6.1.2 Overall effectiveness

To assess the effectiveness of the proposed model, the FSA analysed the types of intervention comparing those using (pilot) and not using (control) the proposed model. They also assessed the number of non-compliances identified, whether a business was broadly compliant with food law, the number of follow-up interventions scheduled, as well as the proportion of proactive (programmed) and reactive (intelligence led)  interventions by type of intervention completed. 

Following an intervention, the food business is risk assessed. Using the outcome of the risk assessment, a food business can be classed as broadly or not broadly compliant with relevant food law. A typical food business that is considered to be broadly compliant will have no identified non-compliances or only minor ones that pose a minimal risk to consumers (e.g. equivalent to a Food Hygiene Rating Scheme rating of 3 or above). A business that is considered to be not broadly compliant will have serious non-compliances that need to be rectified by the FBO – LAs will focus on these businesses to bring them back into compliance. 

Figure 3 shows that the scheduled interventions for LAs testing the proposed model led to the identification of a higher number of ‘not broadly compliant’ food businesses when compared to the control group (16% compared to 4% respectively). Moreover, the difference between the control and pilot groups is statistically significant. This demonstrates that the new risk scheme is more effective at identifying non-compliant businesses and directing LA resources to the higher-risk establishments to resolve non-compliance. 

Figure 3 Proportion of proactive (programmed) interventions that found not broadly compliant food businesses, control vs. pilot
 

Similarly, the proportion of proactive (programmed) interventions that led to a follow up corrective action by the LA to prevent or address an issue was higher for the pilot group than the control group, at 57% and 37% respectively, as shown in Figure 4.

Figure 4 Proportion of proactive (programmed) interventions that resulted in follow up corrective action by LAs

A chart bar graph with control and pilot in the axis. It shows that control LAs, with a base size of 477 interventions, had a total of 37% of proactive (programmed) interventions that resulted in follow up corrective action by LAs. It also shows that pilot LAs, with a base size of 1,399 interventions, had a total of 57% of proactive (programmed) interventions that resulted in follow up corrective action by LAs.

Figure 5 shows the proportion of interventions by type within the pilot vs control groups. As can be seen, 14% of the pilot group interventions were TRIs instead of on-site interventions. 

Figure 5 Proportion of intervention types, control vs pilot.^(footnote) 

Group	On-site intervention	TRI	New food business	Remote intervention due to COVID-19	Total
Control	90%	-	104%	-	100%
Pilot	73%	14%	-	9%	100%

2.6.2 Better targeting of interventions 

The proposed model tested in the pilot project led LAs to identify and resolve more non-compliances thanks to the better profiling of FBO risk because of the changes to the risk assessment scheme. This enabled LAs to more accurately identify those businesses deemed to be the highest risk. 

The proposed risk scheme allowed officers to assess the risk of food businesses more accurately and determine their intervention frequency based on the officers’ professional assessment of risk. This would not have been possible when operating under the previous model. Under the proposed risk scheme officers can use their knowledge of an FBOs’ previous performance and management to assess the risk posed by these businesses and determine the appropriate intervention frequency.

2.6.3 Proportion of LA resources driven by intelligence

Another aspect of the model was that it led some LAs to use intelligence to drive their work more. There was a suggestion by one LA that intelligence received under the pilot was viewed as an improvement compared to before the pilot. This was also identified by another LA who highlighted that officers’ perception of intelligence had become broader, and they were recognising when problems may not be local and sharing more information with FSA.

“Our intel both ways, to be fair, from FSA was always massively lacking and under the pilot it’s been much better and I think making the team think about intelligence has definitely come out from the pilot rather than [officers] saying ‘we’re going to do our 400 premises’ and [now they’re] thinking about where do we want to go and what intelligence have we got coming through” – Unitary Authority

This indicative finding could be viewed as positively suggesting that the pilot is contributing to improvements in the use of intelligence and facilitating collaborative working.

The data gathered by FSA analytics team showed that by embedding intelligence into the delivery of food standards official controls, the regulatory model became more dynamic, enabling the system to identify non-compliances more effectively and earlier, and to disseminate intelligence across stakeholders more often. However, the intelligence function requires further development (as discussed in section 2.3.3). 

The proposal to embed intelligence into the delivery of food standards official controls will result in a regulatory model that is more dynamic by nature, enabling FSA and LAs to identify non-compliances more effectively and earlier, and to disseminate intelligence across stakeholders more often.

2.6.3.1 The proportion of reactive (intelligence led) interventions

FSA measured the proportion of interventions led by any form of intelligence (reactive interventions) in comparison with the proportion of interventions that are scheduled according to the indicative risk profile (proactive interventions) within the pilot group versus the control group. The sample size for the pilot group was 1,305 interventions, where for the control group was 473. 

As can be observed from Figure 6, the number of reactive interventions is much higher in the pilot group, accounting for 20% of the total number of interventions, compared to 1% in the control group. This means that the proposed FSDM is more dynamic, allowing LAs to use intelligence more frequently and carry out specific directed corrective actions to prevent or address food standards non-compliances in a more timely and more targeted way. 

Figure 6 Proportion of proactive (scheduled) vs reactive (intelligence led) interventions in control LAs vs pilot LAs 

A chart bar graph with control/pilot, and proactive / reactive in the axis. It shows that control LAs, with a base size of 473 interventions, had a total of 99% of proactive interventions vs .1% of reactive interventions. It also shows that pilot LAs, with a base size of 1,305 interventions, had a total of 80% of proactive interventions and 20% of reactive interventions

As can be observed from Figure 7, each LA within the pilot group was disseminating intelligence to other stakeholders, compared to a single intelligence report from an LA within the control group. Overall, this demonstrates that the process of intelligence sharing has increased the level of understanding of potential risks, driving up the quality of intelligence reports shared across all stakeholders. This also suggests that the process of communication has improved within the pilot, LAs being able to coordinate and collaborate better with their neighbouring LAs, FSA, Primary Authorities, etc. This statement is also verified and supported by the qualitative assessment.

Figure 7 Intelligence reports LAs disseminated to other stakeholders, control vs pilot

Other	Other LA	Environmental health	FSA	PA	Industry	NFCU	Total
Control	0	0	1	0	0	0	1
Pilot	44	20	16	12	5	1	98

2.6.3.2 Directed sampling 

The directed sampling programme has enhanced the national intelligence picture on food standards and facilitated national action to address areas of concern. Overall, the directed sampling programme supported FSA’s priorities to ensure food is safe and is what it says it is. The directed sampling programme was evidence-led, based on intelligence, and helped to develop a coordinated approach to sampling. During interviews, LAs supported the way that intelligence influenced the directed sampling programme to target emerging risk areas. 
Directed sampling during the pilot identified that half of the 317 products sampled were unsatisfactory to some degree. The highest rate of unsatisfactory results (i.e. not compliant with all applicable legal requirements considered as part of the analysis) was identified in phase II of the directed sampling programme, which focused on products where existing non-compliance data (IDB and FSA Incidents and Resilience data) indicated products were likely to be unsatisfactory.

2.6.3.3 Other sampling during the pilot

Figure 8 below shows that the pilot group took significantly more proactive samples (as a proportion of total sampling) than the control group because of focusing more on the use of intelligence. The numbers do not include the directed sampling figures. It can be noted that the proportion of reactive samples that were unsatisfactory within the pilot was larger than the proportion of unsatisfactory (non-compliant) proactive samples, which demonstrates a more efficient use of intelligence. Also, as shown in the base sizes, control LAs completed a lower number of samples than the pilot LAs (control reactive completed 3 interventions, control proactive 16; pilot reactive completed 36 interventions, and pilot proactive 769). Based on the small size, the control LAs found no unsatisfactory samples.  

Figure 8 Proportion of proactive and reactive sample outcomes, pilot vs control (excluding FSA directed sampling) 

A chart bar graph with control reactive, control proactive, pilot reactive and pilot proactive; and satisfactory / unsatisfactory. It shows that control reactive LAs, with a base size of 3 interventions, had a total of 100% satisfactory results, control proactive LAs, with a base size of 16 interventions, had a total of 100% satisfactory results. Pilot reactive LAs, with a base size of 36 interventions, had a total of 50% satisfactory results and 50% unsatisfactory, and pilot proactive LAs, with a base size of 769 interventions, had a total of 64% satisfactory results, and 36% unsatisfactory

2.6.4 Unintended consequences

The evaluation team did not identify any unintended consequences because of piloting the proposed model, except the challenges discussed in section 2.3.4.2. Overall, participating in the pilot was perceived favourably by LAs.