Skip to main content
English Cymraeg
Evaluation of the Food Standards Pilot in Wales

Evaluation of the Food Standards Pilot in Wales: Introduction

Wales specific

This is the final report of the evaluation of the proposed food standards delivery model in Wales.

Last updated: 28 January 2025
Last updated: 28 January 2025

This report presents the findings from the evaluation of the pilot of the proposed food standards delivery model in Wales. The pilot and the evaluation build on the findings from the England and Northern Ireland pilot evaluation

The FSA piloted a proposed food standards delivery model between January 2021 and March 2022 in England and Northern Ireland. The proposed model was then subject to a formal consultation and amended further, taking formal responses into account, and has now been introduced in England and Northern Ireland.

LAs in Wales were unable to pilot the proposed model at the same time as England and Northern Ireland. Building on the previous pilot and implementation of the model in England and Northern Ireland, the FSA piloted the model introduced in England and Northern Ireland as a proposed new food standards delivery model to use in Wales between September 2023 and February 2024. The pilot tested a proposed new model for the delivery of food standards official controls. This introduced a modernised risk assessment approach that aims to support LAs target their resources more effectively; provide more flexibility to LAs; and help LAs meet their statutory obligations. 

This is the final report of the evaluation of the proposed food standards delivery model in Wales. 

The report is organised as follows: 

  • The rest of this section (section 1) introduces the background context for this pilot project and discusses the proposed food standards delivery model in Wales. It concludes by summarising the methodology followed by the evaluation study. 
  • Section 2 summarises the findings of the evaluation, organised by research question.
  • Section 3 includes a series of considerations and lessons learned.
  • Section 4 closes the report with the conclusions. 

1.1 Background

This section introduces the background context for this pilot project. It discusses the findings from the England and Northern Ireland pilot evaluation study, describes the proposed food standards delivery model, introduces the Wales pilot project and provides background information on the local authorities in Wales. 

1.1.1 Summary of findings of the food standards delivery model pilot in England and Northern Ireland 

Between January 2021 and March 2022, the FSA tested a proposed food standards delivery model with eleven LAs in England and Northern Ireland (the ‘initial pilot’). The proposed food standards delivery model aimed to support LAs to target resources more effectively; provide better assurance and more flexibility to LAs; and to help LAs meet their statutory obligations. The proposed model introduced a modernised risk assessment approach, including a new risk assessment scheme, a decision matrix and the development of an intelligence-led approach to LA regulatory activity. 

The evaluation of the initial pilot found that the new elements introduced by the proposed food standards delivery model worked well. The new risk scheme better identified high-risk food businesses and offered benefits to LAs such as greater flexibility in how to use the model, the ability to re-score premises and the ability to use intelligence to select the most appropriate intervention. Local Authorities (LAs) highlighted that the new risk scheme was straightforward to use, and, by the end of the pilot, they had already embedded the new way of working into their work practices. 

LAs identified some challenges with the model. Most were resolved within the pilot. Other challenges like the use of intelligence, the use of targeted remote interventions (TRIs) or the identification of non-compliances linked to allergens, were addressed by adapting the proposed model to mitigate them. 

This adapted proposed model was then subject to a formal consultation and amended further, taking formal responses into account, and has now been introduced in England and Northern Ireland.

1.1.2 The proposed food standards delivery model in Wales

The Food Law Code of Practice establishes a framework for the delivery of official food controls by LAs. It determines the appropriate intervention frequency for food businesses based on the associated risk profiles for different establishments and businesses. Each country in the UK has their own Food Law Code of Practice, and while those issued in Wales, England and Northern Ireland are generally similar, there are some country-specific differences. LAs in Wales must have regard to the Food Law Code of Practice (Wales) (hereafter, ‘the Code’) while fulfilling their duties in relation to food (both food hygiene and food standards). 

For food standards, the existing intervention rating scheme groups food establishments into Category A (high risk, requires an intervention every 12 months), Category B (medium risk, requires an intervention every 24 months), and category C (low risk, should be subject to an intervention at least once every 60 months). This allows LAs to prioritise their interventions. LAs will visit all new premises to determine compliance with applicable food law and allocate a risk rating. Interventions by LAs can take the form of an inspection, partial inspection or audit. Under this rating scheme, some establishments, due to the nature of their activities, are identified as Category A (high risk) regardless of their level of compliance.

Additionally, the FSA and LAs can monitor the change in compliance from businesses over time using a metric of ‘non-broadly compliant’ and ‘broadly compliant’ businesses. These metrics are used by the FSA and LAs to get a high-level understanding of how FBOs are operating. The current definition of “broadly compliant” is set out below: 

  • Definition under the Code: an establishment that has a score of not more than ten points under both the Level of (Current) Compliance and the Confidence in Management/Control Systems.

“Non-Broadly Compliant” would be any establishment that does not satisfy the above requirements. So, under the Code, if an establishment had a score of more than 10 for either the Level of Current Compliance or the Confidence in Management they would be regarded as non-broadly compliant. 

A series of reports (Research on the modernisation of the risk intervention rating systems for UK food establishmentsFood Standards Delivery Review, and Ensuring food safety and standards)  highlighted the need to change the current food standards delivery model under the Code. The reports identified several challenges with the current model, which, combined with a decline in LA resources and a rapid change and growth in the types of food establishments, showed that the current food standards delivery model is no longer fit for purpose. 

The challenges highlighted by the reports were: 

  • LAs are taking inconsistent approaches to regulating food standards, as the current risk model doesn’t always accurately reflect the overall level of food business risk. 
  • The current model follows an establishment risk-based approach, which is perceived as not the most effective in identifying non-compliances. 
  • The current model does not support LAs in targeting their resources towards the areas of greatest risk. 

The FSA Board approved a root and branch review of the food standards delivery in December 2018. Following this decision, the FSA designed the proposed new food standards delivery model in consultation with LAs from the three nations (England, Wales and Northern Ireland). The proposed new model intends to give a more comprehensive reflection of the risks posed by food businesses (see Annex 1 for further details of the proposed model). 

To address the challenges identified above, the proposed model incorporates three elements:

  • A modernised risk assessment scheme to unify the way that LAs risk assess establishments (a new risk scheme).
  • An assessment scheme using a matrix approach based on levels of inherent risk and compliance to identify the appropriate frequency for official control activity.
  • Greater integration of intelligence as a driver of local authority regulatory activity and to inform the national understanding of food standards risk.

Additionally, due to the change in scoring approach the proposed model introduces a new definition for ‘broadly compliant’. The new definition of ‘broadly compliant’ is set out below: 

  • Definition under the proposed food standards delivery model: an establishment receiving an overall compliance risk assessment score of 3, 4 or 5.

“Non-Broadly Compliant” would therefore be any establishment that does not satisfy the above requirements. So, under the proposed food standards delivery model, if an establishment had a score of 1 or 2 as an overall compliance risk assessment score they would be regarded as non-broadly compliant. 

Based on the findings from the initial pilot and the evaluation report, the FSA decided to adjust the proposed food standards delivery model. A summary of the changes is presented in Table 1.1.

Table 1.1 Changes to the proposed model following the initial pilot
Lessons learned during the pilot in England and NI Changes to the proposed model to be tested in Wales
England and NI pilot LAs found it difficult to identify non-compliances linked to allergens as the proposed risk assessment scheme did not include allergens as a separate element to be considered (allergens were embedded as part of other elements).  Introduced a new risk factor on allergens to the proposed risk assessment scheme.
England and NI pilot LAs did not find useful having Targeted Remote Interventions (TRIs) linked to a particular risk score under the matrix. They consider TRIs useful in some circumstances but prefer to consider their use on a case-by-case basis rather than a risk score. TRIs are not part of the decision matrix anymore. The model changed to allow LAs to use remote interventions if and when they consider them to be appropriate.

The England and NI pilot had a new category called No Actionable Risk (NAR), where no future intervention date would be allocated to the food business. This would mean LAs had to react to intelligence or complaints and visit the NAR business accordingly.

After the pilot it was considered that the NAR category did not meet the Official Control Regulations (OCRs) requirement. OCRs require regular official controls to be conducted at an appropriate frequency based on risk. 

NAR was replaced by a new intervention frequency of 120 months.
England and NI pilot LAs asked for a way to prioritise the backlog of unrated businesses.

The model introduced a desktop assessment (DA) for LAs to evaluate the inherent risk of new food businesses. 

The assessment should occur within 28 days of the business registering or the LA becoming aware of its operation. Information for this assessment is sourced from the documents completed by businesses when registering, such as the Register a Food Business (RAFB) completed by new FBOs (Food Business Operator) to comply with the Hygiene Regulations (Food Hygiene (Wales) Regulations 2006 and Article 6(2) of assimilated Regulation (EC) 852/2004), supplemented by other relevant information (for example from company websites, telephone conversations, emails, or questionnaires.)

There were several challenges identified with the intelligence function during the pilot, such as: what type of data should be shared, how frequently, what mechanisms to use and who should the data be shared with. The Local Authority Intelligence Coordination Team (LAICT) have adapted their training and engagement with LAs to streamline this element. They have introduced, for example, regular intelligence newsletters that share trends and risks with LAs, and identified the type of data and frequency that they would like LAs to share.

1.1.3 Local Authorities in Wales

There are 22 Local Authorities in Wales, and these are all unitary authorities. Unitary authorities operate under a single tier structure, with LAs responsible for all services in their area, including food hygiene and food standards. 

Food standards official controls in Wales are delivered by a combination of dedicated food standards teams and teams that deliver both food hygiene and food standards controls. For the purposes of this report, the latter will be referred to as joint services. In joint services, food hygiene and food standards controls are sometimes delivered together by the same officer during one visit.  

1.1.4 The pilot project

The pilot project in Wales commenced in September 2023 and ended in February 2024. The pilot tested the proposed food standards delivery model (see Annex 1 for further detail on the proposed food standards delivery model) with four LAs. Two LAs implemented the proposed delivery model (pilot LAs), and two LAs continued to work to the existing model detailed in the published Code (control LAs). 

The aim of the pilot was to:

  • demonstrate how the proposed new model applies to the Welsh context, and 
  • provide opportunities for LAs in Wales to participate and identify areas where the model can be made more suitable and applicable to the delivery of food standards official controls in Wales.

The proposed new model tested under the pilot does not amend the food standards legal requirements that FBOs must comply with.

The four LAs participating in the pilot project were selected based on their willingness to participate and in consideration of the following characteristics to ensure a representative sample: 

  • LAs from North and South Wales.
  • LAs delivering food standards controls through a joint service (food hygiene and food standards delivered jointly) and others delivering solely via dedicated food standards teams. 

As the pilot in Wales built on the lessons learned from the initial pilot in England and Northern Ireland, the FSA decided to use a smaller sample of LAs in Wales. The Wales pilot involved only four LAs meaning that the findings in this report cannot be completely representative of the Welsh LA population in total. However, the above criteria ensured that the different types of LA in Wales were represented in the pilot. Statistical tests have been included where possible however due to the small sample size differences between LA groups (control vs pilot) and between LAs more generally may be due to other factors and not necessarily due to the proposed model. 

While all LAs in Wales are unitary, the selection represents the delivery method of food standards official controls in Wales: jointly or separately. Table 1.2 below indicates the characteristics of the LAs participating in the pilot: 

Table 1.2 Local Authority Profile
Pilot / Control Delivery method North Wales / South Wales
Pilot Joint service delivery (food standards and food hygiene) North Wales
Pilot No joint service delivery (food standards) South Wales
Control Joint service delivery (food standards and food hygiene) North Wales
Control No joint service delivery (food standards) South Wales

1.2 Evaluation objectives and research questions

This evaluation aims to answer 5 questions: 

  1. How did the proposed model perform compared to the current framework? What worked well and less well? 
  2. What has been the experience of each of the stakeholders with respect to specific elements of the proposed model and the proposed model changes as a whole?  
  3. What has been the effect on resources for each of the stakeholders because of the proposed model? 
  4. What has been the overall effect of the proposed model? Did it deliver its objectives? Were there unintended consequences? 
  5. What lessons were learned from piloting the proposed model?

The ICF study team developed an evaluation matrix to answer the questions, shown in Table 1.3  below. 

Table 1.3 Analytical approach to answering each evaluation question
Evaluation question Data collection method Examples of data collected / indicators
1. How did the proposed model perform compared to the current framework? What worked well and less well?

Two waves of interviews with LAs

Interviews with FSA staff 

A series of meetings with FSA staff

Quantitative data gathered by the FSA

LAs and the FSA perspective on ease of use of the new approach

Enablers / barriers to using the new proposed model (IT, skills, resources, location, type of food business)

LAs and the FSA perspective on the comprehension of the proposed model

2. What was the experience of each of the stakeholders with respect to specific elements of the proposed model and the proposed model changes as a whole

Two waves of interviews with LAs

Interviews with FSA staff 

A series of meetings with FSA staff

Perceptions by LAs on quality of training received to prepare for proposed model 

Opinion of users of the proposed model on ease of communication, frequency, and quality of data

The FSA perceptions on the proposed model

3. What was the effect on resources for each of the stakeholders because of the proposed model?

Two waves of interviews with LAs

An online meeting with FSA staff

Opinion on the adequacy of resources for implementing the proposed model

Changes to how LAs use resources 

Changes made by LAs and the FSA to adapt to proposed model (costs, staff, IT systems, skills)

4. What was the overall effect of the proposed model? Did it deliver its objectives? Were there unintended consequences? 

Two waves of interviews with LAs

Interviews with FSA staff 

A series of meetings with FSA staff

Data generated by FSA evaluation plan

LAs perceptions of impact on the identification of non-compliance and actions to resolve 

Frequency and type of inspections

Perceptions of consistency under the proposed model

LAs views on potential unintended consequences

5. What lessons were learnt from the pilot?

Data generated by evaluation plan

Interviews with FSA staff 

A series of meetings with FSA staff

Adaptation of the pilot using data generated and analysed if needed

Use of lessons learned from the pilot to inform the proposed model

1.2.1 Methodology

1.2.1.1 Phase 1: Scoping

The scoping phase included: 

  • Review of existing documentation, including the updated pilot guide for the proposed new food standards delivery model in Wales, and other relevant documents outlining the activities and outcomes to date. 
  • Interviews with FSA staff to understand the pilot objectives, activities and progress to date, and to inform the evaluation methodology. Three interviews were conducted. Interviewees included members of the Wales FSA team, the FSA Local Authority Intelligence Coordination Team (LAICT) and the FSA Analytics Unit. 
  • Definition of the evaluation framework, including developing the interview guides and finalising the method.  

1.2.1.2 Phase 2: Data collection 

The data collection phase included: 

  • Two rounds of interviews with LAs to understand their expectations, their experience with the process including any challenges, as well as to document the changes implemented due to the proposed model. The first round was completed before the start of the project between June and July 2023 and the final round was completed a few weeks after the pilot project had ended in March 2024. Interviews were held with all four LAs participating in the pilot. The interviews followed a semi-structured format and were recorded. 
  • A series of regular meetings with the FSA to share progress updates and discuss any issues which had arisen, as well as defining timelines.
  • Interviews with FSA staff at the end of the pilot, to capture their experience with the process. Two interviews were conducted, one with a member of the FSA in Wales and the other with a member of the Regulatory Compliance Division (RCD) leading on the England and Northern Ireland roll out.
  • Quantitative data collected by the FSA Analytics Unit (AU). AU collected monthly data (between September 2023 and February 2024) from all participating LAs (control and pilot groups) in a standardised format. LAs self-reported this data during the pilot period. Data collected included risk scores from every inspection, data on reactive (intelligence led) vs proactive (programmed) interventions and compliance scores. For pilot LAs the risk assessment data was provided for both the current model and proposed model. For control LAs the risk assessment data was only for the current model as they did not operate under or see the proposed model.

The pilot LAs carried out 353 interventions (of which, full details were given for 198). The control LAs carried out 225 interventions (of which, full details were given for 184). These figures exclude any premises visited by LAs that were found to be permanently closed. It should be noted that all data shown in this report is for premises that were visited by LAs during the pilot period only, data is not shown for the full set of premises within an LAs area. 

1.2.1.3 Phase 3: Data analysis  

The data analysis phase was continuous through the life of the pilot project. The data analysis phase included:  

  • A series of meetings with FSA staff to share the findings collected, ensure a common understanding of the main challenges and enablers, and refine the evaluation data collection tools. It also included meetings with FSA Analytics Unit to understand their data collection process to integrate the data into this report. 
  • Analysis of the evidence collected such as interview responses and a review of documents provided by the FSA. Thematic analysis was used to analyse the interview responses in line with the evaluation questions. The ICF team collated and assessed the evidence based on the themes in this report. 
  • Analysing and integrating the quantitative data collected by the FSA Analytics Unit. Chi-squared tests were performed at a 5% significance level. 

Limitations:

  • The sample size (4 LAs) was small and the length of the pilot (6 months) short. This meant that there were limitations to the representativeness of the data collected (in terms of LA characteristics, mix of premises types, and period of time for the LA to complete interventions). 
    • As mentioned in section 1.1.4, the small sample size of 4 LAs for the pilot project meant that the results were not representative of all LAs in Wales. While statistical tests have been used where possible, it is important to note that differences between LA groups (control vs pilot) and between LAs more generally may be due to other factors and not necessarily due to the proposed model. 
    • The small sample size should also be noted when considering the qualitative findings as there was a limited number of individuals involved in piloting the proposed model.
    • Further, the short timeframe of the pilot (6 months) made it more difficult to run robust statistical analysis with which to draw conclusions. It also meant it was not possible to observe evidence of changes in compliance of targeted premises. 
  • This pilot project built on the positive findings and lessons learned from the initial pilot whilst testing the proposed model in the Welsh context. Therefore, the shorter timescale and small sample size were deemed appropriate for this purpose.

Despite these limitations the evaluation team were able to assess the fit of the model in the Welsh context and identify any unintended consequences.