Evaluation of the Food Standards Pilot in Wales: Lessons learned

3.1 Implementation of the pilot

The implementation of the pilot highlighted that: 

• The support provided by the FSA throughout the pilot was well received and crucial to the success of the pilot. This level of support was specific to the pilot and any support under a potential roll out would likely be conducted in a different format to accommodate the larger number of LAs involved as well as the number of different MIS systems involved. Learning from the Welsh pilot has already been fed into preparation of the support for the roll out of the model in England and Northern Ireland. This feedback can be found in sections 2.2.3 around further guidance needs and 2.3.1.2 regarding feedback on preparation and support activities within the pilot. Overall, support was important to: 
  • Ensure consistent understanding and interpretation of the terms used to define the proposed risk scheme and decision matrix.
  • Quickly resolve any queries or provide support around any feedback LAs had during the testing of the proposed model.
  • Build confidence across LAs and help embed the pilot and model into the LAs work practices.
  • Create peer to peer learning opportunities for the two pilot LAs where they were able to share examples of best practice between themselves and present their experiences to a wider audience. 

The two pilot LAs also noted feedback on two practical elements of the pilot itself:

• More consideration should be given to the timing of pilots in the future. One LA reported finding the end of the pilot particularly stressful as it ended a month before the end of the financial year which increased the administrative burden during an already busy time.
• Both LAs noted that a longer pilot period would have helped to show the impact of the proposed model more as they felt that 6 months wasn’t long enough to see the changes in non-compliance. As this pilot builds on the findings of the initial 18-month pilot in England and NI it is likely that given a longer time period the Welsh LAs would have seen similar patterns in non-compliance.

3.2 Lessons ahead of a potential roll out of the proposed model 

The evaluation has also illustrated some lessons about how the proposed model operates and what needs to be considered ahead of a potential roll out. 

The LAs reported that they would be happy to continue with the proposed model and found it beneficial. However, they both felt additional training is needed. One in particular felt that much more guidance was needed from the FSA about how to run the joint service intervention frequency reports. The LA also identified that they would require further guidance on creating alternative enforcement questionnaires, suitable to different premises for use when food standards interventions are needed without food hygiene, to be beneficial (see section 2.2.1.3).

Both LAs emphasized that, if rollout of the proposed model was to take place, then the FSA auditors would need to accommodate the time needed for LA teams to adapt to implementing a new scheme. There is likely to be a slowdown in activities whilst confidence and experience are built up and LAs were looking for reassurance that they would not be penalised for this whilst they adapted. 

Both pilot LAs suggested that regional or database based LA support groups could be useful if a wider roll out was to take place. One pilot LA would also recommend their consistency exercises (developed by the LA) as they found it to be beneficial for their team bonding as well as for the quality of their service.

3.2.1 Reflections from the England and Northern Ireland process 

There are a number of reflections to be considered in a potential roll out of the proposed model in Wales, following the experience of the roll out in England and Northern Ireland: 

• Training and support could follow a similar approach – in England and Northern Ireland it will be provided within LAs’ MIS provider-based groups. 
  • Each group could have a single point of contact at the FSA who will provide support and training materials, and access to an online space (subject to data protection) on the Knowledge Hub where LAs can discuss issues and solutions. 
  • Additional support could include attendance of the FSA at regional food standards group meetings to give up-dates on the roll out and provide an opportunity to ask questions. 
  • Consistency exercises could be offered following roll out, where officers have had an opportunity to better familiarise themselves with the model. 
• The process for mapping LA databases will change when the proposed model is rolled out in England and Northern Ireland, due to both the algorithm used and the data quality within some LA systems, leading to some food businesses not being rated correctly (as discussed in 2.3.1.4). This is only relevant once, as it will form the basis of the new interventions. LAs will then update the risk ratings based on their interventions and intelligence received. 
  • To reduce this risk the new process will ask LAs to identify key businesses of concern prior to mapping. These can be assessed individually to ensure they received an accurate score. 
  • LAs will be asked to validate the scores of 10% of their mapped database to ensure accuracy. 
  • These two actions could help reduce the risk of establishments not being rated correctly as well as be an opportunity for LAs to familiarise themselves with the new model. 
• The FSA also noted that MIS providers could be involved in the data mapping process, helping to promote a smooth transition. MIS providers have been regularly engaged by the FSA to keep informed of the pilot and progress.

3.2.2 Reflection from the evaluation team

The evaluation team would recommend the following during a potential roll out (some of these may have already been taken into account by the FSA team):

• Facilitate a shared understanding of the terminology used in the proposed risk scheme and decision matrix, together with practical guidance illustrated with examples from the LAs in the pilot. LAs mentioned that some of the descriptions initially used for the proposed risk scheme were not very clear. LAs suggested they could benefit from re-wording or some examples. An improvement in the definitions and descriptors would help with consistency in interpretation between LAs or officers during a national roll-out. 
• Continue to provide a forum for sharing of experiences between LAs. This could be in the form of regional or MIS based discussion groups or online discussion threads. This could be coupled with consistency exercises facilitated by the FSA to encourage discussion around consistency and interpretation.
• Ensure that specific guidance is provided to joint-service LAs to maintain smooth working practices between the food standards and food hygiene services. This guidance should cover how to compare food standards and food hygiene intervention frequency reports. 
• Explain how allergens are assessed and integrated in the proposed model more clearly, by improving the descriptors in the risk rating. This particularly relates to scoring of businesses with no allergen risk due to the specific products they produce or supply as well as further clarity on the thresholds between each of the allergen risk scores.
• Provide support to LAs during the process of mapping databases from the current model to the proposed new model to reduce the risk of high-risk businesses being reclassified as non-priority due to issues with data entry. LAs should be supported in understanding how the mapping process will work and how they should go about verifying if the mapping has been successful to promote confidence in the new model and its set up processes.