Evaluation of the use of remote assessments for FHRS requested re-inspections in England

Remote assessments for FHRS requested re-inspections: Annexes

Penodol i Loegr

Annexes, including structure of the study.

Diweddarwyd ddiwethaf: 17 July 2023

Gweld yr holl ddiweddariadau

Gweld as PDF

Diweddarwyd ddiwethaf: 17 July 2023

Gweld yr holl ddiweddariadau

Gweld as PDF

Annex 1: Study structure

The study questions are set out below, alongside the sections of the report that respond to them.

Study questions for LAs

Study questions	Sections
1. How do LAs define remote assessments and how are they being delivered? How are remote assessments are being undertaken in practice and what technologies are being used (e.g. are photos/video/documents being sent by email, or are video calls being used to allow the inspector to see the establishment?) How aware LAs are of the guidance in the LA Recovery Plan regarding the use of remote assessments; how useful the guidance is; how effective it is in supporting the decision to use remote assessments or not; and whether LAs require additional information to improve its effectiveness?	Section 3.1
2. When, where and why are LAs using remote assessment for re-rating inspections? How often do they use them, what circumstances the LA has chosen to use them in and if there are any particular types of businesses they have chosen to use remote assessment for? How many remote assessments for requested re-ratings has the LA completed since the Recovery Plan came into force in July 2021? How does this number compare to the total number of re-visit requests they have received from businesses within the same timeline?	Section 3.2
3. What have LAs’ experiences of using remote assessments for a re-rating inspections been? How do LAs think they compare with traditional in-person inspections?	Sections 3.2, 4.1, 4.2 and 4.3
4. In what circumstances have remote assessments for re-rating inspections worked well and where have they worked less well? What factors have contributed to this? What has gone well when using remote assessment for re-rating and not so well (for example if remote assessment has been a hinderance during the assessment itself)?	Sections 4.2, 4.3 and 4.4
5. What have been the benefits, if any, to LAs of using remote assessments to re-rate businesses? What is the perceived impact of the changes on LA resources (e.g. time spent on remote assessments compared to physical inspections (including time undertaking the assessment, time spent following the assessment, travel time, etc.), capacity to complete inspections, finances etc.)?	Section 4.3
6. What have been the limitations, if any, of using remote assessments to re-rate businesses? Do LAs have any concerns about the data they get back from FBOs through remote assessments and how confident they are with the quality of the data? What is LAs confidence in the overall process of using remote assessment for a re-rating inspection?	Section 4.4
7. What are the barriers and enablers to LAs using remote assessment to re-rate businesses? What are the internal (e.g. officer confidence with using remote assessment systems; not having the technology access required; staff resource limitations) and external factors (e.g. connectivity in the area the FBO is located; FBO confidence with using the remote assessment technology; the limited scope within which remote assessment can be used for FHRS reinspection) that may act as barriers and facilitators to the LA using remote assessments to re-rate businesses (gathering the views of LAs who have and who have not introduced remote assessment)?	Sections 4.5 and 4.6
8. What are the impacts on LAs of using remote assessments to re-rate on the cost of delivery? What are the perceived impact on costs for LAs and have these cost savings/increases been or could be transferred to FBOs?	Section 4.3
9. How do LAs think the current criteria in which remote assessment is permitted for re-rating could be improved or optimised? What changes could be made to improve LA experience or confidence in remote assessments?	Sections 4.6 and 6

Study questions for FBOs

Study questions	Sections
1. How do FBOs who have been re-rated using remote assessment view the experience? What worked well and less well?	Sections 4.1.2
2. What do FBOs perceive to be the benefits for them, if any, of using remote assessments?	Section 4.3
3. What do FBOs perceive to be the limitations for them, if any, of remote assessment?	Section 4.4
4. What are the barriers and enablers to FBOs using remote assessments for obtaining a re-rating? Have FBO concerns identified as part of the 2020 work come to fruition (particularly those relating to FBOs’ digital skills and access)? Have the concerns identified have been alleviated or worsened?	Sections 4.5 and 4.6
5. How do FBOs think remote assessments compare to face to face inspections/interventions? How do FBOs experience of remote assessments differ from their previous experiences of face to face inspections? What are the FBOs confidence levels in the overall process of using remote assessment to re-rate? Do the levels of confidence differ (if at all) between face to face inspections and remote assessments? Do FBOs perceive the rating awarded following a remote assessment re-rating inspection to be accurate?	Sections 4.1.2, 4.3 and 4.4
6. How do FBOs perceive the quality / appropriateness of guidance provided to them to conduct the remote assessment.	Section 4.1.2
7. How do FBOs think remote assessments could be improved?	Sections 4.6, 5 and 6

Annex 2: Challenges and mitigation measures

Several challenges were encountered with the proposed methodological approach, and several adjustments had to be made in collaboration with FSA. A summary of the challenges and changes made is provided in the table below.

Desk research

Original approach	To review: FSA documentation on remote assessment LA documentation on internal policies & guidance relating to remote assessment and FHRS FBO documentation on hygiene ratings & re-ratings (either received from LA or internal policies for larger businesses)
Challenges encountered	Very few LAs had used remote assessment to carry out a hygiene re-rating. Where remote assessment is being used, it tended to be informal and in support of a physical inspection. Therefore, limited documentation on this (from either FBOs or LAs) was available.
Mitigation implemented	Where remote assessment is being used informally, we aimed to get clarity on the situations where it was appropriate and how it was administered during interviews Internal documentation from LAs (where available) was reviewed and considered as part of the analysis

Original approach

To review:

FSA documentation on remote assessment
LA documentation on internal policies & guidance relating to remote assessment and FHRS
FBO documentation on hygiene ratings & re-ratings (either received from LA or internal policies for larger businesses)

Challenges encountered

Very few LAs had used remote assessment to carry out a hygiene re-rating. Where remote assessment is being used, it tended to be informal and in support of a physical inspection. Therefore, limited documentation on this (from either FBOs or LAs) was available.

Mitigation implemented

Where remote assessment is being used informally, we aimed to get clarity on the situations where it was appropriate and how it was administered during interviews
Internal documentation from LAs (where available) was reviewed and considered as part of the analysis

Interviews with LAs

Original approach	To carry out 20 interviews with 15 LAs that had used or currently use remote assessment for hygiene re-ratings, and 5 LAs have never used it. LAs interviewed to be sampled accounting for LA type, area type (urban/rural) and region.
Challenges encountered	Fewer than anticipated LAs consenting to participate in the research Very few LAs using remote assessment for hygiene re-ratings overall Confusion over what constituted a remote assessment, leading to inaccuracies in LA survey responses as to whether they had used remote assessment
Mitigation implemented	All LAs that agreed to participate and reported having used remote assessment were interviewed (regardless of sampling) LAs that agreed to participate but had not used remote assessment were sampled LAs not responding to invites to interview were replaced after three reminders

Interviews with FBOs

Original approach	To carry out interviews with 10 FBOs that had received a remote assessment for a hygiene re-rating.
Challenges encountered	Few LAs using remote assessment for hygiene re-ratings meant few FBOs that had received one Unclear to what extent LAs were disseminating information to FBOs to encourage them to participate in research Agreement to participate in the research has been low among FBOs
Mitigation implemented	Extended scope to include exploratory interviews with FBOs about what they thought about the concept of remote assessment for hygiene re-ratings Utilised FSA’s FBO representative association contacts, to assist in identifying potential interviewees

Annex 3: Vignettes

Some of the data collected fed into short vignettes, which demonstrate the specific approaches and perspectives of LAs and FBOs. These have been compiled below.

Box 1 Internal documentation on remote assessment used by LAs

The internal documentation shared by the LAs that were using remote assessment for hygiene re-ratings at the time of interviewing were both designed for EHOs, though they varied in the volume and type of information provided.

Common features in the remote assessment documentation of both LAs were:

Draft emails to FBOs who were being invited to have a remote re-rating inspection - both of which contained a list of documents that the FBO needed to supply in advance of the remote inspection, as well as a caveat that in some cases the FBO may still need a physical inspection.
Circumstances in which a remote assessment could be used – both LAs described it appropriate for FBOs with a minimum hygiene rating of a 4. One LA additionally asked EHOs to assess the suitability of the premises for a remote assessment, based on compliance history, their knowledge of the premises’ layout and their familiarity with the trader.
Criteria as to requirements for a remote assessment: both require it to be done through video call, with one using a specifically developed technology and another using WhatsApp, and both have some practical requirements relating to data storage (one requires an office smartphone to be used, one sets out GDPR requirements that EHOs must comply with during the remote assessment).
An outline of the remote assessment procedure, including specific actions to be taken by EHOs before, during and after the remote assessment (with one LA containing more detail on this than another).

One LA’s documentation also contained:

A detailed description of best practice for undertaking a remote assessment, including practical information about using the specifically developed app, carrying out a technical test with the FBO (ensuring they have good network connectivity and can use the app), the best time to schedule the inspection, hints and tips and a support number for any technical issues or questions relating to the app.
A flowchart illustrating the different stages of a remote inspection.

Box 2 Inspector ShowMe by Scores on the Doors®

During the pandemic, a food network of LAs within a two-tier LA area (i.e. a county LA, with corresponding district and borough LAs) were looking at how they could continue with their FBO controls when they were unable to visit FBO premises in person. This led to a private-public partnership to design Inspector ShowMe: a web-based app, specifically for carrying out remote assessment.

Inspector ShowMe has various features to facilitate remote assessment:

Two-way recorded video calls, with facility to allow guests / other EHOs to join
Ability to take, save and annotate high quality photos during the video call
Automated text extraction (e.g. from documentation and labels)
Text messaging functionality, for notifications to FBOs
Portal for FBOs to upload documentation, as well as for LAs to send information and issue reports to FBOs
Use of geolocation, to authenticate an inspection is carried out at the right FBO location

LAs had used Inspector ShowMe for hygiene re-ratings, but also for ratings (in order to prioritise in-person visits to non-compliant FBOs). Remote assessment for ratings was an infringement of FSA guidance, so they had discontinued its use for this purpose. One LA had used it for controls on beauty premises and animal licensing.

One of the LAs had stopped using Inspector ShowMe because they had been using for ratings, in violation of FSA guidance, and they did not receive a sufficient number of re-rating requests to make paying for a new license worthwhile. During the pandemic, they had access to the software because the county council had purchased licenses for its use for all the district and borough LAs, but this had expired. Another LA had continued to use Inspector ShowMe, including for hygiene re-ratings, but its primary purpose was for controls unrelated to hygiene ratings. This was because they too received a low number of eligible re-rating requests from FBOs. Both LAs would like to see use of remote assessment to be extended to hygiene ratings.

Both LAs were enthusiastic about the software and its capabilities. They felt it had helped them reduce a food hygiene inspection backlog created during the pandemic, improved the work-life balance of EHOs, and aligned with the 'green agenda' and path to net zero. The software itself was considered straightforward to use for both LAs and FBOs, and generally positively received by EHOs.

Box 3 Experience of remote assessment from an FBO perspective

Two interviewed FBOs had been subject to a remote assessment.

FBO 1 was an independent, family-run café with three permanent staff members – a couple and their daughter. Over summer holidays they also employed temporary seasonal staff (usually school children). The FBO received a remote assessment during the pandemic using the Inspector ShowMe technology (described in section 2.2.4). They were very conscious of food hygiene and were proud to have a hygiene rating of 5.

The FBO was given a date and time for their remote assessment in advance. On the day, they received a link in a text message which they clicked on to start the inspection. The couple lacked confidence with technology, but it was easy to follow the instructions of the EHO and their daughter was there to provide technical support. Once the video call started, they walked around the premises, following the instructions of the EHO. The EHO asked them to stop at different points (e.g. to open cupboards, fridges and freezers, point out where raw and cooked meat was handled and cooked, where the hand soap was located etc), and asked questions (e.g. about their food hygiene systems and staff training).

The FBO found the experience a positive one overall. They did not see it as being particularly different from an in-person inspection, other than for the fact it was scheduled in advance rather than unannounced. They had a good relationship with the EHO and knew they could contact her at any time if they had questions, either before or after the remote assessment. However, they felt unannounced in-person inspections were still needed for other FBOs with a history of low compliance.

"Very satisfied [with the process]. Once the video call was on, walking around, opening fridges and freezers, inside and outside cupboards, looking at food prep areas… it was the same as if they were there in person. I wasn’t brought up with computers, and I'm 70 now, that's the only thing – if my daughter wasn't there, I would have asked if [the LA] could do it at a time when one of the school children was working."

FBO 2 was a large, multinational enterprise with around 550 catering and retail units located at UK travel locations (e.g. airports, train stations), employing 1,000s of staff. Units had to notify head office of the outcome of a hygiene rating through an internal system. They required premises receiving ratings of 3 or below to be re-rated. This was not particularly common, and on average it equated to 5-10 re-ratings for a premises each year.

The FBO recalled a couple of occasions where they had received a re-rating without a corresponding in-person visit. This had never involved a video call – instead, the FBO had been required to send documentation and/or photos via email demonstrating that a particular issue had been rectified. The premises re-rated in this way felt positive about it - it was a straightforward process, which had saved the staff members time. Like FBO 1, however, FBO 2 felt sceptical about using remote assessment for less compliant FBOs. They suggested that clear guidelines on the 'minimum criteria' for a remote assessment (e.g. LAs seeing every room from different aspects) would be needed if it was in widespread use, to reassure LAs and FBOs alike of its validity.

"Some businesses might only share photographs that paint them in the best light. So [remote assessment] would only work if all businesses had the same ethos as us - we only want high ratings if it’s deserved – but that is not the case everywhere."

Yn ôl i’r brig