Appendix D: ESRG interview topic guide

Specifically, the evaluation will look at how effective the processes involved in developing the new system have been, and the efficacy of the package developed, as well as the effectiveness of its implementation.

My name is [Name of interviewer], and I am an evaluator from the RSM research team. Thank you for agreeing to participate in this interview. Is now still a good time to complete the interview? [Proceed if yes, reschedule if no]

As part of this evaluation, we are hoping to interview ESRG members and FSA/ FSS stakeholders involved in developing the reforms and support package. The purpose of these interviews is to explore:

• the rationale for system change (including the policy context and problem statement) and the development of objectives to address needs
• the process for developing the new system and package. How effective was the system redesign?
• whether stakeholder engagement was sufficient and effective
• how effective is the system redesign in meeting objectives?
• programme management by the FSA and FSS and governance arrangements
• recommendations for case studies
• any further points you would like to raise.

This discussion should take 45 to 60 minutes via MS Teams. Your comments will be completely anonymous and confidential, will be stored securely by RSM, and will not be attributed to you in our final report. [Informed consent taken – check participant understands how their data will be processes and check if they have any questions. Gain explicit consent for audio recording of interview. Confirm confidentiality – for example, no individual names used in reports or outputs].

Background questions

1.    Please can you describe your role and responsibilities and how this relates to the programme of activity?
2.    What is your understanding of the overarching aims of the new system and package of support?
[Next, I’m going to ask you about your reflections on efficacy of the internal programme process.]

Rationale for system change

[In this section, we want to understand the strength of the evidence base and how it informed system re-design.]
3.    Please describe from your recollection the evidence base which informed development of the four workstreams and the rationale for each workstream ie. what were the problem statements the ESRG was seeking to develop solutions for? Nb. Stakeholders may only be able to answer for the workstreams they were involved with.
4.    Linked to the question above, can you recall the objectives for reform across the 4 ESRG workstreams)? Nb. Stakeholders may only be able to answer for the workstreams they were involved with.
5.    In your view were you happy that the evidence was sound and can you recall the recommendations? Were any best practices identified relevant to the UK context (or were there challenges translating these to the UK context)?
6.    How were insights and pilot approaches with industry, used to inform design/ implementation?

Programme delivery and partnership approach

[Please answer the following questions for the period 2016 up to launch in 2019.]
7.    In Your view do you think the governance structures were effective? Were governance and management structures for the programme of work effective in system re-design? Any thoughts on what worked well or less well?
8.    How do you think the partnership approach worked? Any thoughts on what worked well/ less well?
9.    Which workstreams were you involved in?
a.    What was the purpose/ intention of each workstream? For example, What were the aims and objectives? 
b.    What planned outcomes did you hope to achieve?
c.    What activities did you undertake?
d.    How were tools, guidance and processes decided on?
e.    What outputs/ outcomes did you achieve?
f.    Were approaches piloted (for example, to assess effectiveness) before rolling out?
10.    In your view, could more have been done to engage stakeholders? Were industry/ enforcement authorities/ consumer groups consulted/ engaged with effectively? Why do you say that?
11.    Can you think of any improvements that could have been made during the design and consultation period?

Efficacy of the system redesign and package

[Please answer the following questions for the system and package launched in 2019.]
12.    In your view, does the new system and package meet needs expressed in the problem statements for each workstream (ie. have reform objectives been met)? (please note: this question is focused on design, not implementation).
13.    Is the withdrawal and recall system launched in 2019 founded on a clear and distinct set of roles and responsibilities, agreed, and commonly understood by all participants? Why do you say that?
14.    Is it more likely that information provided to consumers is consistent and accessible, based on proven best practice and underpinned by cross industry sharing of approaches and impact? Why do you say that?
15.    Are the public more likely to be aware of the recall process and what actions they should take? Why do you say that?
16.    Do feedback loops and a commitment to continuous improvement amongst stakeholders underpin the withdrawal and recall system? Why do you say that?

Suggestions for potential case studies

The evaluation team will be undertaking case studies of FBOs, to gather evidence of system effectiveness. We are looking for 8-10 case studies of post 2019 food incidents and 2-4 hypothetical case studies of industries which are at risk of highly impactful recalls/ withdrawals.
17.    Are you aware of any local enforcement teams or industries doing particularly good work in helping to implement the reforms?
18.    Are you aware of any recent incidents, which benefited from the new package and could be a case study? 
19.    Are you aware of any major challenges / risks facing particular industries in the food and drinks sector, that might form the basis of a hypothetical case study?
Thinking to the future
20.    Going forwards, should the programme continue to be delivered as it is, or are further improvements required? Why do you say that? 
21.    Is there anything else you would like to raise in our discussion today? 

Thank interviewee for their time.