Review of FSA Social Science - Chapter 6 Conclusions and Recommendations

This review has attempted to assess the contribution that the FSA social science team makes to the FSA and its mission, and to identify what it does well, where there may be need for improvement, and what might be the direction of future learning and professional development.

What does the social science team do well?

1. The FSA’s social science team is a confident group that is well-regarded by the majority of internal and external stakeholders, and provides a robust evidence base on the social aspects of the FSA’s mission to ensure that food is safe, is what it says it is, and is healthier and more sustainable. 

2. The team was described by stakeholders as having “a huge amount of in-house expertise” and providing a “variety of really interesting high-quality research that contributes towards the evidence base for our policies”. This included the “ability to provide quick and detailed comments on things where input was required”, and “some really strong areas of consumer insight that allows us to make very powerful statements as an organisation”.  
3. The team’s qualitative analysis and their evaluation of the equality issues surrounding food policy and food insecurity were highly valued, as were its exploration of consumers’ attitudes and perceptions on food products. The ‘lived experience’ research of the team was also highly valued.  
4. The team’s ability to identify and clarify the problems to be researched or evaluated, and to articulate the business needs for research or evaluation, was also recognised positively by internal and external stakeholders. So too was their capacity to challenge policy colleagues and contractors in order to specify research and evaluation that can be delivered in a timely manner and with quality.
5. The research outputs reviewed for this report were found to be generally of a high standard in terms of the methodologies used, their design, execution and reporting. 
6. The project management of the FSA social science team was generally good and appreciated by policy colleagues and contractors alike.
7. The social science team meets most of the GSR Code for Products and People well. The team and its work were rated ‘Green’ or ‘Green/Amber’ against measures of: rigour and impartiality, relevance, accessibility, legal and ethical practice, performing role with integrity, being appropriately skilled and continuously developed, and outward facing.

Where is there room for improvement?

The data and analysis presented in this review has identified areas where there is room for improvement.

1. The involvement of the team with policy colleagues does not always happen early enough. Consequently, research objectives, questions and approaches can be ill-defined or considered too late for appropriate specificity, delivery and quality assurance. 

Managing expectations of policy colleagues regarding how long it takes to procure, deliver and quality assure research to high standards can also be problematic. Resolving different stakeholders’ opposing demands or conflicting needs for social research can present additional challenges for procuring and delivering high quality research.

Recommendation 1: Heads of profession from all of the Government analytical services should establish with senior policy makers the importance of early and continuous involvement of researchers in the development of policies, programmes and projects.

2. Procurement procedures for research have been identified as a challenge for some contractors and the social science team. They have been described as “inappropriate for procuring research and evaluation” and “not fit-for-purpose”. These procedures are cross-government requirements and beyond the control of the FSA social science team or other analytical professions across government. 

Recommendation 2: The procurement arrangements for government research should be reviewed with the aim of having separate procedures and requirements from those of general government procurement.

3.    Identifying the impacts of social research outputs is something the social science team would like to improve. This requires not only monitoring the uptake of the FSA social research team’s outputs, but also identifying and assessing their effects on dietary and hygiene-promoting behaviour. 

Recommendation 3: The senior management of the social science team should identify ways in which the impacts of their research and analysis can be identified and evaluated. This might involve linking the data and findings of the FSA’s social research outputs to other sources of data on dietary behaviour.

4.    This review has identified that not all published research outputs have a separate technical report providing in-depth details of how research projects are designed, samples and research instruments are selected, or how analysis will be undertaken. 

Proportionality in the provision of such technical details is an issue, especially given the pace at which research has to be commissioned, delivered and quality assured. However, providing the technical basis of research outputs is one of the hallmarks of good science and should be common practice.

Recommendation 4: Technical details of how research has been conducted should be made available as common practice if the scientific quality of research outputs is to be assured. This is also a matter of transparency and accountability.

5.    External peer reviewing and quality assurance of research outputs is “undertaken on a case-by-case basis, subject to the pace, complexity and purpose of the work” ^{(footnote 1)}. This is usually undertaken by the FSA’s Advisory Committee for Social Science (ACSS) and academics on the FSA’s Register of Experts. 

Proportionality is also an issue in the provision of peer reviewing of research outputs. External peer reviewing, however, is another hallmark of good science and should be common practice. Where and when research outputs are for publication, and/or a potential evidence base for decision making, then wholly independent peer review should be a matter of good practice.

Recommendation 5: All published social science outputs, and those that will provide a potential evidence base for decision making, should be peer reviewed by wholly independent experts as a matter of good practice.

6.    There is some need for professional development of some members of the social science team. This includes professional development in methodology, leadership and coaching to stay abreast of the latest methods of social research and research management. 

It is understood that plans are underway for some professional development of the social science team. The professional development of social researchers would also benefit from greater flexibility in selecting training courses at the appropriate level of technical expertise from organisations within and outside of the CSL prospectus.

Recommendation 6: All members of the team should be able to take the CPD training of their choice, within or outside of the CSL provision, to keep informed of the latest developments in research methods and coaching. There would seem to be a particular need for CPD of quantitative methods of research, systematic review methodology and the coaching of staff.

7. Most FSA social research is outsourced, so most of the team’s work is commissioning, managing and quality assuring research, rather than undertaking data collection and analysis. This may have some limiting effects on social researchers’ abilities to maintain and develop their skills in research and analysis. 

Recommendation 7: The FSA social science senior management team should review the balance of in-house versus contracted-out social research and ensure that all members of the team have the opportunity to maintain and improve their social research skills.

8. The GSR Self-Assessment Tool aims to ensure that GSR’s professional standards are met in all of its products and people. It does this at a rather high level of generality and the indicators do not really capture the quality of research outputs at a sufficiently granular level. 

The Self-Assessment Tool also requires all research products to be assessed in terms of their implications and solutions for policy and delivery. This is often beyond the scope of most external contractors. 

The GSR Self-Assessment might best be used to assess the broader dimensions of professional standards, such as the accessibility, legal and ethical requirements, and recruitment and professional development. This, however, depends on the extent to which the GSR Self-Assessment code is currently used to assess the professional practice and outputs of social science in government. This review suggests that this may not be extensive.

The scoring categories of red, amber, green were seen as insufficiently nuanced and as conflating standards. This is especially so in terms of assessing the rigour of social science methodology. The FSA Quality Assurance Toolkit was found to be more appropriate to appraise the rigour of research outputs as well as the production, assessment and procurement of research.

Recommendation 8: It is strongly recommended that going forward the FSA Quality Assurance Toolkit should be used as the main means of assessing the quality of social science research at FSA and in other government departments.