Review of FSA Social Science: Annex 3A GSR Code self assessment

Project specs are always reviewed internally – at least by G7, often G6.

External peer reviewers may be appointed at the beginning of many projects, and review the project specs.

This isn’t currently applied across every single project: a decision is made on a case-by-case basis subject to the pace, complexity and purpose of the work.

Risk that the quicker turnaround project designs aren’t reviewed to the same extent as other projects. This may be appropriate as there is a need for proportionality, but there could be a more consistent, formal process to decide which projects need external peer review, and for which internal review is sufficient.

Processes are in place for example,: ACSS QA Gateway for (almost) all new projects (there are some exceptions to this (for example, very low value and/or quick turnaround survey work)

The new ACSS QA gateway process may bring this up to a “green”, by ensuring that new projects are externally reviewed at design stage.

Recently have begun uploading project plans to the Open Science Framework: https://osf.io/

Quality assurance of outputs

RED: No formal procedures in place to quality assure outputs.
AMBER: Processes in place for internal quality assurance but no formal mechanisms in place to ensure that outputs are appropriately independently reviewed.
GREEN: Processes and guidance in place to advise on appropriate quality assessment at completion, including external and internal peer review as and when necessary, to ensure that methods and analysis have been rigorously executed, and conclusions are clearly
and adequately supported by data.

Most project outputs are externally peer-reviewed by the ACSS or the FSA’s Register of Experts.

This isn’t currently applied across every single project: a decision is made on a case-by-case basis subject to the pace, complexity and purpose of the work. When there is sufficient capability in- house, outputs are quality assured internally.
Some work has its own Steering / Advisory Group.
Quality assurance also sought from cross- governmental colleagues working on similar issues (for example, behavioural Handwashing project review by cross-government handwashing group)
 

Register of Specialists and ACSS demonstrate well established mechanisms for seeking peer- review.

The new UCL Quality Assurance  Toolkit is a further opportunity to embed good
practice
 

We regularly monitor wider food trends and compile any stories that are of interest in a document on our shared drive. Every few weeks, we circulate the most interesting / relevant news stories across AU and also do an internal Yammer post.

We conduct timely research on our rapid testing platform via the call off contract with Ipsos, for example, Alt Proteins report, Healthy sustainable diets report.

We have started doing a monthly publication for the consumer insights tracker, increasing frequency so that customers can always quote the latest figures.

We also have a suite of long-term tracking surveys, for example, Food and You 2, Consumer Insights Tracker, FBO Tracker.

We led and contributed to various pieces of FSA horizon scanning work in 2020 and 2021 for not only the short term but the medium to long term for example, FSA Covid-19 Horizon Scanning, social media listening

We delivered a whole programme of work at pace to meet evidence needs during Covid, for example, the Covid tracker and the handwashing tracker. 

To inform the development of the new 3rd pillar of FSA’s strategy, we have published evidence reviews and research on healthy sustainable diets and developed new questions added to F&Y2.

Performance resources report which utilises Food and You 2 and Consumer Insights Tracker data

Our data contributes to the published Annual Report and Accounts

We conducted an internal piece of research to improve FSA Diversity and Inclusion Evaluation Action Plan published. 

We produced a whole tranche of work in good time to inform the new third pillar of new FSA Strategy (for example, Public interests needs and concerns around food)

Liaising with policy stakeholders on F&Y2 questionnaire to cover emerging areas of interest,
for example, GE, GM foods

SS team member on each of the cross-FSA Area of Research Interest (ARI) Steering Groups. 

Regular catch ups and key OGDs/research funders for example, Defra.

Portfolio/ARI Steering Groups comprised of the policy stakeholders sign- off the bid for funding for the project.

Investment Board (IB) process – all projects have to be signed off at Director level for funding approval.

Our commissioning process – policy teams approach us to conduct work

ACSS QA Gateway signs off new work design

We get sign off from policy area leads for final outputs, for example, fieldwork materials, reports Future Publication Panel (FPP) process requires project reports to be signed off by a DD panel for all publications not considered low risk by head of SERD.

We brief the top of the office on new publications, including the
research methodology and its limitations, so CEO/CSA Chair have sight prior to publication. 
 

EPI forms help us establish impact throughout the lifecycle of a project.

Social Science Impact Log, which tracks our dissemination activities and how our data has been used.

Food and You 2 feedback survey sent to policy and other stakeholders (not yet sent – draft survey uploaded as evidence).

Tracking DOIs

Engaging with external research users.

Post project review of impact – following up with policy to understand which impacts we anticipated have been realised / which recommendations implemented. 

Published

RED: Outputs are not routinely published according to GSR Publication Guidance. Embedding the guidance is an aspiration but there is still work to do on promoting and integrating it within the organisation. No record of exceptions is kept. No clear communications plans.
AMBER: Outputs are usually published according to GSR Publication Guidance but some work to do on promoting its use and embedding it into organisational procedures. A record of exceptions is kept but this is not comprehensive.
Communications plans in development. GREEN: Research outputs are routinely published to time according to GSR Publication Guidance, and all exceptions are recorded. GSR Publication Guidance
is promoted and integrated into organisational procedures. Clear communications plans in place. 

Social research reports published on the FSA website under ‘Research reports’ in line with accessibility guidance (now in HTML format).

Social research reports and data tables are published in line with accessibility guidelines. The FSA Intranet has a whole area and pages dedicated to accessibility, with guidance and templates (to use internally and share with contractors) and training videos. We also have a folder in the Social Science Teams channel with accessibility guidance, resources and templates. All members of the social science team have attended accessibility training (provided by the FSA comms team), and refresher sessions and lunch and learns are run frequently.

Procurement specification advice can be found on the Intranet, and procurement specification templates are shared with contractors which states that: in line with the Government’s Transparency Agenda which aims to encourage more open access to data held by government, the Agency is developing a policy on the release of underpinning data from all of its science- and evidence-gathering projects. Data should be made freely available in an accessible format, as fully and as promptly as possible. 

Project officers update the SERD publication tracker on a monthly basis with estimated dates of publication to track research publications and comms.

All publications (unless granted special exception) go through our Future Publications Panel process to ensure publication is handled appropriately and received sign off from each Director. This process also checks for assurance of accessibility (see FPP form)

DOI (Digital Object Identifier) codes are assigned to each published report to help users to identify the object online (for example, publication in online journals) and act as a unique persistent identified to publications. 

FSA research is published as  standard in line with our open and transparent principles, not always but usually within 12 weeks of receiving final output as per GSR publication guidance.
The majority of our research has a clear communications plan in place, where a comms plan is not in place there is usually a clear reason for this.

We usually have a rationale for when publications do not follow the GSR publication principles, but our exceptions are not typically recorded.

We could do more to promote the GSR publication guidance internally, and ensure it’s used.
GSR share the guidance with Perm Secs every so often to remind them. It’s sometimes hard to make this a reality – though as a non- ministerial department this isn’t too big an issue.

Principle 2 of publication principles guidance states: Protocols and analysis plans should usually be developed and published in advance of any study being started. We don't usually do this (although trial protocols were published on Open Science Network). 

Example comms plans from recently published research projects: The UK Public’s interests, needs and concerns around food
A rapid evidence review on sustainability

Working groups / workshops with FSA colleagues across different teams, with reports shared and published.
For example the Climate Change and Consumer Behaviours workshop with colleagues from Policy and Strategic Insights. 

All data published via FSA data catalogue, or for smaller projects data is available on request (F&Y2 data also published on UKDS).

A quarterly FSA Science newsletter is sent out to around 700 external stakeholders to flag research and science news.
Example from June send out saved here. This demonstrates our
work on the sub-principle around 'relating to existing work in field'. Our SERD stakeholder list also enables us to connect with the wider network (OGDs, academia, consumer, trade associations, learned societies, research agencies etc.)

Cross- governmental networks and workshops (e.g. upcoming Household Food Insecurity workshop) and ACSS working groups to share learning and ensure our research is accessible and known (related to
existing work in field). 

Format

RED: Outputs are not sufficiently accessible for all stakeholder groups. Local guidance for authors is not available.
AMBER: There is a move towards clear, concise and more accessible outputs but practice is inconsistent or in development.
GREEN: All outputs are written in clear, concise and jargon free language with short summaries routinely produced. Key messages are easily identifiable and understood. Guidance is available for authors of reports and enforced by GSR project managers.

Transparency policy (published on the Intranet here) shows that openness is one of the core values of the FSA.

All research reports are published as HTML now and checked by Samantha Merrett (our accessibility lead in Comms) prior to publication and data tables are shared with John Clowes (data team) for publishing on the FSA data catalogue. Our KIMS team also assess each new dataset for suitability of publication using an online form (process outlined here).

The Welsh language policy published on our Intranet details how we are legally obliged to 

provide all services in Welsh. We liaise with the Welsh Translation Unit to check requirements when publishing (N.B. not always required but a part of the process).

All reports routinely include an ‘Executive summary’ so that readers can identify the key messages. Larger projects such as Food and You 2 and Kitchen Life 2 have webpages that summarise the project so that readers do not have to download the full report.

Example reports with executive and/or short summaries included:

The UK Public’s interests, needs and concerns around food (Executive summary included at start of report)

Food and You 2 (main findings summary included on webpage in addition to full final report and technical report. Executive summary also included at start
of report).

Consumer Insights Tracker.

Rapid evidence review on sustainability is an example report with both an executive summary published as part of report and a standalone 3- page summary for internal use to key messages are easily identifiable and understood. 

For all externally commissioned work, ethics are required in the research specification to be considered by tenderers using the GSR ethical guidelines. We highlight particular areas and additional sensitivities for consideration.

We have, on occasion, sought external advice from an ethical advisory. In addition, our recent behavioural trials have used Kantar’s ethical panel. And had extensive ethical procedures for for example, Kitchen Life 2 

Ethical issues are included in our discussions regarding new work at the ACSS QA Gateway.

Principle 1: Research should have a clear user need and public benefit
•    Investment Board Process for approval of research projects to ensure they are it offers value for money and aligns with the needs of the organisation
•    Policy of transparency - all research is published and presented to internal FSA stakeholders. Research methods used are outlined in all publications along with findings.

Principle 2: Research should be based on
sound research methods and protect against bias in the interpretation of findings
•    ACSS Steering Group - all new research projects are reviewed by the ACSS to ensure that the research questions and methods used are sound.
•    QAT asks some high level questions about methods and justification

Principle 3: Research should adhere to data protection regulations and the secure handling of personal data
•    PIA form completed for some projects
•    Data protection processes form part of the tendering process
•    Data protection response for call off contract checked by
call off contract lead

Principle 4: Participation in research should be based on specific and informed consent
•    Informed consent is obtained for all research projects which are conducted.
•    Good example from KL2 where multiple layers of consent were gained in businesses.

Principle 5: Research should enable participation of the groups it seeks to represent
•    Aim to gain a representative sample based on geography, SES, ethnicity, age.
•    F&Y2 have paper based and web data collection options

Principle 6: Research should be conducted in a manner that minimises personal and social harm

•    Currently research is predominately conducted with those who are less at risk of harm. When conducting research with businesses/ organisations where case studies are used- provided with opportunity to review to ensure that reputational harm is not being caused.
 

For GSR ethics principles:

1.    None
2.    None
3.    New team members may not be aware of PIA guidance. Team training session to be arranged
4.    None
5. Considerations for enabling participation for those who are harder to reach (for example, those who don't have internet access those with English as a Second Language)

6. Agree processes for engaging with vulnerable groups and for asking questions on the sensitive topics listed. 

Training on aspects such as 6 point plan and support for researchers (both internal and external).
 

1.    Research is prioritised through ARI steering groups to ensure projects with a clear policy and business need are prioritised.

2.    Methods reviewed through ACSS. Research work packages/ protocols signed off prior to work commencing.

3.    Data protection part of tendering process. PIA process in place, with guidance on when this should take place.

4. Ethics included in tendering process. There is a check list for the privacy notices and the participant information sheet.

5. Our call off contractor always offers to provide tablets/ dongles for those who are not web enabled so that they can take part in research. FSA has a Welsh translation unit for surveys to be translated if requested. 

Procurement

RED: Social research is not consistently procured in line with GSR Guidance on the Procurement of Government Social Research. Good practice is not followed to reduce burdens to suppliers or maximise value for money and use to the customer.
AMBER: Inconsistent application, or knowledge of, GSR Guidance on the Procurement of Government Social Research and application of good practice.
GREEN: GSR research is conducted in line with GSR guidance on the Procurement of
Government Social Research, backed by departmental procedures and the advice of procurement experts. Unless there is a strong, justifiable reason, all contracts are awarded as the result of competition. Good practice in procurement is followed to ensure that processes are proportionate, burden on suppliers and commissioners is reduced as far as possible, the supplier market is developed, and processes are operationalised to maximise value for money and to meet the needs of policy customers.

Unless it is not possible due to time constraints, all tenders go through the full competitive tendering process.

Procurement team are involved in all tendering processes and moderate the panels.
Individual scores are then agreed by consensus.

High value tenders have an external impartial expert on the panel to help ensure quality and value for money. 

Data security
RED: No systems or guidance in place for handling, storing and sharing data. There is concern that GSR members are not fully aware of their responsibilities.
AMBER: Systems and guidance in development, GSR members’ awareness of data security being addressed as a priority.
GREEN: Systems and guidance in place for handling, storing and sharing data in line with CO core minimum standards and relevant legislation. GSR members are fully aware of their responsibilities and actively manage contractors’ data security and handling processes.

Systems and guidance in place for handling, storing and sharing data

All contractors must provide details of how they will securely store, process and share personal data in their tender application form/ work package response. 

Business case processes for example, VFM

ACSS QA gateway process for example, VFM

Procurement processes for example, VFM Work with policy stakeholders to understand existing evidence base.
Conduct REAs/literature reviews to understand evidence base and evidence gaps and use this to inform commissioning.

Link across projects to ensure making best use of commissioned work (for example, secondary analysis of kitchen life 2 data, citizen science linking up with research ARIs /
stakeholders).

EPI form, which includes elements on making best use of available resources Steering groups to ensure cross-org knowledge is leveraged.

Knowledge management

RED: Knowledge management is patchy with no mechanisms in place to encourage GSR members to keep up with, share and retain knowledge (including research/methodological/policy developments) at individual or organisational levels.
AMBER: Most GSR members undertake knowledge management activities, including keeping up with research/methodological/policy developments relevant to work area. No mechanisms yet in place to help promote the sharing and retention of knowledge at the individual or organisational levels.
GREEN: GSR members routinely keep up with emerging research/methodological/policy developments relevant to work area.
This activity may be written into their objectives. Mechanisms are in place to ensure that organisational knowledge is
shared and retained as appropriate.

Social Science Library to capture existing research

L and D channel library of resources

See section on ''Appropriately skilled and continuously developed''

Ongoing maintenance of resources

Degree to which GSR members are embedded into policy areas (for example, invited to relevant meetings which don’t have explicit research needs, have documents shared etc.)

Production of summary papers of existing FSA research in relevant policy areas (for example, FHS consumers' information preferences, labelling.

Membership of cross government groups (for example, on behavioural research and evaluation) 

Open, fair and honest
RED: GSR staff are not given appropriate support and guidance to enable them to earn trust and respect of users of government social science research, research participants and the wider public. Appropriate mechanisms do not exist to ensure learning received from stakeholders is acted upon.

AMBER: GSR staff are given some support and guidance in this area, and learning is acted upon in most cases, however, more could be done to support staff and to develop skills. There is some concern that GSR staff are not always making the right judgements in how they deal with stakeholders.
GREEN: GSR members work to gain the trust and respect of users of government social science research, research participants and the wider public, and are given appropriate support and guidance to do this. As a profession, GSR deals openly and fairly with research customers and other stakeholders, acting upon feedback and information received. GSR members use good judgement to balance rigour and relevance, build constructive relationships within and outside their profession, and perform their challenge
role appropriately.

Insights are published. We measure trust in the FSA and track the reputation of the FSA (FSA Reputation tracker, Food and You 2)

Membership of cross-government groups (for example, on behaviour research and evaluation)

Disclosure of data and reports for transparency. 

In primary research conducted by the team, we follow best practice principles as laid out by bodies such as the GSR, SRA and MRS when engaging with research participants (for example, ensure suitable measures in place for data protection, anonymity etc.)

Use of stakeholder list (OGD, academics, NGOs) ensures we are open in communicating insights

The individual, with the support of their line manager, are expected to produce their own development plan

We have made an active choice not to conduct a skills audit.

Individual CPD logs

On a team level and individual level we have a lot of CPD and have now created a team evidence log for this

L&D Teams chat channel to share events. Feedback given to the rest of the team from individuals regarding L&D. 

Career and talent management

RED: HoP has little or no control over facilitating/supporting career moves of GSR members. Little guidance offered to members about career management and how to gain broader/deeper experience within/external to the organisation. Little/no promotion of leadership development amongst GSR members at any level. Access to talent management opportunities is limited.
AMBER: HoP has some influence in facilitating/supporting career moves of GSR members but in organisational context more could be done.
Opportunities such as leadership development, talent management, broader experience and take-up of generic analyst roles are available but these could be better promoted.
Promotion of leadership skills development focuses only on senior members.
GREEN: HoP plays active role (as appropriate) in managing the career development of GSR members and
promotes opportunities to develop and demonstrate leadership skills at all levels within the GSR community. HoP identifies and develops those with the potential to progress to senior positions, including SCS. Access to talent management opportunities is open and transparent. Other opportunities e.g. for broader/deeper experience, or take-up of generic analyst roles, are identified and promoted in the context of career management.

HoP, with team, enables all individuals to gain experience of different work areas (as best as possible subject to business needs) as well as line management opportunities

Fast Streamers have successfully rotated within the team to date

2 members of staff have recently been seconded and loaned to other departments.
 

Managed moves are not really feasible due to workload / resourcing pressures

The small and centralised nature of the team means there are limitations on internal career moves (for example, we have no GSR SCS)
 

Regular career conversations with team members as part of line management responsibilities,
for example, Talent and career conversations,
for example, Mentoring

All GSR members are encouraged to play an active role in X-govt groups & GSR working groups. Several are currently active.

Access to talent management opportunities,
for example, Civil Service
talent and accelerated development schemes.

Badging exercise will be held in 2023 for non GSR members. 

All team members (including non-GSR) are constantly using research skills and knowledge whether via project delivery or providing advice to policy.

Where possible (subject to making the best use of limited resources) staff conduct research and analysis. 

External research community: 

RED: Some links with the external research community pursued but not very well developed. No plans in place to identify more. Potential for external
engagement not realised.
AMBER: Some well-established links with external research community. Further links identified and plans may be in place to follow up, but not systematic. More emphasis needs to placed on this kind of activity for example, engaging with both
individual external experts and external
research bodies.
GREEN: GSR members engage actively in developing links with the external research community in their work/policy area. Strategic plans in place to support
this activity, for example, external social research
advisory committee in place, joint funding of research programmes or centres with external research bodies, consulting externally on research programme. 

Other government analysts

RED: Few opportunities for GSR members to work with, or gain knowledge of, the other analytical professions. Limited engagement with other analytical HoPs.
AMBER: Some notable interaction and working by GSR members with the other analytical professions. Mechanisms for planning or facilitating joint working are in their infancy. Established contact/engagement with other analytical HoPs.
GREEN: GSR members are aware of the contribution of the other analytical professions, know when and how to engage them and there are good examples of working together where appropriate. Mechanisms for the joint planning of analysis are well in development or have been implemented. Routine and constructive contact with other analytical HoPs.

Regular engagement with wider Analytics Unit (AU) through regular unit level meetings with other analytical professions (and their HoPs) including statistics, operational research, economics and strategic insights (intelligence)

Collaboration with other analytical professions within the FSA on joint projects with other professions for example, consumer insights tracker, F&Y2, Kitchen Life 2, evaluation work

Regularly seek out other professions for advice or support when appropriate for example, statistics, economics

Collaboration with other analysts across government in the UK (for example,
Defra, FSS, DHSC) on
research projects and cross- government reports, and internationally (through the ISSLG)
 

Research and Evidence Programme Steering Groups in place allow close working relationships with
Policy. Project working groups in place to support collaboration with Policy on research projects from start to finish.