Review of FSA Social Science
Review of FSA Social Science: Annex 2a Assessment of Food and You 2 Wave 4
Review of FSA Social Science: Annex 2a an assessment of research outputs using GSR code and the FSA QAT assessing research reports checklist.
Output: Food and You 2 Wave 4 Technical report and Key findings [combined assessment]
Authors: Key findings reports: Armstrong, B., King, L., Clifford, R., Jitlal, M., Ibrahimi Jarchlo, A., Mears, K.,
Authors: Technical Report: Gallop, K., Smith, P., Ward, K., Horton, S., Candy, D., Hossein-Ali, H., Peto, C., and Lai, C.
Date: August 2022
Assessment of Food and You 2 Wave 4 using the GSR code
Rigorous and impartial
| Rigorous and impartial | Rating | Comments |
|---|---|---|
| Based on sound methodology and established scientific principles | High | The technical report by IPSOS provides extensive evidence of meticulous conceptual and methodological preparation and testing before, during and after fieldwork. The participation of FSA researchers and the FSA Advisory Group in this methodological work is exemplary. |
| Quality assured | High | Quality assurance was certainly built into the survey’s design, execution and reporting, both by IPSOS internally and by FSA researchers. |
| Based on best design, given constraints | High | The survey design, development and execution are excellent. The attention to detail was considerable and the range of questions addressed by the Wave 4 is admirable. |
| Conclusions are clearly and adequately supported by data | High | The conclusions of the ‘Key Findings’ Report of the Wave 4 F&Y2 survey have been clearly presented in the Executive Summary and they are well supported by the data presented in the main body of the report. This has been done in a systematic and rigorous way. It represents the ‘Findings’ or ‘Results’ section of a scientific paper in an academic journal. What is missing is a ‘Discussion’ and ‘Implications for Policy and Practice’ section. (See the ‘Relevant’ section below) |
Relevant
| Relevant | Rating | Comments |
|---|---|---|
| Anticipates future policy issues as well as addressing current ones | High | The Key Findings report, and the Technical Report, are addressing policy issues about food content, food choice and the behavioural drivers of healthy eating. The Key Findings report indicates many areas where the public’s awareness of food content, the food supply chain and policy measures such as the Food Hygiene Rating Scheme is considerable. The Key Findings report also identifies areas where the public’s attitudes about sustainable eating are not as one might have anticipated – for example, “around a third of respondents thought that eating more fruit and/or vegetables (38%), contributed most to a sustainable diet (page 76). These findings might help policy makers at the FSA, Department of Health and DEFRA decide on how current and future food policy issues might be prioritised. |
| Answers clear and researchable questions | High | The Technical and Key Findings reports develop the conceptual and methodological approach of the F&Y2 survey around the policy and behavioural questions of the FSA and DEFRA. Hence, they address clear and researchable questions. |
| Contributes to all stages of the policy and delivery processes | Medium | Contributing to all stages of the policy and delivery process is beyond the scope of the Food and You 2 survey. However, the evidence base that the F&Y2 survey provides does provide a sound empirical foundation upon which discussions about food policy and delivery can be made. |
| Delivers solutions that are viable, actionable and represent value for money | Low | The Key Findings and the Technical Report are less concerned with delivering solutions or determining value for money than establishing the scientific evidence base of the public’s knowledge, attitudes and behaviours in relation to food, food safety and food security. Hence, there are few solutions that are viable, actionable and represent value for money. |
Accessible
| Accessible | Rating | Comments |
|---|---|---|
| Published | High | Publication of the findings of the F&Y Survey has been extensive and of a high quality. The publication of both the Technical report and the Key Findings is clear and well presented. |
| Data made available where possible | High | There is an abundance of relevant and appropriate data from the development and piloting of the study (Technical Report), and the data are presented in a systematic and rigorous way in the Key Findings report. Both reports are easy to read and comprehend. |
| Clear and consistent | High | The technical report is very clear. Though not concise (68 pages) it provides all the technical detail necessary to understand the F&Y2 Survey. It is an excellent ‘back office’ record of the scientific background to this important substantive survey. Although being almost 100 pages in length, the good and clear presentation of the data in the Key Findings report make is clear and readable. |
| Related to existing work in field | High | This technical report builds upon earlier survey work of the FSA and uses the legacy of that work very well. It has also incorporated the USDA 10-item US Adult Food Security module. The Key Findings report also refers well to this other existing work. |
Legal and ethical
| Legal and ethical | Rating | Comments |
|---|---|---|
| Complies with relevant legislation | High | This Technical Report and the Key Findings report comply with GDPR legislation, and both reports are mindful of the legislative framework and obligations within which FSA (and DEFRA) operates. |
| Complies with GSR ethical guidelines | High | Both reports comply with the GSR ethical guidelines for the development, execution, analysis and reporting of scientific research. They go to considerable lengths to meet these requirements. |
FSA QAT Assessing Research reports checklist Food and You 2 Wave 4 Key Findings
Checklist 2: Assessing research reports
Q1. Title, lead author and year
Food and You 2: Wave 4 Key Findings. Lead author: Dr Beth Armstrong, August 2022.
Q2. Has a clear research need been outlined?
Yes – fully - This report builds upon, and replaces, FSA’s face-to-face Food and You survey, the Public Attitudes Tracker and the Food Hygiene Rating Scheme Tracker survey. Hence, a clear research need has been outlined to meet key issues about a range of FSA’s policies.
Q3. Has a precise research question/aim been specified?
Yes – fully - This report addresses a very clear set of questions about consumers’ knowledge, attitudes and behaviours related to food safety and other food issues.
Q4. Is the research design…
Cross-sectional - This report is on Wave 4 of a cross-sectional study
Q5. Is the research method…
Quantitative
Q6. Is there a good match between the research question/aim, research design and research method?
Yes – fully - The match between the research questions/aims, research design and research method is excellent, largely due to careful preparation by FSA social researchers in collaboration with the FSA Advisory Board and the contraction (IPSOS).
Q7. Is the study population and setting specified?
Yes – fully - The study covers consumers in England, Wales and Northern Ireland, and local communities within these countries.
If Q5 = Qualitative, go to Q8a. If Q5 = Quantitative, go to Q8b. If Q5 = Both, go to Q8a and Q8b.
Q8b. Is the sampling method…
Stratified sampling.
Go to Q9.
Q9. Is the sampling method appropriate for addressing the research question?
Yes – fully - The study design is well stratified to include relevant sub-groups on food consumers in each country, and uses the Index of Multiple Deprivation for England (only) effectively.
If Q5 = Qualitative, go to 9a. If Q5 = Quantitative, go to 9b. If Q5 = Both, go to Q9a and Q9b.
Q9b. Has a sample size calculation been conducted?
Yes – fully - A great deal of work has gone into establishing a sample size that is large enough to provide statistical power.
Go to Q10.
Q10. Are the research instruments valid and reliable?
Yes – fully - Again, a great amount of preparatory work has been undertaken by the contractor, working in collaboration with FSA social researchers and other analysis, to ensure that the research instruments valid and reliable.
If Q5 = Qualitative, go to Q11a. If Q5 = Quantitative, go to Q11b. If Q5 = Both, go to Q11a and Q11b.
Q11b. Is the analytical approach…
Chi-square test, correlation test, t-test or analysis of variance
Go to Q12.
Q12. Is there a good match between the analytical approach, the research method and the research question?
Yes – fully - This report uses mainly percentage differences in consumers’ knowledge, attitudes and behavioural responses across socio-demographic and sub-groups. Differences of 10 percentage points between groups are mainly reported and are tested for statistically significance at the 5% level (p<0.05). Some differences between respondent groups are included where the difference is fewer than 10 percentage points when the finding is notable or of interest. This provides a good match between the analytical approach, the research method and the research question.
Q13. Has a relevant checklist from the EQUATOR Network been used in the reporting of the results?
Yes – fully - This report meets the guidelines of the EQUATOR network for quantitative methods. That is, there is a basic presentation and analysis of results using descriptive tables and figures (e.g., histograms) that are useful for conveying the results. The tables and figures are concise and easy to read. The statistical commentary brings the numbers to life and does so in a fairly simple and straightforward way. Important differences between different socio-demographic groups are noted in the commentary.
Q14. Have descriptive data on the characteristics of participants been presented?
Yes – fully - Considerable descriptive data are provided on the characteristics of participants both in this ‘Key Findings’ and in the accompanying ‘Technical Report’.
If Q5 = Qualitative, go to Q15. If Q5 = Quantitative, go to Q19. If Q5 = Both, go to Q15 and Q19.
Q19. Have descriptive data on exposures/interventions and potential confounders been presented?
Yes – fully - The report provides data on respondents’ awareness of FSA’s programmes and policies aimed developing and maintaining food standards (which are the ‘exposures and interventions’ of this study). It also analyses the contribution of other government programmes, such as the Healthy Starts voucher programme, to consumers’ knowledge, attitudes and behaviours.
Go to Q20.
Q20. Have unadjusted and adjusted point estimates and confidence intervals been presented alongside statistical significance?
Yes – partly
Yes – fully
This report mainly presents percentage distributions of respondents’ knowledge, attitudes and behaviours. This provides an indication of the range of results which have been weighted to reflect socio-demographic factors. This is not the same as presenting unadjusted and adjusted point estimates and confidence. However, it does make the presentation of the findings easy to read and comprehend, especially for those who may be less oriented towards statistical detail.
Go to Q21.
Q21. Has generalisability been considered in the interpretation of the results?
Yes – fully - This report provides sufficient data on how generalisable the findings are across the populations of England, Wales and Northern Ireland. The report clearly points out that the data are “typically reported only where the absolute difference is 10 percentage points or larger and is statistically significant at the 5% level (p<0.05). It also reports that “some differences between socio-demographic and other sub-groups are included where the difference is less than 10 percentage points, when the finding is notable or judged to be of interest.” This enhances the generalisability, and limitations, of the findings.
Q22. Has causality been considered in the interpretation of the results?
Not applicable - This is not an experimental or quasi-experimental study. Hence, it does not attempt to provide evidence of causality. The data presented are correlational rather than causal.
Q23. Has uncertainty been considered in the interpretation of the results?
Yes – fully
Q24. Has a clear study conclusion been presented?
Yes – fully - There is an excellent ‘Summary of Key Findings’ at the beginning of the report. These are well presented with clarity, and in large print that is nicely spaced, over just five pages.