FY2 Wave 6 Technical report: Fieldwork and response rates

Letters and reminders

Letters and reminder strategy

The mailing approach followed Ipsos’ standard push-to-web methodology:

1. An initial invitation letter was issued to sampled addresses inviting up to two adults to go online and complete the online questionnaire. This letter was mailed on the 10th October 2022.
2. The first reminder letter was issued on 21st October 2022 and began to arrive at sampled addresses on 1st – 4th November 2022. Reminder letters were only sent to non-responding addresses and addresses where one adult had completed the online questionnaire but not a second adult (the presence of an eligible second adult was determined in the first questionnaire). 
3. The second reminder letter was issued on the 11th November and began to arrive at sampled addresses on 16th November with the first postal returns logged on 27th November. The second reminder was only sent to non-responding addresses and addresses where one adult had completed the online questionnaire but not a second adult. All of these letters were accompanied by one or two postal questionnaires, to allow those who could not access the internet, and those who may have been less comfortable completing online questionnaires, to take part. Those in Wales received one questionnaire in English and one in Welsh. Further detail is provided in the section on the postal questionnaire. 
4. A final reminder letter was issued on 7th December 2022, arriving at sampled addresses on 10th December 2022. The survey remained open until 10th January 2023.

There were numerous Royal Mail postal strikes during Wave 6 fieldwork, which had an impact on how quickly letters arrived.

Letter design

The principles for designing the invitation and reminder letters, which were kept substantially the same as those used for previous waves, were primarily based on the Tailored Design Method, which was described in Dillman, DA. Smyth, JD. Christian, LM. (2014) Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method . A host of other literature and best practice based on previous studies (mainly the Active Lives survey and Labour Force Survey) were also used to inform the design. The main aim of the letters was to provide all the relevant information a participant requires to complete the survey, and to answer immediate questions which they may have. 

Our guiding principles for designing the letters were:

Use simple and easy to understand language, with no unnecessary complicated text
To cover key messages that needed to be conveyed in the letters including:

(a)    Importance
(b)    Motivators for taking part
(c)    How to take part
(d)    Your personal data are safe
a)    Importance was conveyed in all four letters in the following ways:

• FSA and Defra logos were prominent
• visual clutter which could distract from the logos and the importance of the survey was avoided
• professional letter format with address of recipient and full date
• signed by someone with authority (signified by their job title and organisation details) 
• highlighted key messages in the text; using these to break up the text made it easier to read

b)    The main motivational statements varied across the four letters, with the aim of increasing the likelihood of converting non-respondents:

• 1st letter: It’s easy to take part and why take part
• 2nd letter: Taking part will benefit you and your community
• 3rd letter: We want to hear from as many people as possible
• 4th letter: This is the last chance to have your say
• iIn addition, all letters placed a degree of emphasis on the financial motivator for taking part receiving a £10 gift voucher 

In addition to this the letters also provided key information about Ipsos and the Food Standards Agency and contact details for Ipsos should the participant have any queries about the survey. 

Online questionnaire

The Food and You 2 Wave 6 survey was hosted using Ipsos’ global Dimensions platform within Rackspace, a managed hosting facility and Europe’s most successful managed hosting company. The security features offered by Rackspace, and Ipsos are listed below:

At Rackspace:

• Rackspace has SAS 70 type II and Safe Harbor certifications and operates management systems which are compliant to International standards (IS09001, ISO27001);
• the servers and network infrastructure are physically located in England;
• the servers and network components are fully redundant;
• Rackspace guarantees recovery of hardware failures within one hour.

At Ipsos:

All applications and data for Dimensions were managed by Ipsos. Access to Dimensions’ questionnaires and data was password protected. Only a small number of online survey experts had access. Survey data and any participants personal information were stored in separate databases at Ipsos. 

Survey URL

As done in all previous waves, we used a dedicated URL that specifically included ‘food and you’ for the Food and You 2 Wave 6 survey. When deciding on the URL we wanted to choose an address that was short enough for participants to remember and one which would not easily be mis-typed. It also needed to give some indication of survey content.

Online questionnaire accessibility

The online questionnaire was made to be as accessible as possible to participants. Key to this was offering those in Wales the opportunity to complete the survey in Welsh (in line with the Welsh Language Act 1993). Participants could request to complete the survey in another language by calling the Food and You 2 survey helpline, or by asking someone to complete it on their behalf.

The Food and You 2 survey was designed to be accessed using a range of devices, including desktop computers, laptops, tablets and smart phones. The survey was designed with a ‘mobile first’ approach to minimise drop offs and improve response rates. A ‘mobile first’ approach means that the online questionnaire was designed with smart phone users in mind initially, as this is increasingly how participants choose to access online questionnaires. Additionally, the online questionnaire was designed in a way that made it easy for people to adjust colour contrasts and increase font size. 

Topline data checks

Once the online survey is in field and a sufficient number of responses have been received, topline data are reviewed to ensure no errors are detected with the filtering. Whilst extensive testing is carried out prior to fieldwork launch, on occasion unusual combinations of answers, particularly those involving non-valid answers, mean that minor errors are identified. By reviewing the data early on in fieldwork such errors can be identified and corrected. 

For Wave 6, topline checks were carried out after eight days of fieldwork when 864 responses had been received to the online survey. 

Results of the topline checks and refinements to the data checking syntax revealed an inconsistency for HSVOUCH (receipt of Healthy Start vouchers) in Waves 1-4 and Wave 6 (HSVOUCH was not asked in Wave 5). HSVOUCH should be asked of participants who were pregnant or had a child aged 0-4 in the household. Checks showed the online script was treating those who answered ‘Prefer Not To Say’ at CHILDAGE as having at least one child age=0 and routing them to HSVOUCH. This meant more participants were being asked HSVOUCH than required.

Numbers affected:

• W1 – 55
• W2 – 33
• W3 – 51
• W4 – 42
• W5 – n/a
• W6 – 17

However, waves 1 and 2 had been corrected in the data and tables before publication, and therefore, corrections were only needed for waves 3 and 4.

Changing the routing during fieldwork would introduce inconsistencies within the Wave 6 data waves. Therefore, this was addressed by:

(a)    Applying a back edit to HSVOUCH at the data processing stage of Wave 6 data
(b)    Changing the routing of this question if it is used in future waves

Break-offs and questionnaire length

A total of 7.8% of all participants (324) who started the online survey did not complete it, and therefore are not in the final dataset. A low number of breakoffs were observed across all questions asked in the online survey for Wave 6, for example, no question in particular caused people to leave the survey. 

The median completion time of those who completed the online survey was 25 minutes and 46 seconds.

Postal questionnaire

At the second reminder (Mailing 3) non-responding households were sent postal questionnaires. Households in England and Northern Ireland where one adult had completed the questionnaire and in which a second adult had been identified were sent one postal questionnaire, otherwise non-responding households were sent two postal questionnaires in these countries. All non-responding households in Wales were sent two postal questionnaires – one in English and one in Welsh. 

Each household that received two postal questionnaires received one Eating at Home and one Eating Out postal questionnaire. Households that were sent one postal questionnaire received only one of these versions.

In total 2,171 participants completed the postal questionnaire with 1,073 completing the Eating at Home postal questionnaire and 1,098 completing the Eating Out questionnaire. There were 20 participants in Wales who completed the Welsh language postal questionnaire in total, 14 completed the ‘Eating at Home’ version and six completed the ‘Eating Out’ version. The number of returns of the postal questionnaire for each country is detailed in Table 2. The highest number of postal returns were received from participants in England (1,177 returns), followed by 607 in Northern Ireland and 387 in Wales. 

Table 2: The number of postal questionnaires returned, by questionnaire version, for each country
 

Postal questionnaire	Number returned
Eating at Home (England)	612
Eating Out (England)	565
Eating at Home (Norther Ireland)	290
Eating Out (Northern Ireland)	317
Eating at Home Wales (English)	157
Eating at Home Wales (Welsh)	14
Eating Out Wales (English)	210
Eating Out Wales (Welsh)	6
Total	2.171

Storage of scanned images and survey results

All scanned images and survey data were stored on a secure server, which is isolated from the Ipsos network and has restricted access controls. Our secure file servers are housed in server rooms/data centres with appropriate physical access controls and monitoring procedures. The network is protected by appropriate use of firewalls, DMZ and intrusion detection systems. Public facing servers are also appropriately protected and are based on a secure (minimum) two tier or, our general standard, three-tier architecture. We used AES256 as a minimum standard for encryption.

Vouchers for participants

Participants were offered a £10 gift voucher as a thank you for taking part in the survey. 
Participants who completed the survey online who wished to receive a voucher entered their email address at the end of the survey. They were then emailed a Love2shop e-voucher of the nominal amount which they could redeem online at the Love2Shop website. 

Those who completed the postal questionnaire were given the choice of receiving a Love2shop e-voucher or paper Love2shop voucher via post, either of which could be redeemed at a wide range of high street stores. Participants were asked to give their name in order to address the voucher to the correct person, but even without a name a voucher would be sent to that address.

Handling queries

The survey website provided information about the survey and included a list of frequently asked questions (FAQs) which had been developed based on similar studies.

Additionally, a dedicated freephone telephone helpline and email address were set up allowing participants to contact Ipsos if they had any queries about the survey. Telephone queries were first recorded by an answer machine and a member of the research team returned the call when they had identified an appropriate solution. Emails sent to the Food and You 2 survey inbox were first answered with automatic responses, which included the most commonly asked questions and answers. Each query was then followed up individually within five working days. 

There were 288 queries, the majority of which were regarding when participants would receive their voucher or to opt out of the survey. Other queries included participants requesting a postal questionnaire or experiencing difficulties accessing the online survey. 

Response rates

The overall response rate for Food and You 2 Wave 6 was 28.8% with 1.43 adults participating per household on average. Of the surveys completed, 63.8% were online and 36.2% were postal questionnaires. Response rates varied by region. Table 3 shows the variation in response rate by region and country. 

Table 3: Overall individual level response rates by region and country

Region/Country	Issued addresses	Number of returns overall	Proportion of returns that were online (%)
East Midland	597	272	58.8%
East of England	768	362	58.6%
London	971	336	64.9%
North East	351	157	59.2%
North West	933	376	64.9%
South East	1,109	519	63%
South West	721	373	55.5%
West Midlands	710	305	62.3%
Yorkshire and the Humber	689	332	61.4%
England	6,849	3,032	61.2%
Wales	3,542	1,315	70.6%
Northern Ireland	4,109	1,644	63.1%
Total	14,500	5,991	63.8%

Table 4: Household level response rates by region and country

Region/Country	Issued addresses	Number of returns overall	Proportion of returns that were online (%)
East Midland	182	30.5%	1.49
East of England	252	32.8%	1.44
London	239	24.6%	1.41
North East	103	29.3%	1.52
North West	255	27.3%	1.47
South East	358	32.3%	1.45
South West	251	34.8%	1.49
West Midlands	205	28.9%	1.49
Yorkshire and the Humber	227	32.9%	1.46
England	2,072	30.3%	1.46
Wales	1,015	28.7%	1.30
Northern Ireland	1,088	26.5%	1.51
Total	4,217	28.8%	1.43

Profile of achieved sample

The tables below show the profile of those who completed the survey online and those who completed the postal questionnaires.

Table 5:  Demographic profile of survey responders - Gender

Demographic	Percentage of online participants	Percentage of postal participants	Percentage of total participants
Male	42.3%	41.0%	41.8%
Female	56.3%	58.0%	56.9%
In another way	0.5%	*%	0.3%
Prefer not to say	1.0%	0.9%	1.0%

*indicates a value higher than 0 but lower than 0.05.

Note, the not stated cases are excluded from the above table. 

Table 6: Demographic profile of survey respondents

This table shows that those who are aged 54 or younger are more likely to complete the online questionnaire than the postal questionnaire, with the opposite true for those aged 65 and over.

Demographic	Percentage of online participants	Percentage of postal participants	Percentage of total participants
16 to 24	5.1%	2.2%	4.1%
25 to 34	14.9%	5.7%	11.6%
35 to 44	17.9%	10.3%	15.2%
45 to 54	18.9%	11.6%	16.2%
55 to 64	19.5%	20.7%	19.9%
65 to 79	20.7%	36.2%	26.3%
80+	2.8%	11.7%	6.0%
Prefer not to say	0.1%	1.7%	0.7%

The 65-74 and 75+ age groups were changed to 65-79 and 80+ in the questionnaire for Wave 5 when the age question was amended to collect age or age band only, rather than a specific date of birth – this was continued for Wave 6. These age bands were chosen as they aligned with the weighting approach, however in future questionnaires, although the same approach will be taken collecting age and age band, the 65-74 and 75+ bands will be added so that the age bands align with how age has been reported in the data tables in Waves 1-4. 

Table 7: Demographic profile of survey responders - Ethnicity

Demographic	Percentage of online participants	Percentage of postal participants	Percentage of total participants
White	90.3%	94.6%	91.9%
Mixed	1.1%	0.7%	1.0%
Asian or Asian British	3.8%	2.2%	3.2%
Black or black British	1.0%	1.1%	1.0%
Other ethnic group	0.5%	0.3%	0.4%
Prefer not to say	3.2%	1.1%	2.5%

Table 8: Demographic profile of survey responders - Household size

Demographic	Percentage of online participants	Percentage of postal participants	Percentage of total participants
1	12.8%	22.6%	16.3%
2	45.7%	49.5%	47.1%
3	16.4%	12.5%	15.0%
4	16.4%	9.5%	13.9%
5+	8.7%	5.8%	7.7%