Food and You 2: Technical Report: Sampling

Sample design

The sample for each wave of Food and You 2 is selected from the postcode address file (PAF) in England, Wales and Northern Ireland (and Scotland, when applicable). The sample of addresses is unclustered within each country. Households are sampled to achieve interviews in 1,000 households in Wales and Northern Ireland, and 2,000 households in England (4,000 overall). In other words, a greater proportion of households are sampled in Wales and Northern Ireland compared to England. This is done to improve the precision of estimates for Wales and Northern Ireland. When Scotland is included, the sample of addresses is aimed to achieve 1,000 household responses.

The size of the issued sample in each country is calculated by dividing the target sample by estimated address yield (proportion of addresses with at least one productive response). Yield estimates are based on actual yields obtained in previous waves. An additional reserve sample is drawn to be issued (in whole or in part) if response rates are lower than anticipated.

The sample sizes and assumptions for each country can be found in the accompanying technical report spreadsheet.

The samples of main and reserve addresses are stratified proportionately by region in England (with the other countries treated as separate regions), and within region (or country) by local authority (district in Northern Ireland) to ensure that the issued sample is spread proportionately across the local authorities. National deprivation scores are used as the final level of stratification within the local authorities (in England the Index of Multiple Deprivation (IMD), in Wales the Welsh Index of Multiple Deprivation (WIMD), in Northern Ireland, the Northern Ireland Multiple Deprivation Measure (NIMDM) and in Scotland the Scottish Index of Multiple Deprivation (SIMD)). In practice stratification is achieved by ordering the population of PAF addresses by (i) region (country), (ii) local authority (district) within region, and (iii) national deprivation score of Lower Layer Super Output Areas (LSOA) (OA on Northern Ireland) within local authority (district), and then selecting addresses by the method of random start and fixed interval. The steps for sampling that are taken are:

1. From the PAF file, exclude all business addresses and private addresses that were selected in previous waves of the Food & You 2 survey
2. Order the address list by region (for England only)
3. Within each English region, and country, order addresses by local authority (district in Northern Ireland)
4. Within local authority / district, order addresses by IMD of LSOA in England, WIMD of LSOA in Wales, NIMDM of Super Output Areas (SOA) in Northern Ireland, and Scottish IMD (SIMD) data zones in Scotland
5. Select numbers of addresses agreed with the FSA by method of random start and fixed interval from these ordered lists
6. Divide stratum-ordered selections into successive groups of 3 selections
7. Within each group of three, randomly allocate two cases to the main sample, and one case to the reserve sample.

Household sample design

As stated above, addresses are selected from the Postcode Address File (PAF) systematically using the random start and fixed interval method. At each address, up to two adults are invited to take part in the survey. Two unique login codes for the online survey are provided in the initial invitation. Up to two postal questionnaires are provided in the postal questionnaire mailing (Mailing 3). In the reminders, two logins / questionnaires are sent to completely non-responding addresses. At any address where one adult has already completed the questionnaire only one login code and one postal questionnaire are sent. Each adult who completes the questionnaire receives a £10 online or paper voucher.

Process for selecting adults within a household

There are many approaches that could have been used for selecting adults within households. For instance, the two adults with the most recent birthdays or the adults with the two next birthdays could be selected. These are commonly referred to as quasi-random approaches, as they are roughly equivalent to a fully random approach. While this would have randomised the selection process to a degree in households where there were more than two adults, in self-administered surveys it adds another barrier to completing the survey and has been shown to be incorrect in about 20% to 25% of cases. Further details are available from TNS BMRB’s 2013 report of web experiments prepared for the Cabinet Office on the Community Life Survey or a journal article from 2014 by Kristen Olson and Jolene D. Smyth focusing on the accuracy of within-household selection in general population web and mail surveys published in Field Methods (volume 26, issue 1, pages 56–69).

With this in mind, it was decided to allow any two eligible adults (aged 16 years or over) to participate in the survey. Given the household size distribution in the UK, it was estimated that 93% of the sample selected in this way would also have been selected had we managed to successfully implement a random selection method.

This approach is taken for all waves of the Food and You 2 survey.

Using the reserve sample

After four weeks of fieldwork, the number of completed online surveys is reviewed to determine whether it is likely that the target of 2,000 household returns for England and 1,000 household returns for each of the other countries will be met. If the target is unlikely to be met, the reserve sample may be used to meet the household returns target.

When a reserve sample is used, the same push-to-web survey design that is used for the main sample is replicated for the reserve sample, meaning that participants are able to complete the survey online or on paper.

For example, in Wave 5, to help ensure the target was met, invitation letters were sent to 2,000 additional addresses selected randomly from the reserve sample list. The reserve sample was split by country in the same proportions as the main sample (944 in England, 489 in Wales and 567 in Northern Ireland). The interval between mailings and the total number of mailings was reduced slightly compared to the main sample as described in the Fieldwork and response rates section.