F&Y2 trends weighting note: Calculation of trend weights

Wave weights

For each wave of the Food and You 2 survey, weights were calculated with two objectives in mind:

• to compensate for known differences in sample member selection probabilities; and 
• to reduce non-response bias through post-stratification. 

Separate samples were selected in England, Wales and Northern Ireland and Individual country weights were calculated which equalised weighted and unweighted sample totals for each country. Additional all-country weights, which weighted sample numbers in each country in proportion to country population estimates were also calculated.

An additional weight was calculated with the sole purpose of aligning selected demographic characteristics of the English sample to those of the Welsh sample. This was termed the ‘Welsh England’ weight. 

Weighting complications arose because, in each wave, not all questions were asked of all sample members. In most waves questions were asked either of: 

• all sample members 
• or online respondents only (i.e. the questions were excluded from all postal questionnaires)
• or online respondents and half (randomly selected) of postal questionnaire respondents. 

There were also some additional complications in wave 3 when a small number of questions were fielded by different modes in different countries.  Because of this in each wave several weights, depending on mode of administration, were calculated (i) for each country (ii) all countries combined and (iii) for ‘Welsh England’ estimates.

Trend weights

Separate trend weights have been calculated (i) for each country, (ii) for all countries combined and (iii) for ‘Welsh-England’ estimates. 
The purpose of trend weights is to allow data for individual questions to be compared across waves. For each trend weight, relevant wave weights were identified and then rescaled in order to equalise the weighted sample size in each wave. This was done for each individual country, for all countries combined and for ‘Welsh-England’ estimates.
In doing this, the first step was, for each variable, to identify the pattern of mode administration across the six survey waves. Variables were then allocated to weighting groups as follows:

1. If a variable was administered to the online sample only for any individual wave, it was automatically allocated to an online-only trend weighting group (A in the table below). This was done because we know that different kinds of respondent tend to reply online and by post, and, for this reason, felt that administration mode should not be allowed to vary by wave for inter-wave comparisons (i.e. comparisons should be made using online questionnaires throughout, or online plus postal questionnaires throughout). 
2. For all variables including some postal administration in each wave, those sharing the same pattern of mode administration across waves were allocated to the same weighting group.

Nine weighting groups were identified. For each, the individual wave weights used in its construction are shown in the table below.

Table 1: Weighting groups for each wave

Notes:  

• All = online and postal questionnaire(s)
• EH = Eating at home version of postal questionnaire
• EO = Eating out version of postal questionnaire
• FP = Food poisoning version of postal questionnaire
• A = version A of postal questionnaire
• B = version B of postal questionnaire

If a variable does not appear in a wave, it can be assigned to a weighting group for which the weights for the waves in which the variable was fielded are correct (i.e. the weights allocated to unfielded waves are irrelevant).  For example, if a variable were fielded only on waves 4, 5 and 6, using respectively the online + EH weight, the All weight and the online + EH weight in those waves, it would be assigned to weighting group E because Group E uses the correct weight in each of the three relevant waves.

For each trend weight, relevant wave weights were rescaled to equalise weighted wave sample sizes in each wave. For a given weight, the all-waves sum of the relevant original wave weight (for each country for the individual country weights, and for all-countries for the all-country weights) was divided equally between waves, so that the overall sum of each trend weight and its constituent wave weights were equal.  

For example, consider the calculation of the online-only wt_all_A trend weight.  They are based on rescaling the existing online-only weights from each wave.  These existing weights were designed to sum to the total number of online respondents in their wave e.g. 5936 in Wave 1. Across all six waves, there were 25,632 online respondents - an average of 4272 per wave. So the existing W1 online-only weights were rescaled by multiplying them by 0.7197 ( = 4272 / 5936).  The online-only weights in the other waves were also rescaled using the average number divided by total of online respondents relevant to their relevant wave. These rescaled weights were then complied to produce the online-only wt_all_A trend weight.

When the original wave weights were calculated they were scaled to equalise weighted and unweighted sample sizes, and therefore, for the trend weights, the same should apply. However, because in wave 3 eight blank cases were dropped after the wave 3 weights had been calculated a small discrepancy between weighted and unweighted sample sizes appeared in this wave. Because trend-weight totals were scaled to original wave weight totals, not to original wave sample sizes, the trend weight totals differ slightly from the relevant sample size totals. (This discrepancy never exceeds more than 0.1% of the total sample size and is usually less than one). This has no impact on variable means and percentages.

The final trend weights are as follows:

• wt_all_A    Estimates for All countries: group A
• wt_all_B    Estimates for All countries: group B
• wt_all_C    Estimates for All countries: group C
• wt_all_D    Estimates for All countries: group D
• wt_all_E    Estimates for All countries: group E
• wt_all_F    Estimates for All countries: group F
• wt_all_G    Estimates for All countries: group G
• wt_all_H    Estimates for All countries: group H
• wt_all_I    Estimates for All countries: group I
• wt_ind_A    Estimates for Individual countries: group A
• wt_ind_B    Estimates for Individual countries: group B
• wt_ind_C    Estimates for Individual countries: group C
• wt_ind_D    Estimates for Individual countries: group D
• wt_ind_E    Estimates for Individual countries: group E
• wt_ind_F    Estimates for Individual countries: group F
• wt_ind_G    Estimates for Individual countries: group G
• wt_ind_H    Estimates for Individual countries: group H
• wt_ind_I    Estimates for Individual countries: group I
• wt_we_A    Estimates for Wales & Welsh-England: group A
• wt_we_B    Estimates for Wales & Welsh-England: group B
• wt_we_C    Estimates for Wales & Welsh-England: group C
• wt_we_D    Estimates for Wales & Welsh-England: group D
• wt_we_E    Estimates for Wales & Welsh-England: group E
• wt_we_F    Estimates for Wales & Welsh-England: group F
• wt_we_G    Estimates for Wales & Welsh-England: group G
• wt_we_H    Estimates for Wales & Welsh-England: group H
• wt_we_I    Estimates for Wales & Welsh-England: group I

In all waves except wave 3, category E variables are included in at least some postal questionnaires. However, in wave 3 some category E variables (defra2_1 to defra2_3, defra3_1 to defra3_3 and defra4_1) were asked of the whole sample in England and Wales, but of only the online sample in NI. In wave 3 therefore, a special all countries weight (weight 3) was created for these variables only. This weight was applied to the whole sample in England and Wales and to the online sample only in NI. 

As a rule, trend weights have been based solely on online cases when measuring trends for variables that were online-only in any single wave. We did this because mode effects are likely to be significant, and we therefore cannot legitimately compare online + postal wave estimates with online-only wave estimates. For this reason, the category E individual country trend weight for NI is restricted to online cases only. However, uniquely, the all-countries category E weight does apply both to online + postal cases and online only cases (the wave 3 NI cases only). 

It is true that if mode effects were substantial, this will mean that all country estimates are slightly biased in wave 3 but, because NI has such a small population relative to that of England and Wales, any such bias across all countries could only have a trivial impact on the all-country trends.