Appendix O: Protest Votes, Task Difficulty and Debriefs on DCE and BWS

If people declined free removal in all 3 training sets (ie including the removal of FHS for 20 years for free) they were skipped past the priced DCE – since they had declined the ‘best’ offer (20 years) at zero price.

People who always chose to buy the pill in the 9 sets were presented with debrief questions to allow identification of protest or problematic behaviour – for example people not taking the price seriously. Likewise, people who never chose to buy the pill in the 9 sets, in this case problematic responses included not thinking that the pill would be safe and therefore not considering buying it.

Everyone who completed the 9 priced choice sets were asked debrief questions on

• how hard it was to understand the DCE choices
• how hard it was to make the DCE choices.

DCE Adults

Declining 3 zero price options: Adults

53% accepted the free removal in all 3 choice sets, but 16% declined removal of their FHS in one choice set and 11% rejected free removal in all 3 training sets, one of which was for a duration of 20 years.

Table 1: Number of Free Options Declined

Number of options	Frequency	Percent	Cumulative
0	754	52.88	52.88
1	222	15.57	68.44
2	289	20.27	88.71
3	161	11.29	100.00
Total	1426	100.00	-

The finding that 11% of adults with a FHS would not accept its removal for 20 years at zero cost was not anticipated at the start of the research process. The issue of adjustment costs had featured in the focus groups which had prompted the researchers to increase the maximum duration used in the zero price DCE training sets to 20 years – to identify people who would never buy in the priced DCE sets.

Figure 1. Number of zero price removals of FHS declined, adults.

The 161 people declining all 3 free removals were excluded from the priced DCE experiment. This group were asked what was the minimum duration (if any) that they would accept for unpriced removal of their FHS condition. 

Table 2: What is the minimum period that would induce you to accept free removal?

Period of time	Frequency	Percent	Cumulative
25 years	3	1.86	1.86
35 years	4	2.48	4.35
40 years	4	2.48	6.83
45 years	1	0.62	7.45
50 years	4	2.48	9.94
Permanent removal only	88	54.66	64.60
Decline removal	57	35.40	100.00
Total	161	100.00	-

Of the 161 people who always rejected free removal, 10% would choose removal if it was for a longer, but still temporary, period, 55% would only accept permanent removal and 35% (57 people) did not want their FHS removed at all.

Protest Choices: Adults

People who had accepted the free removal but who never bought the pill in any of the 9 choice sets that followed were directed to a question asking why that was. This was in part to distinguish potential buyers who could be included in the DCE analysis (ie potential buyers who could not afford the prices shown) from people who rejected the premise of the choice tasks, would never buy and should be excluded from the DCE analysis.

Table 3: Why did you never pay for the pill?

Reason for not paying	Frequency	Percent	Cumulative
Cost too high for benefit	149	36.79	36.79
I did not trust it's safety	9	2.22	39.01
I did not trust it would work	10	2.47	41.48
I could not afford what was asked	124	30.62	72.10
Removal length was not worthwhile	34	8.40	80.49
The government should pay	37	9.14	89.63
Other	42	10.37	100.00
Total	405	100.00	-

Responses in bold (the pill regarded as not safe, or ineffective, or a view that the government should pay) were treated as indicating rejection of the valuation process and excluded from the choice models estimated on the DCE data.

People who always bought the pill in all 9 choice sets were directed to a question asking why that was. 

Table 4: Why did you always pay for the pill?

Reasons for not paying	Frequency	Percent	Cumulative
Prices unrealistic so ignored	7	10.77	10.77
The cost was small compared to benefit	46	70.77	81.54
Other	12	18.46	100.00
Total	65	100.00	-

Ignoring the price was regarded as a rejection of the valuation process and people citing that as a reason for always ‘buying’ were excluded from the choice models estimated on the DCE data.

A total of (56 + 7) 63 people who completed the priced DCE were therefore excluded from the analysis, this is less than 4.5%.

DCE Debriefs: Adults

Aggregating across the three conditions 88% of the adult sample regarded the DCE as “easy” or “very easy” to understand and less than 10% of the sample regarded the DCE as “difficult” or “very difficult”.

Table 5. How easy was it to understand the WTP questions? – Full Sample

Understand scale	Frequency	Percent	Cumulative
Very difficult	14	1.11	1.11
Difficult	99	7.83	8.93
No opinion	85	6.72	15.65
Easy	561	44.35	60.00
Very easy	506	40.00	100.00
Total	1265	100.00	-

The pattern was consistent across the FHS conditions, as shown in Figure 2. 

Figure 2: Ease of understanding DCE choices, adults

A separate question was regarding the difficulty of making the DCE choices. Fewer than 3.5% of the sample found making the DCE choices “very difficult” but about a quarter did find them difficult. This response pattern is not in itself a cause of concern, in fact researchers want people to have to exert to make their choices as it suggests respondents are considering the tradeoffs when making their choices, and that the attribute levels are such that tradeoffs have to be considered carefully when making choices. For example, if the prices were excessively low then people would find the choices ‘easy’, but the price term would likely be insignificant and WTP estimates not retrievable).

Table 6: How easy was it to decide your responses to the WTP questions? Full sample

Understand scale	Frequency	Percent	Cumulative
Very difficult	41	3.24	3.24
Difficult	305	24.11	27.35
No opinion	77	6.09	33.44
Easy	556	43.95	77.39
Very easy	286	22.61	100.00
Total	1265	100.00	-

The pattern is similar when disaggregated by condition, as shown in Figure 3.   The percentage reporting the choices as “very difficult” ranged between 1.6% and 4.1%.  The proportion reporting the choices as “easy” or “very easy” ranged from 62% (allergy) to 73% (coeliac disease).

Figure 3. Ease of making DCE choices, adults. 

DCE: Children

Declining 3 zero price options: Children

If people declined the unpriced removal of their FHS in all 3 choice sets, even for the longest period of 20 years, they were not presented with the priced choice sets which are the basis of the WTP analysis presented below. 

In the Parent sample 18% declined removal of their child’s FHS in one choice set, 26% declined removal of their FHS in two choice sets and 17% of respondents declined all 3 free removal options. The probability of declining the free removal declines as the duration increases.

Table 7.  Number of Free Options Declined

Number of options	Frequency	Percent	Cumulative
0	277	38.69	38.69
1	129	18.02	56.70
2	188	26.26	82.96
3	122	17.04	100.00
Total	716	100.00	-

Figure 4. Number of zero price removals of FHS declined, children

Those who declined all three free removal options were asked the minimum period of removal of their child’s FHS which would induce them to opt for removal.   

Of the 122 people who always rejected free removal in the training sets, one fifth specified a period of temporary removal they would accept, 65% indicated they would only accept permanent removal and 15% (18 people) did not want their child’s FHS removed at all.

Table 8. What is the minimum period that would induce you to accept free removal?
 

Number of years	Frequency	Percent	Cumulative
25 years	10	8.20	8.20
30 years	4	3.28	11.48
35 years	3	2.46	13.93
40 years	3	2.46	16.39
50 years	5	4.10	20.49
Permanent removal only	79	64.75	85.25
Decline removal	18	14.75	100.00
Total	122	100.00	-

Protest Choices: Children

People who never bought the pill in any of the 9 choice sets were directed to a question asking why that was. This was in part to distinguish potential buyers who could be included in the DCE analysis (for example, potential buyers who could not afford the prices shown) from people who rejected the premise of the choice tasks, would never buy and should be excluded from the DCE analysis.

Table 9. Why did you never pay for the pill? 

Reasons for not paying for the pill	Frequency	Percent	Cumulative
Cost too high for benefit	19	21.59	21.59
I could not afford what was asked	26	29.55	51.14
Removal length was not worthwhile	5	5.68	56.82
My child would have to re adjust	22	25.00	81.82
I did not trust it's safety	2	2.27	84.09
I did not trust it would work	1	1.14	85.23
The government should pay	7	7.95	93.18
Other	6	6.82	100.00
Total	88	100.00	-

Responses in bold (the pill regarded as not safe, or ineffective, or a view that the government should pay) were treated as indicating rejection of the valuation process and excluded from the choice models estimated on the DCE data.

People who always bought the pill in all 9 choice sets were directed to a question asking why that was. 

Table 10. Why did you always pay for the pill?

Reasons for always paying for the pill	Frequency	Percent	Cumulative
Prices unrealistic so ignored	7	10.77	10.77
The cost was small compared to benefit	46	70.77	81.54
Other	12	18.46	100.00
Total	65	100.00	-

Ignoring the price was regarded as a rejection of the valuation process and people citing that as a reason for always ‘buying’ were excluded from the choice models estimated on the DCE data.

A total of (10 + 7) 17 parents who completed the priced DCE were therefore excluded from the analysis, this is less than 3%.

DCE Debriefs: Children

Aggregating across the three conditions 70% of the adult sample regarded the DCE as “easy” or “very easy” to understand and less than 4% of the sample regarded the DCE as “very difficult”.

Table 11: How easy was it to understand the WTP questions? Full sample

Understand scale	Frequency	Percent	Cumulative
Very difficult	23	3.87	3.87
Difficult	86	14.48	18.35
No opinion	67	11.28	29.63
Easy	265	44.61	74.24
Very easy	153	25.76	100.00
Total	594	100.00	-

The pattern was consistent across the FHS conditions, as shown in Figure 5. 

Figure 5: Easy of understanding DCE choices, children

A separate question was regarding the difficulty of making the DCE choices. 11% of the sample found making the DCE choices “very difficult” and a third found them difficult. 

As discussed regarding Adult sample, this response pattern is not in itself a cause of concern.

Table 12. How easy was it to decide your response to the WTP questions? – Full Sample

Easy to decide	Frequency	Percent	Cumulative
Very difficult	66	11.11	11.11
Difficult	195	32.83	43.94
No opinion	52	8.75	52.69
Easy	195	32.83	85.52
Very easy	86	14.48	100.00
Total	594	100.00	-

The pattern is similar when disaggregated by condition, as shown in Figure 6. 

Figure 6. Ease of making DCE choices, children. 

BWS Debriefs

The BWS tasks were completed only by adults, not by parents (in part because the components of the child FIQLQ and FAQLQ instruments vary by child age and so a single BWS design was not possible for the Parent survey). It is the debrief data from the adults that is presented here. 

Understanding the BWS Choices.

Figure 7. Ease of understanding BWS allergy choices, adults.

Table 13: BWS understanding Food Allergy

Easy to decide	Frequency	Percent	Cumulative
Very difficult	12	3.12	3.12
Difficult	38	9.87	12.99
No opinion	48	12.47	25.45
Easy	176	45.71	71.17
Very easy	111	28.83	100.00
Total	385	100.00	-

Figure 8. Ease of understanding BWS coeliac disease choices, adults.

Table 14: BWS understanding Coeliac disease

 

BWS choice difficulty	Frequency	Percent	Cumulative
Very difficult	8	1.33	1.33
Difficult	73	12.15	13.48
No opinion	102	16.97	30.45
Easy	282	46.92	77.37
Very easy	136	22.63	100.00
Total	601	100.00	-

Figure 9: Ease of understanding BWS intolerance choices, adults

Table 15: BWS understanding Food intolerance

 

BWS choice difficulty	Frequency	Percent	Cumulative
Very difficult	8	1.82	1.82
Difficult	64	14.55	16.36
No opinion	53	12.05	28.41
Easy	202	45.91	74.32
Very easy	113	25.68	100.00
Total	440	100.00	-

Making the BWS Choices

Figure 10: Ease of making BWS allergy choices, adults

Table 16: BWS choice difficulty Food Allergy

BWS choice difficulty	Frequency	Percent	Cumulative
Very difficult	25	6.49	6.49
Difficult	124	32.21	38.70
No opinion	37	9.61	48.31
Easy	138	35.84	84.16
Very easy	61	15.84	100.00
Total	385	100.00	-

Figure 11: Ease of making BWS coeliac disease choices, adults

Table 17: BWS choice difficulty: coeliac disease

BWS choice difficulty	Frequency	Percent	Cumulative
Very difficult	22	3.66	3.66
Difficult	206	34.28	37.94
No opinion	86	14.31	52.25
Easy	228	37.94	90.18
Very easy	59	9.82	100.00
Total	601	100.00	-

Figure 12: Ease of making BWS intolerance choices, adults

Table 18: BWS choice difficulty: Food intolerance

BWS choice difficulty	Frequency	Percent	Cumulative
Very difficult	16	3.64	3.64
Difficult	133	30.23	33.86
No opinion	50	11.36	45.23
Easy	179	40.68	85.91
Very easy	62	14.09	100.00
Total	440	100.00	-

Summary

The survey was characterised by low levels of protest behaviour in the price DCE, although significant numbers were not included in the priced DCE because of their rejection of the 20-year removal at zero cost in the training choice sets.

The extent to which people found the DCE choice sets easy to understand was regarded as a positive finding. Finding those choices difficult to make was a more common occurrence, but was not regarded as problematic, given the feedback from the focus groups and the statistic results derived from the choice data.

A similar pattern was observed for the BWS choice sets – making the choices was harder than understanding the choice required.