Appendix E: BWS Method, Choice Tasks and Experimental Design

BWS is a technique designed to elicit the relative importance of items (desirability of brands priorities in policy, preferences for ice cream flavours,…). 

The items being ranked here are the set of impacts of FHS which comprise the condition-specific quality of life measures:
•    Food Allergy Quality of Life Questionnaire, FAQLQ; 
•    Food Intolerance Quality of Life Questionnaire, FIQLQ; 
•    Coeliac Disease Quality of Life Questionnaire, CDQ.

The FAQLQ and FIQLQ quality of life instruments  use a 7-point Likert Scale format (from “not” to “extremely”) when eliciting the magnitude of each impact on respondents, as shown in this set of questions from the FAQLQ and FIQLQ instruments. The FAQLQ comprises 29 items, and the FIQLQ 22 items

Figure 1. Example FIQLQ questions

How troublesome do you find it, because of your food intolerance, that you... (marked on a scale from 0 'not' to 6 'extremely'. 

• must always be alert as to what you are eating?
• have less variety in the food that you can eat?
• have less variety in the products that you can buy?
• must read labels?
• have the feeling that you have less control of what you eat when eating out?

The image shows the 7-point likely scale (ranging between ‘not’ and ‘extremely’) on which impacts were scored in the FIQLQ instrument. 
Some example FIQLQ items are shown, of the form: ‘how troublesome do you find it, because of your food intolerance, that you…
1.    Must always be alert as to what you are eating
2.    Have less variety  in the food that you can eat?
3.    Have less variety in the products that you can buy?
4.    Must read labels?
5.    Have the feeling that you have less control of what you eat when eating out?

Figure 2. Example FAQLQ questions

How troublesome do you find it, because of your food allergy, that you... (marked on a scale from 0 'not' to 6 'extremely'. 

• must always be alert as to what you are eating?
• are able to eat fewer products?
• are limited as to the products you can buy?
• must read labels?
• have the feeling that you have less control of what you eat when eating out?
• must refuse many things during social activities?

The CDQ comprises 20 items and is scored on a 5-point scale (from “not at all” to “a great deal”):

Figure 3. Example CDQ questions

For each statement, please select the option that best describes your feelings. 

• I feel limited by this disease
• I feel worried that I will suffer from this disease
• I feel concerned that this disease will cause other health problems
• I feel worried about my increased risks of cancer from this disease
• I feel socially stigmatized for having this disease

The items which comprise the FAQLQ, FIQLQ and CDQ instruments were converted into statements that could be placed alongside each other in BWS sets, and respondents asked which had the (a) greatest, and (b) least, impact on their quality of life.

The items from the 3 instruments required a minor rephrasing so they could be displayed next to each other, and respondents asked to pick those with least/greatest impact. A small number of items 

Example BWS sets for the FAQLQ, FIQLQ and CDQ are shown in Figures 3.1 and 3.2 in section 3.5 of the report. 

BWS Experimental Design

The BWS sets were the product of an experimental design in which the items are combined into varying subsets of the full set of items.

The design requires decisions to be made regarding (i) number of items per set, and (ii) how many sets to shown to each respondent. 

For all 3 BWS exercises the items were combined into sets of 4 items; research has indicated that a maximum number of 5 items should be presented within a subset, as sets greater than this yield little in terms of statistical power relative to the costs of additional cognitive load (Sawtooth Software, 2020;  Chrzan and Patterson (2006)).

The experimental design (created using Sawtooth Software’s MaxDiff design module.) of the BWS exercise used a programming-based algorithm which generated an orthogonal design in which each item appeared the same number of times and there was positional balance of the items within the subsets. The designs were of the form:

Table 1. BWS design details for each of the 3 conditions / QoL measures

In each case the design a respondent saw was randomly selected from one of 50 blocks, - having multiple blocks of the design increased variation in item co-occurrence and item position within the sets (which can have an impact on the probability of selection by a respondent).  They were presented with the condition that they had identified as the sole, or most significant, food hypersensitivity that they had.

Analysing and Interpreting BWS data and results

In each BWS set the respondent chooses the impact which has ‘most’ and ‘least’ impact on their quality of life. These are yes/no choices from a fixed set of options (like the DCE reported above) and hence logit models are estimated on the BWS data. 

To increase the interpretive power of the results, the logit coefficient for each impact included in the BWS exercise was transformed into an ‘Importance Score’ (Sawtooth Software, 2020) where the Importance Score is defined as:

Φi /( Φi  + n-1)

where:

Φi =zero-meaned logit score for impact i
n=number of items shown per BWS set (in this study, n = 4)

The advantage of this transformation is that Importance Scores are ratio scaled - meaning that an item with a score of 6, for example, is regarded by the sample as 6 times more important than an item with a score of 1. This property makes for more intuitive assessment and interpretation of the results (it provides information on the magnitude of the differences in quality-of-life impacts rather than just a ranking).