Development of a framework for evaluation and expression of uncertainties in hazard and risk assessment

Background

In 2007 The Committee on Toxicity of Chemicals in Food, Consumer Products and the Environment (COT) published its report on variability and uncertainty in toxicology. The COT report concluded there was a need to develop a framework for the transparent expression of uncertainty in hazard characterisation which would enable COT, and other committees that perform toxicological evaluations, to improve communications to a wide range of stakeholders including the public with regards to the sources of variability and uncertainty in their risk assessments.

Research Approach

This study aimed to address this requirement by developing and testing a framework for the qualitative expression of uncertainties. A review of existing approaches for qualitative evaluation of uncertainties in risk assessment was undertaken. The potential suitability of these approaches was evaluated, and the most promising combination of approaches was identified and incorporated into the framework.

Forty experts were invited to provide a direct and thorough test of the applicability and usability of the identified framework using one of four case studies, based on previous COT statements. This was achieved as part of a workshop on evaluation and expression of uncertainties in risk assessment, held in conjunction with the COT’s annual out-of-town meeting.

Results

A draft framework was produced which incorporated promising approaches to help identify uncertainty. In a workshop setting, four previous assessments published by the COT (this included caffeine, perfluorooctanoic acid (PFOA), peanut allergy and results of a Total Diet Study (aluminium and palladium)) were used to determine the effectiveness of the framework by applying it retrospectively to each of them individually. COT members provided feedback and this helped to further refine the framework.

The findings of the workshop, together with a draft copy of the final report, were presented at a Committee on Toxicity (COT) meeting held in June 2010. The document can be located on the COT website

COT members agreed that their feedback at the workshop had been taken into consideration. A key point to arise was that members were less keen on the use of numerical values or symbols to express uncertainty about qualitative conclusions, but accepted that the report reflected on these points. The minutes on the discussion of this paper can be located on the COT website

Overall, it was concluded that it is fundamentally important for all risk assessments to evaluate uncertainty.

Further work is required to evaluate the application of the framework to different types of risk assessments. This will help to develop effective approaches when communicating uncertainty to decision makers and other key stakeholders.

In light of the findings, the following recommendations were made:

• Further investigation to refine the theoretical basis for the evaluation of uncertainties in quantitative questions, and to establish a theoretical basis for categorical questions.
• Further evaluation and, if appropriate, refinement of the proposed approaches by applying them to a range of practical examples.
• Exploration of challenges in standardising terminology for communicating uncertainties associated with toxicological risk assessments.

If the above recommendations are fulfilled, the credibility of addressing uncertainty of science-based policy will be further improved. This will ensure the COT, COM and COC utilise the framework to its maximum potential and be fully confident of making accurate assessments and decisions.