You are in ePLANETe >
The Uncertainty Guidance: a checklist based approach to uncertainty assessment
Following the classic conception of scientific policy advice, certainty is the ‘first best’ knowledge situation for the management of complex problems. However, uncertainty is a fact of life. Scientific assessments today have to integrate information across a wide spectrum from well-established scientific knowledge to educated guesses, preliminary models, and tentative assumptions. In such contexts, uncertainty is endemic and, in general, can mostly not be remedied through additional research or comparative evaluations of evidence by expert panels searching for a consensus interpretation of the risks.
The challenge to scientific advisers in a deliberative governance or inclusive risk management perspective is to be as transparent and clear as possible in their treatment of uncertainties. Several
institutions that interface science and policy have adopted knowledge quality assessment approaches in response to emerging needs. One example is the Netherlands Environmental Assessment Agency (Planbureau voor de Leefomgeving, PBL), which in the year 2000 commissioned Utrecht University to develop, together with MNP, a Guidance for Uncertainty Assessment and Communication,3 a state-of-the-art reflective approach that is summarized below. This guidance has been developed in close consultation with international uncertainty experts.
It aims to facilitate the process of dealing with uncertainties throughout the whole scientific assessment process and explicitly addresses institutional aspects of knowledge development, openly deals with indeterminacy, ignorance, assumptions, and value loadings. It thereby facilitates a profound societal debate and a negotiated management of risks. The Guidance is not set up as a rigid protocol. Instead, it provides a heuristic that encourages self-evaluative systematization and
reflexivity on pitfalls in knowledge production and use. It also provides diagnostic help as to where uncertainty may occur and why. This can, its originators hope, contribute to more conscious, explicit, argued, and well-documented choices.
Following a diagnostic checklist approach, the Guidance consists of a layered set of instruments (Mini-Checklist, Quickscan, and Detailed Guidance) with increasing level of detail and sophistication (Janssen et al. 2005; van der Sluijs et al. 2008). It can be used by practitioners as a (self-) elicitation instrument or by project managers as a guiding instrument in problem framing and project design.
Using the Mini-Checklist and Quickscan Questionnaire, the analyst can flag key issues that need further consideration. Depending on what is flagged as salient, the analyst is referred to specific sections in a separate Hints and Actions document and in the Detailed Guidance. Since the number of cross-references between the documents comprising the Guidance is quite large, a publicly available interactive web application has been implemented. This web application also offers a prioritized to-do list of uncertainty assessment actions, and generates reports of sessions (traceability and documentation), which enables internal and external review.
Six foci are distinguished where systematic critical reflection on uncertainty and quality can help to gain a better understanding of knowledge. These are: problem framing, stakeholder participation, indicator selection, appraisal of the knowledge base, mapping and assessment of relevant uncertainties, and reporting of the uncertainty information. The table below summarizes key issues that require systematic reflection for each of the six foci.
In order to facilitate communication about the different types of uncertainty that arise in scientific assessments, an uncertainty typology is part of the Guidance. The typology is based on a conceptual framework that resulted from a process involving an international group of uncertainty experts, most of whom participated in developing or reviewing the Guidance. Uncertainty can be classified along the following dimensions: its ‘location’ (where it occurs), its ‘level’ (whether it can best be characterized as statistical uncertainty, scenario uncertainty, or recognized ignorance), and its ‘nature’ (whether uncertainty primarily stems from knowledge imperfection or is a direct consequence of inherent variability). In addition, the typology distinguishes the dimensions ‘qualification of knowledge base’ (what are weak and strong parts in the assessment) and ‘value-ladenness of choices’ (what biases may shape the assessment). For a full description, see the uncertainty matrix.
The guidance is available as interactive web application.
References
J.P. van der Sluijs, A.C. Petersen, P.H.M. Janssen, James S Risbey and Jerome R. Ravetz (2008) Exploring the quality of evidence for complex and contested policy decisions, Environmental Research Letters, 3 024008 (9pp)
Janssen, P.H.M., Petersen, A.C., Van der Sluijs, J.P., Risbey, J., Ravetz, J.R. (2005), A guidance for assessing and communicating uncertainties. Water science and technology, 52 (6) 125–131
A.C. Petersen, P.H.M. Janssen, J.P. van der Sluijs, J.S. Risbey, J.R. Ravetz (2003) RIVM/MNP Guidance for Uncertainty Assessment and Communication: Mini-Checklist & Quickscan Questionairre, RIVM/MNP; ISBN 90-6960-105-1.
P.H.M. Janssen, A.C. Petersen, J.P. van der Sluijs, J.S. Risbey, J.R. Ravetz (2003). RIVM/MNP Guidance for Uncertainty Assessment and Communication: Quickscan Hints & Actions List. RIVM/MNP, ISBN 90-6960-105-2, 2003.
J.P. van der Sluijs, J.S. Risbey, P. Kloprogge, J.R. Ravetz, S.O. Funtowicz, S.Corral Quintana, Â Guimarães Pereira, B. De Marchi, A.C. Petersen, P.H.M. Janssen, R. Hoppe, and S.W.F. Huijs (2003). RIVM/MNP Guidance for Uncertainty Assessment and Communication: Detailed Guidance Utrecht University & RIVM, 2003.
J.P. van der Sluijs, P.H.M. Janssen, A.C. Petersen, P. Kloprogge, J.S. Risbey, W. Tuinstra, M.B.A. van Asselt, J.R. Ravetz (2004), RIVM/MNP Guidance for Uncertainty Assessment and Communication: Tool Catalogue for Uncertainty Assessment Utrecht University & RIVM.
H. Visser, A.C. Petersen, A.H.W. Beusen, P.S.C. Heuberger, P.H.M. Janssen (2006). Guidance for uncertainty assessment and communication: checklist for uncertainty in spatial information and visualising spatial uncertainty. PBL, Bilthoven.