The scandal at the Netherlands Environmental Assessment Agency

In early 1999 a senior statistician (De Kwaadsteniet) of the Netherlands National Institute for Public Health and the Environment (RIVM; nowadays the Netherlands Environmental Assessment Agency PBL) accused the institute of ‘lies and deceit’ in their State of the Environment Reports and other studies. He criticized RIVM for basing their studies almost completely upon the ‘virtual reality’ of poorly validated computer models and seldom on real measurements. According to De Kwaadsteniet, RIVM presents these results as point values with unjustified significant digits and without clarifying the uncertainties. His criticism was published in a Dutch quality newspaper (Trouw). It triggered a vehement public debate on the credibility, reliability and quality of environmental statistics produced by model-studies by semigovernmental research institutes such as RIVM. The case received front page and prime time coverage in the mass media and provoked questions and debate in the Netherlands parliament. RIVM responded by immediately suspending the employee and serving a writ on him against public speaking, which he ignored, consequently increasing the controversial nature of the debate.

The context of this case is that RIVM has, over the past decade, become the authoritative provider of environmental statistics and forecasts, which have provided the basis for the Netherlands environmental policy. RIVM had recently obtained legal status as the office for environmental policy assessment of the Netherlands and its reports have been very influential in policy debates on environmental issues.
 
The scope of the media-debate that followed can be sketched by providing an anthology of headlines from the quality newspapers:
  • “Environmental institute lies and deceits”;
  • “Society has a right on fair information, RIVM does not provide it”;
  • “Then truth is less important...”;
  • “The bankruptcy of the numbers”;
  • “Dispute on environmental numbers, RIVM disapproves criticism of employee”; “Publish RIVM report together with criticism”;
  • “Measuring would not benefit the RIVM research”;
  • “Agitation in parliament after criticism on environmental numbers”;
  • “RIVM angry about accusations by employee”;
  • “Kafka and the environmental numbers”;
  • “Stand for independence and dissent”;
  • “Calculation model is always better than a crystal ball”;
  • “Research poorly organized”;
  • “Parliament wants to hear Minister about RIVM, Minister takes a too laconic position”,
  • “RIVM operates in a mine-field”;
  • “Suspended employee summons RIVM for judge”;
  • “Not everything can be scientifically sound”;
  • “Not method but model determines outcome”;
  • “Models give meaning to measurements”;
  • “Thank the whistleblower”;
  • “Credibility crisis surrounding environmental numbers”;
  • “Wouldn’t competition be better for RIVM?”;
  • “Independent validation of environmental numbers necessity”;
  • “Commentary/homage to the whistleblower”;
  • “Work at RIVM not further examined”;
  • “De Kwaadsteniet disappointed in parliament”;
  • “RIVM wants to get rid of researcher De Kwaadsteniet”;
  • “Judge considers suspension of De Kwaadsteniet unjustified”;
  • “RIVM punishes critical employee after all”;
  • “Trade union: More protection for whistleblowers”;
  • “Aid for critical officials”;
  • “Quality of RIVM-data will be controlled”;
  • “Commentary: the whistleblower as sitting target”;
  • “RIVM cheats us with over-exact prognoses”;
  • “RIVM changes clothes”.
 
One of the immediate consequences of the debate has been a decision to carry out a comprehensive external review of the scientific quality of the methodology used by RIVM to produce the environmental numbers published annually in the State of the Environment Report (Milieubalans). One of the findings of this evaluation was that the quality of the methodology itself is in most cases sufficient in view of its function, but the documentation, management and communication of uncertainties needs substantial improvement.
 
It is interesting to note that the discourses that emerged around this scandal focus on the dominant position of computer models in science for policy in relation to the questions of credibility and quality. In their report ‘Understanding Risk, Informing Decisions in a Democratic Society’, the American National Research Council observed that: “Mistrust is often at the root of the conflicts that emerge over risk analysis” and “A combination of psychological tendencies to notice, believe, and give more weight to trust-destroying than to trustbuilding information, and social factors, such as the tendency of mass media to favour bad news and of some special interest groups to encourage distrust to influence policy debates, makes trust very fragile”. This mechanism is exactly what happened around the trust in the RIVM in the Netherlands. The case was for instance immediately utilized by environmental NGOs to dismiss a recent RIVM study on the possibility of growth of air traffic at Amsterdam Airport, within given environmental standards. Others immediately used the case to put the issue of money-flows to research institutes and the monopoly position of the RIVM on the policy agenda.
 
This example highlights the importance of openness about uncertainty and the need for well developed practices of knowledge quality assessment. Ignoring this need ultimately underminses the credibility of the scientific basis for (environmental) policies.
 

References:

J.P. van der Sluijs (2002), A way out of the credibility crisis around model-use in Integrated Environmental Assessment, Futures, 34, 133-146.
 
Arthur C. Petersen, Maria Hage, Albert Cath, and Jeroen P. van der Sluijs (in press), Post-Normal Science in Practice at the Netherlands Environmental Assessment Agency,  Science Technology & Human Values
 
Stern PC, Fineberg HV, editors. Understanding risk, informing decisions in a democratic society. Washington DC: National Research Council, National Academy Press; 1996.