Quality assurance

Quality assurance (QA) may be defined as protocols and guidelines to support the proper application of (often model based) scientific analysis for policy support.

Quality is defined as fitness for function. Important aims of QA are to ensure the use of best practise, to build consensus among the various actors involved and to ensure that the expected accuracy and performance are in accordance with the project objectives.

Key elements of QA procedures include:

(1) framing of the problem and definition of the purpose of the study;

(2) assessment of sources of uncertainties jointly by scientists, policymakers and stakeholders and establishment of accuracy requirements by translation of user's needs to preliminary performance criteria;

(3) performance of model validation tests, i.e. testing of model performance against independent data that have not been used for calibration in order to assess the accuracy and credibility of the model simulations for situations comparable to those where it is intended to be used for; and

(4) reviews carried out by independent auditors with subsequent consultation between the modeller, the water manager and possibly the stakeholders at different phases of the modelling project.

Many QA guidelines exist such as Middlemis (2000) and Van Waveren et al. (1999). For instance, the HarmoniQuA project (Scholten et al., 2007; Refsgaard et al., 2005a) has developed a comprehensive set of QA guidelines for multiple modelling domains combined with a supporting software tool, MoST
(downloadable via http://www.harmoniqua.org; for instance http://harmoniqua.wau.nl/public/Reports/SOA%20chapters/SOA.pdf).

QA improves the chances that best practise is used, it makes it possible to involve stakeholders into the modelling process in a formalised framework, and it improves the transparency and reproducibility. If not designed and performed thoroughly, QA may become a ‘rubber stamp’ and generate false credibility.

References

Middlemis, H., 2000. Murray-Darling Basin Commission. Groundwater Flow Modelling Guideline. Aquaterra Consulting Pty Ltd, South Perth, Western Australia. Project no. 125.

[**Recommended Reading] Ravetz, J.R.(1997), Integrated Environmental Assessment Forum: developing guidelines for "good practice ULYSSES Working Paper 97-1.

Refsgaard, J.C., Henriksen, H.J., Harrar, W.G., Scholten, H., Kassahun, A., 2005. Quality assurance in model based water management - review of existing practice and outline of new approaches. Environmental Modelling & Software 20, 1201-1215.

Scholten, H., Kassahun, A., Refsgaard, J.C., Kargas, T., Gavardinas, C., Beulens, A.J.M., 2007. A methodology to support multidisciplinary modelbasedwatermanagement. EnvironmentalModelling&Software 22, 743-759.

Van Waveren, R.H., Groot, S., Scholten, H., Van Geer, F., Wo¨sten, H., Koeze, R., Noort, J., 1999. Good Modelling Practice Handbook, STOWA. Utrecht, The Netherlands (in Dutch). English version from http://www.landandwater.tudelft.nl/Downloads/GMP-uk.pdf.