The Critical Appraisal of Scientific Inquiries with Policy Implications

Under the title "The Critical Appraisal of Scientific Inquiries with Policy Implications", Clark and Majone (1985) presented one of the first comprehensive frameworks for quality assessment in the science policy interface. The framework acknowledges that each actor that has a stake in quality control has a different role in the process of critical evaluation. For instance, scientists will emphasize other criteria in quality control than policy-makers. Further, Clark and Majone’s taxonomy distinguishes three general modes of critical appraisal: the input, the output and the process by which inquiry is conducted. Input refers to data; methods, people, competence, (im)matureness of field etc. Output relates to questions as whether the problem is solved and the hypothesis tested. Process concerns issues such as good scientific practice, procedures for review, documenting etc.

The resulting framwork is displayed below.

In addition, Clark and Majone proposed four meta quality criteria: adequacy, value, effectiveness an legitimacy. Adequacy covers issues such as reliability, reproducibility, uncertainty analysis etc.
Value has three aspects: Internal: how well is the study carried out? and external: fitness for purpose or  fitness for function. Personal value has to do with subjectivity, preferences, choices, assumptions and biases. Effectiveness is about the question whether it help to solve practical problems. Legitimacy has tow aspects: numinous, which is about natural authority, independance, credibility and competence, and civil, which has to dow with compliance with agreed procedures.

Reference
W.C. Clark and G. Majone, The Critical Appraisal of Scientific Inquiries with Policy Implications, in: Science Technology and Human Values, 10 (3), 6-19, 1985.