Review of the evaluation of the Energy Innovation Agenda

Publication number: 2016-84
Author: C. Koopmans
Commissioned by: Ministry of Economic Affairs
Published by: SEO Amsterdam Economics

The Netherlands Enterprise Agency ( has carried out an evaluation of key parts of the Energy Innovation Agenda. The evaluation concerns €117 million in public funds that were spent on energy innovation in the period 2008-2016. Since RVO evaluates a scheme that RVO itself has implemented, the impression could arise that the evaluation may not be unbiased (“the student grades his own paper”). Therefore, RVO has asked Carl Koopmans to provide an independent review of the evaluation.

The overall conclusion of the review is that – given the limited possibilities to evaluate innovation policy – a good approach has been followed, which is also partly innovative. A lot of useful information has been collected. The evaluation not only gives rise to conclusions about past policy measures, but also to important recommendations for future innovation policy.

Koopmans makes the following recommendations for future policy and evaluations:

  • Make the set-up of future policy less complex and fragmented than the Energy Innovation Agenda;
  • Use the so-called ‘effect ladder’ in future evaluations as well;
  • In future evaluations of innovation policy, use the operationalisation of system failure and transformative failure presented in this study as a reference and source of inspiration;
  • Clearly define the distinction between system failure and transformative failure on the one hand and market failure on the other;
  • In future evaluations, do not use estimates of effectiveness by stakeholders or directly involved parties themselves.

Category: 2017, Carl Koopmans