Learning from KfW’s ex-post evaluations? How conflicting objectives can limit their usefulness

Dörrbecker, Nicola M.
Discussion Paper 14/2023

Bonn: German Institute of Development and Sustainability (IDOS)

ISBN: 978-3-96021-217-1
DOI: https://doi.org/10.23661/idp14.2023
Price: 6 €

Dt. Ausg. u.d.T.:
Lernen aus Ex-post-Evaluierungen der KfW? Wie Zielkonflikte den Nutzen einschränken können
(Discussion Paper 12/2023)

The effectiveness of development cooperation (DC) is a topic of extensive debate in this policy field. Yet despite numerous review and evaluation formats designed to promote learning processes and hence enhance effectiveness, it is often impossible to document these improvements. Against this backdrop, the present paper aims to analyse the usefulness of ex-post evaluations (EPEs) by KfW Development Bank – both within KfW Development Bank and at the German Federal Ministry for Economic Cooperation and Development (BMZ), from which it receives its commissions. Research indicates that EPEs are conducted with great care. Moreover, EPEs can contribute to the legitimacy of (financial) DC, as project results are considered and presented in a structured manner. Nevertheless, the people interviewed for this study regard EPEs as (highly) subjective assessments and believe that these evaluations may under certain circumstances not be comparable with one another. Yet EPEs need to be comparable, because their overall ratings are used to calculate the success rate, which is currently around 81%. This in turn affects KfW’s reporting on its performance to BMZ and to the public. The data from the interviews shows that trade-offs during the production and use of EPEs appear to limit the usefulness of this format. EPEs are designed to deliver accountability to the public and to BMZ and to promote learning within KfW. These are conflicting objectives, however, as they would each require a different approach. According to those interviewed at KfW and BMZ, EPEs are seldom read or used. Interviewees explain that EPEs are rarely relevant to people working in operational areas, as the evaluations are not published until several years after the project concerned has been completed and only occasionally contain information that is relevant to current projects. The evaluations cannot be conducted sooner, however, as otherwise they would not be able to assess the sustainability and development impact of a project. Moreover, interviews and evidence from other studies indicate that EPEs are of limited relevance to political steering at BMZ, even in aggregated form. Nonetheless, the author believes that it would not be an option to no longer conduct EPEs, as they are the only way to review the development impact and sustainability of a representative number of projects in an affordable way, thus forming the basis for delivering accountability. Reconciling the conflicting goals of learning and accountability is challenging. For the learning component, it would appear to be a good idea to make greater use of cross-sectional analyses and to establish a central support structure for all implementing organisations and BMZ with a view to compiling all the key information from the evaluations and forwarding it to both BMZ and KfW and to the partner countries in a form tailored to meet their needs. For the accountability component, transparency also needs to be enhanced by making completed evaluation reports available to the public promptly and in full. In addition to an evaluation of international research literature, this paper particularly draws on empirical interview data. A total of 13 specifically selected experts from the German DC system were interviewed. This interview data thus forms an illustrative but not representative sample.