Central project evaluation of the project Competitive African Cashew Value Chains for Pro-Poor Growth
The cashew value chain offers African producing countries interesting opportunities to create jobs, increase business and economic income and improve food security. Up to the time of the evaluation, however, the potential had hardly been exploited. At 300 to 550 kilograms of raw nuts per hectare, the yields of African producers were far below the yields in India and Vietnam and the processing capacities and labor productivity of the farms were still too low, which meant that only a small proportion of the raw nuts were processed in the African producing countries. The “Competitive African Cashew Value Chains for Pro-Poor Growth” project of the Deutsche Gesellschaft für internationale Zusammenarbeit GmbH supported the expansion and intensification of the cashew value chain in the six countries Benin, Burkina Faso, Côte d’Ivoire, Ghana, Mozambique and Sierra Leone.
The project was based on four levels of action: (i) improving the supply of high-performance seeds/seedlings through research and educating producers through training and consulting, (ii) advising processors of cashew nuts and by-products to improve process efficiency, (iii) intensifying business relations and exchange between all participants in the value chain, and (iv) promoting national and regional framework conditions by advising the relevant ministries and intensifying regional exchange between all actors.
CEval GmbH was commissioned by GIZ with the central project evaluation. The evaluation was aligned with the OECD DAC criteria and thus examined the relevance, effectiveness, efficiency, impact, sustainability and coherence of project implementation. Methodologically, the evaluation was based on the available quantitative secondary data of the project and expanded on this by conducting guided interviews and focus group discussions with project managers, stakeholders and target group representatives. Due to the corona pandemic prevailing at the time of the evaluation, the project was implemented as a remote evaluation. The evaluators from CEval GmbH worked together with local consultants in the African countries, who carried out the data collection in the field.
Scientific advice to the GIZ Evaluation Unit on the development of aggregated impact indicators.
The Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH contributes in many ways to solving regional and global problems. These include mitigating the effects of climate change, fighting poverty and hunger, or creating equal access to education and employment for men and women. To record the contributions made by its projects, GIZ has a number of instruments and procedures for impact measurement and evaluation. However, these tools and procedures are mainly geared toward reporting to the BMZ and generating evidence to improve individual interventions (PEVs) or to further develop the strategic direction of GIZ in the respective sector (USEs). At the level of the overall organization, however, there had been a lack of comparable approaches for presenting the contributions of GIZ as a whole, e.g. to the public or parliament (esp. EEZs).
In order to make GIZ capable of providing information at this level as well in the future, the Monitoring and Evaluation Unit initiated a process in 2014 to develop aggregated indicators that allow statements to be made about the joint contribution of as many projects as possible in the various sectors in which GIZ is active.
Evaluation of a Capacity Development Project on Child Protection Systems in Ethiopia and India
Worldwide across all socio-economic backgrounds, cultures and religions, children are exposed to a wide spectrum of risk factors in their daily lives. Considering the complexity and interconnectedness of underlying causes, functioning child protection systems have proven to be key to address risk factors and prevent violence, exploitation and abuse. Kindernothilfe, jointly with Karl Kübel Stiftung, Child Fund and terre des hommes, initiated the project titled Development and implementation of institutional child safeguarding policies as well as networking with local child protection systems through at least 240 organizations in nine countries in Asia and Africa in 2014.
As the project ended in December 2018, the consortium commissioned the Center for Evaluation in August 2018 to conduct an evaluation to understand the impact of the project as well as retrieve learnings and recommendations for future interventions of similar character. A crucial part of the assignment was to retrospectively develop the Theory of Change (ToC) of the project in a participatory manner, integrating consortium members, national trainers as well as the target group and final beneficiaries in the process. The ToC guided the evaluation while being further validated during the course of the evaluation. Field visits took place in Ethiopia and South India in October 2018. In total, six partner organizations had been visited and interviews and focus group discussions had been hold. Children are included in the evaluation by making use of participatory research tools, such as Picture Talk and Smiley-face scales. Each field visit ended with a one-day workshop to which all participating organizations of the respective region had been invited. During the workshop, preliminary results and the ToC were presented, discussed and validated. Finally, the various data were analyzed, triangulated and the results presented to the consortium partners.
International Program for Development Evaluation Training (IPDET)
The International Program for Development Evaluation Training (IPDET) is an executive training program in development evaluation. Since 2018, it is managed jointly by the Center for Continuing Education at the University of Bern (Switzerland), the Center for Evaluation at Saarland University (Germany) and the Independent Evaluation Group of the World Bank.
IPDET aims to provide managers and practitioners in the field of evaluation with the tools required to evaluate policies, programs and projects at local, regional, national, and global levels, as well as to commission, manage and especially use those evaluations for decision-making. Further information on this year’s IPDET can be found on the website: https://ipdet.org/.
Development of an M&E framework, M&E support and facilitation of learning & exchange formats for the initiative “Improved livelihoods perspectives for young people in rural East Africa”
The German Federal Ministry for Economic Cooperation and Development (BMZ), Deutsche Stiftung Weltbevölkerung, Siemens Stiftung and Hanns R. Neumann Stiftung collectively intend to improve upon the prospects of youths in rural areas of East Africa. Within the joint initiative of Improved livelihoods perspectives for young people in rural East Africa they empower youths to actively shape developmental processes and to have their voices heard in the respective policy processes. Further, they intend to improve youths livelihoods through enhanced income generation and health conditions. Through this, they also aim to contribute to jointly fight root causes of migration. The initiative is implemented by three local implementing partners in Uganda, namely Action for Health Uganda, Hanns R. Neumann Stiftung Africa Ltd. and WHAVE Solutions.
The objective of this assignment was threefold: To develop an M&E Framework for the program, to support its monitoring and evaluation through the implementation of three assessments (baseline, mid-term and final assessment), and to facilitate learning and exchange during its pilot phase. For achieving these objectives, a participatory approach was employed, focusing on a strong stakeholder involvement from the planning stage through data collection to results interpretation. During the assessments a multi-method approach was applied, based on a quantitative quasi-experimental and a qualitative case-study design for achieving a comprehensive picture of the programs impact and sustainability, and eventually validating its intervention logic. While the quasi-experimental design comprises the collection of target and comparison group data from approx. 1,000 pax in total, the case-study design included interviews and Focus Group Discussions with key informants from implementing partners, target groups, local authorities and sector experts.
Evaluation Synthesis: Evidences from Central Project Evaluations of Transitional Development Assistance Projects
As a bridge between humanitarian aid and longer-term development cooperation, transition assistance is becoming more and more important, especially against the backdrop of increased conflicts worldwide.
Consequently, the Evaluation Unit of GIZ had commissioned the CEval GmbH to conduct an evaluation synthesis of transitional development assistance (TDA) projects that have been implemented in the period from January 2013 to April 2021.
Subject to this evaluation synthesis had been nine central project evaluations (CPE) of TDA projects and one CPE of a regional EU co-financed project with similar thematic priorities, implemented in four countries. Thereby, the evaluation covered diverse countries and regions: South Sudan, Somalia, Haiti, Ukraine, Iraq, Turkey, Lebanon, and Jordan. In addition to the CPEs, the synthesis had been based on interviews with project leaders and, as possible, with the respective evaluators. The aim was to identify TDA related and context-specific factors of success and failure of the projects and to derive recommendations for action. Methodologically, the synthesis was based on a mixed-method approach, consisting of a combination of qualitative content analysis and QCA (fuzzy set qualitative comparative analysis).
Preparation of an AMPA overview for 20 multilateral organizations as well as two general overviews
The Annual Multilateral Performance Assessment (AMPA) is an instrument of the Swiss Agency for Development and Cooperation (SDC) for the annual performance assessment of its priority multilateral organizations. The aim is to assess these organizations as neutrally and independently as possible, with little resource input on the basis of existing and accessible information. AMPA is based on 1) MOPAN reports (external source), 2) the CCM Annual Reports (SDC assessment) and 3) self-assessments of multilateral organizations (e.g. scorecards).
As part of a consulting evaluation assignment, CEval has worked with the client to develop concrete measures to optimize the performance assessment.
Development of a systematic set of social inequality indicators in the context of German development cooperation
In the context of of the Sustainable Development (SDG 2030) goals, the United Nations have placed the reduction of social inequalities at the center of international development politics. Against this backdrop, altering social inequalities has gained importance in the field of German development cooperation. The evaluation of the impact of developmental projects on social inequalities requires a set of specific and valid indicators.
CEval has compiled a set of social inequality indicators and developed a systematic approach to categorize the indicators by vertical and horizontal dimensions of inequality, sectors and operational level. CEval has developed the adaption of relevant indicators to the context of german development cooperation and the projects of GIZ.
Meta-Evaluation of ADA Project and Programme Evaluations 2016-2018
Subject of the meta-evaluation were ADA evaluations of development cooperation projects and programmes implemented between 2016 and 2018. The objective was to assess the quality and usefulness of ADAs (internal and external) evaluations and to deliver recommendations on how to improve the ADA evaluation tool for the design and implementation of future evaluations.
A document analysis of the evaluation reports was at the heart of the meta-evaluation. Based on a detailed analysis protocol the quality of said reports was assessed. The use of such a protocol, which covered all relevant analysis dimensions, indicators and rating scales, guaranteed the transparency and reliability of the meta-evaluation results. In order to evaluate the usefulness for and use of the evaluations under study, their target audience (e.g. programme managers) were interviewed via guided individual and group interviews and in order to achieve the greatest possible representativeness of the results a semi-standardized online survey.
Meta-evaluation and synthesis of Christoffel Blindenmission evaluation reports
On behalf of the Christoffel Blindenmission (CBM), CEval conducted an evaluation and synthesis of 24 evaluation reports from 2016 and 2017. With its mandate to transform the lives of disabled people in the worlds poorest countries the project focus of CBM has been an exhaustive collaboration with as well as the promotion of local partners in developing countries and conflict areas.
The primary purpose of the evaluation and synthesis was to conduct a general assessment of the evaluation reports quality and summarize findings of those reports that pass the quality control. In a first step, to ensure homogeneity of data, the quality of reports was assessed systematically by creating a quality control analysis grid. Reports that fulfil minimum quality standards were then subjected to an in-depth analysis based on the evaluation questions of CBM. Main interest of the synthesis was to drew up general learnings along CBMs mandate areas and examine the actual contribution of joint projects to changes for CBM partners and clients. Lastly, by employing an online survey directed to end users of project evaluations, the evaluation inquired the usage and usefulness of the evaluation reports for CBM staff members in seven world regions.
