Measurement of customer satisfaction with AIZ services for GIZ projects, follow-up survey 2022
To ensure continuous quality control, the Academy for International Cooperation (AIZ) has mandated CEval GmbH to conduct a customer satisfaction survey concerning AIZ services for GIZ projects. The aim is to determine the level of satisfaction of AIZ customers with their services in the field of technical vocational education and training.
For that purpose, a semi-standardized telephone survey was implemented among GIZ internal clients of AIZ services on matters of communication, consulting and services, work style and development orientation. Based on the findings, strengths and weaknesses of the HCD measures implemented by AIZ as well as resulting potential for improvement had been identified.
Development of a Monitoring- and Evaluation Concept for the Deutsche Stiftung für Internationale Rechtliche Zusammenarbeit e.V. [German Foundation for International Juridical Cooperation]
The German Foundation for International Juridical Cooperation e.V. (IRZ) supports partner countries in the development and consolidation of constitutional and market-economy structures on behalf of and with funds from the Federal Ministry of Justice and Consumer Protection (BMJV) since 1992. Among the approximately 25 partner states are new EU member states and accession candidates as well as other states in Southeast Europe, Central Asia, North Africa and the Middle East. The foundation supports these states in particular in the protection of human rights as well as in the areas of rule of law and justice, civil and commercial law, public law, criminal justice and European law.
The CEval GmbH supported IRZ in the development of a comprehensive monitoring and evaluation concept for the Foundation’s program line financed by the BMJV. The service included the operationalization of the target system of IRZ by means of suitable indicators and the development of a data collection plan. This included conducting an inventory of IRZ’s existing instruments harmonizing them with the new M&E concept. Within the framework of the implementation of the M&E concept, CEval GmbH had carried out a three-day capacity building training for the IRZ staff and accompany the test implementation of the monitoring instruments. Finally, the results of the development process had been documented in a handbook with hints for the practical implementation of the M&E concept.
Measurement of Customer Satisfaction with AIZ Services for GIZ Projects, Follow-up Survey 2020
To ensure continuous quality control, the Academy for International Cooperation (AIZ) had mandated CEval GmbH for the second time in a row to evaluate customer satisfaction with AIZ services for GIZ projects. For this purpose, a total of eleven GIZ internal clients had been interviewed by telephone between December 2020 and January 2021 and the data had been analyzed using descriptive statistics.
The evaluation design was divided into four key areas: communication, consulting and services, functioning and development Orientation. Based on the findings, both strengths and weaknesses of the HCD measures implemented by AIZ as well as resulting potential for improvement had been identified. Finally, the data had been compared with that from the previous year with regard to potential trends.
M&E support of two WaSH projects in Ethiopia and Kenya.
Knorr-Bremse Global Care e.V. (KBGC) is planning to implement two WASH projects in Ethiopia and Kenya. The common goal of the two projects is to reduce mortality and morbidity rates from water-related diseases. For this purpose, a series of measures adapted to the framework conditions in the respective implementation regions is planned, such as improving access to clean water, increasing the knowledge and skills of the actors involved, rehabilitating sanitary facilities, disseminating more energy-efficient stoves, health education and, finally, changing attitudes and behaviour among the target groups. These activities are intended to contribute to the improvement of the general living conditions of the target groups.
CEval GmbH is responsible for the scientific monitoring and evaluation of the two projects with the aim of assessing their effectiveness. The implementation will be carried out in close cooperation with national evaluators in both countries, while CEval GmbH is responsible for the preparation of the evaluation design and methodological concept, the training of those responsible for monitoring as well as accompaniment of the entire process.
Mid-Term Evaluation of the „Palestinian-German Science Bridge” (PGSB)
The program Palestinian-German Science Bridge is a pilot project financed by the German Federal Ministry of Education and Research (BMBF), which is jointly implemented by Forschungszentrum Jülich and the Palestinian Academy for Science and Technology (PALAST). It aims at establishing a joint education and research cooperation between Forschungszentrum Jülich and the Palestinian partner universities. To this end, the program combines measures focusing on student and researcher mobility, scientific exchange among faculty staff, the support of research collaborations and the reintegration of young Palestinian researchers into the Palestinian academic sector.
Forschungszentrum Jülich mandated CEval GmbH to counsel the Forschungszentrum with regard to the conceptualization of a mid-term evaluation of the program and to assist the Forschungszentrum in implementing it. The purpose of the evaluation was to assess the short- and medium-term results of the program as well as the prospects for long-term goal achievement. The objective, further, was to identify potentials for improvement in relation to the program.
Methodologically, the evaluation was based on a mix of qualitative and quantitative data collection and analysis methods. Data collection combines a document and secondary data analysis with guided interviews and semi-standardized online surveys among the project stakeholders. During data analysis, descriptive statistics and qualitative content analysis were applied. Due to the COVID-19 pandemic, the evaluation was planned as a remote evaluation.
Joint departmental strategic evaluation of the AA and BMZ engagement in Iraq
Within the framework of its crisis engagement in Iraq, the Federal Government intends to further embed monitoring and evaluation as instruments of learning and quality assurance, accountability and strategic steering. In this context, a joint evaluation of the Iraq engagement of the Federal Foreign Office (AA) and the Federal Ministry for Economic Cooperation and Development (BMZ) was conducted. This was intended to serve as a pilot project and, if possible, as a model for joint evaluations of both ministries in relevant fields of action and at their interfaces. The main objective of this joint evaluation was joint institutional learning. The AA component of the evaluation was realised by a consortium under the leadership of the GFA Consulting Group, in which CEval GmbH was involved; the BMZ component by the German Evaluation Institute (DEval). Both evaluation teams worked closely together in a so called Arbeitsgemeinschaft.
The aim of the evaluation was to conduct an independent and comprehensive analysis and assessment of the engagement of the two ministries in order to gain and use insights for (joint) steering. The evaluation provided insights into the relevance, effectiveness and overarching impacts, efficiency, sustainability and connectivity as well as coordination, coherence and complementarity of the Iraq engagement of both ministries. Furthermore, the evaluation provided recommendations for optimising the overall engagement and individual measures, structures and processes in order to adjust the current portfolio if necessary. Finally, learning experiences from the AA-BMZ engagement in Iraq were identified for similar engagement in other crisis contexts. The evaluation also served as an accompanying or final performance review of the country portfolio in accordance with the Federal Budget Code (Bundeshaushaltsordnung).
Methodologically, the evaluation was based on a broad mix of qualitative and quantitative data collection and evaluation methods. Due to pandemic-related travel restrictions, the data collection was realised or managed remotely by the consortium in both Germany and Iraq.
International Program for Development Evaluation Training (IPDET)
The International Program for Development Evaluation Training (IPDET) is an executive training program in development evaluation. Since 2018, it is managed jointly by the Center for Continuing Education at the University of Bern (Switzerland), the Center for Evaluation at Saarland University (Germany) and the Independent Evaluation Group of the World Bank.
IPDET aims to provide managers and practitioners in the field of evaluation with the tools required to evaluate policies, programs and projects at local, regional, national, and global levels, as well as to commission, manage and especially use those evaluations for decision-making. Further information on this year’s IPDET can be found on the website: https://ipdet.org/.
Development of an M&E framework, M&E support and facilitation of learning & exchange formats for the initiative “Improved livelihoods perspectives for young people in rural East Africa”
The German Federal Ministry for Economic Cooperation and Development (BMZ), Deutsche Stiftung Weltbevölkerung, Siemens Stiftung and Hanns R. Neumann Stiftung collectively intend to improve upon the prospects of youths in rural areas of East Africa. Within the joint initiative of Improved livelihoods perspectives for young people in rural East Africa they empower youths to actively shape developmental processes and to have their voices heard in the respective policy processes. Further, they intend to improve youths livelihoods through enhanced income generation and health conditions. Through this, they also aim to contribute to jointly fight root causes of migration. The initiative is implemented by three local implementing partners in Uganda, namely Action for Health Uganda, Hanns R. Neumann Stiftung Africa Ltd. and WHAVE Solutions.
The objective of this assignment was threefold: To develop an M&E Framework for the program, to support its monitoring and evaluation through the implementation of three assessments (baseline, mid-term and final assessment), and to facilitate learning and exchange during its pilot phase. For achieving these objectives, a participatory approach was employed, focusing on a strong stakeholder involvement from the planning stage through data collection to results interpretation. During the assessments a multi-method approach was applied, based on a quantitative quasi-experimental and a qualitative case-study design for achieving a comprehensive picture of the programs impact and sustainability, and eventually validating its intervention logic. While the quasi-experimental design comprises the collection of target and comparison group data from approx. 1,000 pax in total, the case-study design included interviews and Focus Group Discussions with key informants from implementing partners, target groups, local authorities and sector experts.
Evaluation Synthesis: Evidences from Central Project Evaluations of Transitional Development Assistance Projects
As a bridge between humanitarian aid and longer-term development cooperation, transition assistance is becoming more and more important, especially against the backdrop of increased conflicts worldwide.
Consequently, the Evaluation Unit of GIZ had commissioned the CEval GmbH to conduct an evaluation synthesis of transitional development assistance (TDA) projects that have been implemented in the period from January 2013 to April 2021.
Subject to this evaluation synthesis had been nine central project evaluations (CPE) of TDA projects and one CPE of a regional EU co-financed project with similar thematic priorities, implemented in four countries. Thereby, the evaluation covered diverse countries and regions: South Sudan, Somalia, Haiti, Ukraine, Iraq, Turkey, Lebanon, and Jordan. In addition to the CPEs, the synthesis had been based on interviews with project leaders and, as possible, with the respective evaluators. The aim was to identify TDA related and context-specific factors of success and failure of the projects and to derive recommendations for action. Methodologically, the synthesis was based on a mixed-method approach, consisting of a combination of qualitative content analysis and QCA (fuzzy set qualitative comparative analysis).
Preparation of an AMPA overview for 20 multilateral organizations as well as two general overviews
The Annual Multilateral Performance Assessment (AMPA) is an instrument of the Swiss Agency for Development and Cooperation (SDC) for the annual performance assessment of its priority multilateral organizations. The aim is to assess these organizations as neutrally and independently as possible, with little resource input on the basis of existing and accessible information. AMPA is based on 1) MOPAN reports (external source), 2) the CCM Annual Reports (SDC assessment) and 3) self-assessments of multilateral organizations (e.g. scorecards).
As part of a consulting evaluation assignment, CEval has worked with the client to develop concrete measures to optimize the performance assessment.
