Measurement of Customer Satisfaction with AIZ Services for GIZ Projects, Follow-up Survey 2020
To ensure continuous quality control, the Academy for International Cooperation (AIZ) had mandated CEval GmbH for the second time in a row to evaluate customer satisfaction with AIZ services for GIZ projects. For this purpose, a total of eleven GIZ internal clients had been interviewed by telephone between December 2020 and January 2021 and the data had been analyzed using descriptive statistics.
The evaluation design was divided into four key areas: communication, consulting and services, functioning and development Orientation. Based on the findings, both strengths and weaknesses of the HCD measures implemented by AIZ as well as resulting potential for improvement had been identified. Finally, the data had been compared with that from the previous year with regard to potential trends.
M&E support of two WaSH projects in Ethiopia and Kenya.
Knorr-Bremse Global Care e.V. (KBGC) is planning to implement two WASH projects in Ethiopia and Kenya. The common goal of the two projects is to reduce mortality and morbidity rates from water-related diseases. For this purpose, a series of measures adapted to the framework conditions in the respective implementation regions is planned, such as improving access to clean water, increasing the knowledge and skills of the actors involved, rehabilitating sanitary facilities, disseminating more energy-efficient stoves, health education and, finally, changing attitudes and behaviour among the target groups. These activities are intended to contribute to the improvement of the general living conditions of the target groups.
CEval GmbH is responsible for the scientific monitoring and evaluation of the two projects with the aim of assessing their effectiveness. The implementation will be carried out in close cooperation with national evaluators in both countries, while CEval GmbH is responsible for the preparation of the evaluation design and methodological concept, the training of those responsible for monitoring as well as accompaniment of the entire process.
Mid-Term Evaluation of the „Palestinian-German Science Bridge” (PGSB)
The program Palestinian-German Science Bridge is a pilot project financed by the German Federal Ministry of Education and Research (BMBF), which is jointly implemented by Forschungszentrum Jülich and the Palestinian Academy for Science and Technology (PALAST). It aims at establishing a joint education and research cooperation between Forschungszentrum Jülich and the Palestinian partner universities. To this end, the program combines measures focusing on student and researcher mobility, scientific exchange among faculty staff, the support of research collaborations and the reintegration of young Palestinian researchers into the Palestinian academic sector.
Forschungszentrum Jülich mandated CEval GmbH to counsel the Forschungszentrum with regard to the conceptualization of a mid-term evaluation of the program and to assist the Forschungszentrum in implementing it. The purpose of the evaluation was to assess the short- and medium-term results of the program as well as the prospects for long-term goal achievement. The objective, further, was to identify potentials for improvement in relation to the program.
Methodologically, the evaluation was based on a mix of qualitative and quantitative data collection and analysis methods. Data collection combines a document and secondary data analysis with guided interviews and semi-standardized online surveys among the project stakeholders. During data analysis, descriptive statistics and qualitative content analysis were applied. Due to the COVID-19 pandemic, the evaluation was planned as a remote evaluation.
Joint departmental strategic evaluation of the AA and BMZ engagement in Iraq
Within the framework of its crisis engagement in Iraq, the Federal Government intends to further embed monitoring and evaluation as instruments of learning and quality assurance, accountability and strategic steering. In this context, a joint evaluation of the Iraq engagement of the Federal Foreign Office (AA) and the Federal Ministry for Economic Cooperation and Development (BMZ) was conducted. This was intended to serve as a pilot project and, if possible, as a model for joint evaluations of both ministries in relevant fields of action and at their interfaces. The main objective of this joint evaluation was joint institutional learning. The AA component of the evaluation was realised by a consortium under the leadership of the GFA Consulting Group, in which CEval GmbH was involved; the BMZ component by the German Evaluation Institute (DEval). Both evaluation teams worked closely together in a so called Arbeitsgemeinschaft.
The aim of the evaluation was to conduct an independent and comprehensive analysis and assessment of the engagement of the two ministries in order to gain and use insights for (joint) steering. The evaluation provided insights into the relevance, effectiveness and overarching impacts, efficiency, sustainability and connectivity as well as coordination, coherence and complementarity of the Iraq engagement of both ministries. Furthermore, the evaluation provided recommendations for optimising the overall engagement and individual measures, structures and processes in order to adjust the current portfolio if necessary. Finally, learning experiences from the AA-BMZ engagement in Iraq were identified for similar engagement in other crisis contexts. The evaluation also served as an accompanying or final performance review of the country portfolio in accordance with the Federal Budget Code (Bundeshaushaltsordnung).
Methodologically, the evaluation was based on a broad mix of qualitative and quantitative data collection and evaluation methods. Due to pandemic-related travel restrictions, the data collection was realised or managed remotely by the consortium in both Germany and Iraq.
Development of a systematic set of social inequality indicators in the context of German development cooperation
In the context of of the Sustainable Development (SDG 2030) goals, the United Nations have placed the reduction of social inequalities at the center of international development politics. Against this backdrop, altering social inequalities has gained importance in the field of German development cooperation. The evaluation of the impact of developmental projects on social inequalities requires a set of specific and valid indicators.
CEval has compiled a set of social inequality indicators and developed a systematic approach to categorize the indicators by vertical and horizontal dimensions of inequality, sectors and operational level. CEval has developed the adaption of relevant indicators to the context of german development cooperation and the projects of GIZ.
Meta Evaluation of CoE Project and Programme Evaluations (2016-2019)
In the framework of its Quality Assurance of CoE Decentralised Evaluation Reports, the Council of Europe had mandated the CEval GmbH to perform an independent quality check of a sample of 32 CoE decentralised evaluation reports prepared by various Major Administrative Entities for the period 2016-2019. The overarching objective of this stocktaking is to support the establishment of an Evaluation Framework and the designing of a Quality Assurance System for the CoE.
Accordingly, the quality check informs about the extent to which the decentralised evaluations fulfil widely accepted international standards in terms of quality and gives recommendations on what aspects to improve; provides an overall assessment of the major strengths and weaknesses of the decentralised evaluation reports; identifies key findings, recommendations and lessons learnt to improve the performance of CoE institutional capacity-building; and finally provides recommendations for the establishment of a framework for quality assurance. The assessment has been conducted based on an analysis grid including all relevant evaluation criteria and indicators, with a detailed guidance, in order to assure inter-rater reliability and thus to provide for maximum objectivity of the results.
Sector evaluation of climate projects within the framework of the BMZ Special Initiative on Climate of the Hanns Seidel Foundation e.V. 2019
The Hanns Seidel Foundation (HSS) has been receiving funds from the BMZ Special Initiative on Climate since 2011 and has carried out project activities in the fields of climate, environment and energy in numerous countries since then. The measures in this area are intended to make a contribution to environmental protection and nature conservation as a whole, to combating and adapting to climate change, to sustainable energy supply and resource efficiency, and to sustainable and environmentally conscious agriculture and forestry. A basic strategy was developed from the context of Hanns Seidel Foundation’s current work abroad in the fields of climate, environment and energy and its attempts to find solutions to these topics at an international level. This strategy was evaluated by CEval GmbH using three projects in India, Bolivia and Namibia as examples.
From a methodological point of view, the evaluation included a theory-based part, which consisted of the reconstruction of the Theories of Change (ToC) of the three projects. In addition, the three selected projects were evaluated at project level, for which field phases in all three countries and interviews with partners and target groups have been conducted. In addition, the evaluation included a synthesis part which referred to the climate strategy of the Hanns Seidel Foundation in general and in which the results of the evaluations of the three projects were bundled and supplemented with information from the headquarters of the HSS in Munich.
Rapid Appraisal of two Moringa Investments
Moringa SCA SICAR is a private equity fund that has been established as a response to growing interest of the Financial and the Development sector on investments that provide for a financial, environmental and a social return at the same time. Moringa focuses on investments in agroforestry in African and Latin American Developing and Least Developed Countries. It applies agroforestry land use systems on a commercial basis with a nucleus commercial investment approach that integrates large numbers of neighboring smallholder farmers and processors into its business model through contractual arrangements. The Agroforestry Technical Assistance Facility (ATAF) was created by Moringa Partnership to provide technical assistance in relation to investments of the Fund with the goal to amplify and upscale positive environmental and social impacts triggered through Moringa investments.
CEval GmbH has conducted Baseline Rapid Appraisals for two Moringa Investments, focused specifically on the livelihood situation of the farmers and other relevant target groups, such as factory workers. Methodologically, the Rapid Appraisals followed a mixed-methods approach, which included a small survey with farmers for each Rapid Appraisal, as well as interviews with the relevant stakeholders. Rapid Appraisals were conducted for Moringa investments in Benin and Togo.
Evaluation of two school rehabilitationprojects in Nepal
During the two massive earthquakes in Nepal in 2015, several thousand schools got destroyed. Together with the consortium partners Helvetas and Swiss Red Cross, Caritas Switzerland built and rehabilitated 36 schools in the hard to reach district of Sindhupalchok. The Project encompassed trainings regarding disaster risk reduction, operation and maintainance, WASH (water, sanitation, hygiene) and social interaction with drop out students.
The final evaluation was based on three of the Core Humanitarian Standards (appropriateness and relevance, effectiveness and timeliness, preparedness and resilience) in order to draw lessons-learnt and give recommendations for further humanitarian outreaches.
The evaluation utilized different methods: next to interviews with the leadership of the project, representatives of national authorities, team members, head of schools and mayors; Focus Group Discussions gave voice to school management committees and teachers. A comprehensive survey among hundreds of students, parents and teachers enabled a in-depth quantitative analysis.
Central project evaluation of the project Competitive African Cashew Value Chains for Pro-Poor Growth
The cashew value chain offers African producing countries interesting opportunities to create jobs, increase business and economic income and improve food security. Up to the time of the evaluation, however, the potential had hardly been exploited. At 300 to 550 kilograms of raw nuts per hectare, the yields of African producers were far below the yields in India and Vietnam and the processing capacities and labor productivity of the farms were still too low, which meant that only a small proportion of the raw nuts were processed in the African producing countries. The “Competitive African Cashew Value Chains for Pro-Poor Growth” project of the Deutsche Gesellschaft für internationale Zusammenarbeit GmbH supported the expansion and intensification of the cashew value chain in the six countries Benin, Burkina Faso, Côte d’Ivoire, Ghana, Mozambique and Sierra Leone.
The project was based on four levels of action: (i) improving the supply of high-performance seeds/seedlings through research and educating producers through training and consulting, (ii) advising processors of cashew nuts and by-products to improve process efficiency, (iii) intensifying business relations and exchange between all participants in the value chain, and (iv) promoting national and regional framework conditions by advising the relevant ministries and intensifying regional exchange between all actors.
CEval GmbH was commissioned by GIZ with the central project evaluation. The evaluation was aligned with the OECD DAC criteria and thus examined the relevance, effectiveness, efficiency, impact, sustainability and coherence of project implementation. Methodologically, the evaluation was based on the available quantitative secondary data of the project and expanded on this by conducting guided interviews and focus group discussions with project managers, stakeholders and target group representatives. Due to the corona pandemic prevailing at the time of the evaluation, the project was implemented as a remote evaluation. The evaluators from CEval GmbH worked together with local consultants in the African countries, who carried out the data collection in the field.
