Start of project: April 2022
End of project: March 2023
New challenges and developments – Fair trade and fair procurement in 2022. Follow-up to the study “Does fair trade change Society?”
In 2015, leading stakeholders of the German fair trade movement commissioned a study on the impact of fair trade in Germany, which was carried out [Read More]In 2015, leading stakeholders of the German fair trade movement commissioned a study on the impact of fair trade in Germany, which was carried out by CEval GmbH. The study investigated the impact of fair trade in four societal areas, i.e., civil society, consumers, retailers and producers, and politics and the public sector. Since 2015 the fairtrade sector as well as the framework conditions in which it operated have changed. This includes, in particular, the environment for a fair public procurement.
Against that backdrop, Fairtrade Germany, Engagement Global Service für Entwicklungsinitiativen (Service for Development Initiatives) with its Servicestelle Kommunen in der Einen Welt (SKEW - Service Agency Communities in One World) and in cooperation with Forum Fairer Handel (Forum Fair Trade) and Weltladen-Dachverband (umbrella organization of German world shops) mandated CEval GmbH with the implementation of an updated and extended version of the 2015 study. The objective is to investigate the impact of fair trade in Germany between 2016 and 2022 under the changed framework conditions and taking into account the increased role of the German municipalities for fair trade and fair public procurement. Based on the findings, recommendations on how to further increase the effectiveness of fair trade and how to further support a fair public procurement shall be derived.
Methodologically, the study builds on the approach developed for the 2015 study and relies on qualitative contribution analysis. On the basis of a so-called theory of change and its underlying hypotheses, the study will investigate the changes that are intended by the fair trade community in the various areas of observation as well as the contributions that have been made by the fair trade community to these changes. In doing so, a mix of qualitative and quantitative methods of data collection and analysis will be applied. The starting point will be an extensive desk study of existing literature and secondary data, which will be complemented by empirical data collected by means of guided key informant interviews, (focus) group discussions and a semi-standardized survey.
Management: Sandra Bäthge | Co-Worker: Stefan Silvestrini, Maja FlaigStart of project: June 2021
End of project: March 2022
Meta-Evaluation of MFA’s Decentralised Programme and Project Evaluations in 2017-2020
Together with a consortium of Particip and Niras, CEval was again commissioned by the Ministry of Foreign Affairs of Finland (MFA) to conduct a meta-evaluation [Read More]Together with a consortium of Particip and Niras, CEval was again commissioned by the Ministry of Foreign Affairs of Finland (MFA) to conduct a meta-evaluation of decentralised evaluations of development cooperation programmes and projects covering 2017-2020. The purpose was to build upon and amend the findings of the previous meta-evaluation to generate more learnings on the use and usefulness of decentralised evaluations, investigate the developments since the previous meta-evaluation, and to see if the quality of evaluations has further increased. Thus, this meta-evaluation is responsible for two firsts in MFAs meta-evaluation tradition. It is the first to replicate the methodology of the previous meta-evaluation allowing for sound comparison of results. Furthermore, it is the first to take into consideration the perspective of primary users of evaluations to assess the actual use and usefulness of decentralised evaluations. New features were also added and analysis tools amended to accommodate changes and developments since the previous meta-evaluation. A multi-method approach of qualitative and quantitative analysis was conducted on both secondary and primary data to generate evidence-based findings, conclusion and recommendations.
This meta-evaluation consists of three components:
Component 1- Meta-evaluation of 80 decentralised evaluations to assess the methodological quality of the evaluation reports and corresponding ToRs. The recent meta-evaluation was also compared to the previous one to identify differences and trends and identify gaps regarding MFAs evaluation capacity.
Component 2 Content assessment of 72 evaluation reports, that passed minimal quality standards, to assess the overall quality of the interventions. The assessment included quality of interventions according to the OECD DAC criteria, consideration of cross-cutting objectives (CCOs) i.e. gender equality, non-discrimination and climate sustainability, human rights-based approach (HRBA) and Finnish policy priority areas (PPAs). A qualitative synthesis of evaluators lessons and recommendations to improve quality of Finnish development cooperation was also performed.
Component 3- Primary data of 26 interviewed MFA commissioners at embassy and headquarters level, and 119 survey responses from MFA commissioners and intervention implementers were collected to assess the actual use and usefulness of decentralized evaluations from their perspectives. Facilitating and hampering factors for usefulness, as wells as interviewees lessons and recommendations were also identified from this data.
Management: Susanne Johanna Väth | Co-Worker: Hansjörg Gaus, Stefan Silvestrini, Maja Flaig, Janis WickeAccompanying, Development M&E-System
Start of project: November 2017
End of project: November 2020
Monitoring & Evaluation of slum rehabilitation project in Brazil
Jointly with a local partner, Knorr-Bremse Global Care e.V. (KBGC) supports a training and rehabilitation project in a community in Grajaú, a slum district in [Read More]Jointly with a local partner, Knorr-Bremse Global Care e.V. (KBGC) supports a training and rehabilitation project in a community in Grajaú, a slum district in Sao Paolo, Brasil. The project aims at improving the living conditions of the community by qualifying young people for self-sustained income-generation and rehabilitating the area by building wooden houses. Main activities include training of youth in civil construction and set up of sustainable housing by residents of the slum and the trained youth.
KBGC commissioned CEval to establish a Monitoring & Evaluation framework to gather feedback on the training activities and trace quality of life outcomes over time to assess whether the programs objectives are being achieved. In order to identify the outcomes of the training on the qualification of the participants and their application of knowledge gained, a time-series design was applied, implying several phases of data collection with the help of standardized survey questionnaires. Outcomes of students were traced every six months over a period of three years, with baseline, monitoring and end line reports were drafted after each data collection to support organizational learning and improvement on project design.
Management: Stefan SilvestriniClient: Knorr-Bremse Global Care
Start of project: July 2020
End of project: November 2020
Midterm-Review of the FKPW program
In international comparison, German municipalities have comparably broad competencies due to their strong local self-administration. These competencies are also very valuable for development cooperation. Within [Read More]In international comparison, German municipalities have comparably broad competencies due to their strong local self-administration. These competencies are also very valuable for development cooperation. Within the framework of municipal partnerships, German municipalities and their partner municipalities jointly carry out projects in municipalities of the Global South. In the process, technical knowledge is transferred from the German municipality to the partner municipality. The Service Agency Communities in One World (SKEW in German) offers various ways of supporting the developmental engagement of municipal partnerships.
One of these funding opportunities is the personnel funding instrument "Experts for Municipal Partnerships Worldwide" (FKPW in German). Within the framework of the FKPW program, municipal partnerships can apply for the placement of an expert in the partner municipality to support the creation of local structures and capacity development. The aim of the program is to enhance the value of the municipal partnership and the commitment of the German municipality through the placement of an expert.
CEval GmbH was commissioned by GIZ to conduct a mid-term review of the FKPW program. The aim of the review was to bring together the views of the different actors involved in the program and to give recommendations for the further development of the program. The main objective therefore was institutional learning on the part of the two implementing organizations, GIZ and SKEW. The methodology of the review was purely qualitative and based on the approach of the open capturing of perspectives. For the review, interviews and focus group discussions were conducted with representatives of the German municipalities, the partner municipalities, with the experts and the steering team of the programme, as well as with representatives of the responsible department of the German Federal Ministry for Economic Cooperation and Development (BMZ in German) and representatives of various departments within the organizations GIZ and SKEW.
Management: Stefan SilvestriniEx Ante Evaluation
Start of project: October 2019
End of project: May 2020
Rapid Appraisal of two Moringa Investments
Moringa SCA SICAR is a private equity fund that has been established as a response to growing interest of the Financial and the Development sector [Read More]Moringa SCA SICAR is a private equity fund that has been established as a response to growing interest of the Financial and the Development sector on investments that provide for a financial, environmental and a social return at the same time. Moringa focuses on investments in agroforestry in African and Latin American Developing and Least Developed Countries. It applies agroforestry land use systems on a commercial basis with a nucleus commercial investment approach that integrates large numbers of neighboring smallholder farmers and processors into its business model through contractual arrangements. The Agroforestry Technical Assistance Facility (ATAF) was created by Moringa Partnership to provide technical assistance in relation to investments of the Fund with the goal to amplify and upscale positive environmental and social impacts triggered through Moringa investments.
CEval GmbH has conducted Baseline Rapid Appraisals for two Moringa Investments, focused specifically on the livelihood situation of the farmers and other relevant target groups, such as factory workers. Methodologically, the Rapid Appraisals followed a mixed-methods approach, which included a small survey with farmers for each Rapid Appraisal, as well as interviews with the relevant stakeholders. Rapid Appraisals were conducted for Moringa investments in Benin and Togo.
Management: Stefan SilvestriniDevelopment M&E-System
Start of project: September 2019
End of project: November 2019
Development of a systematic set of social inequality indicators in the context of German development cooperation
In the context of of the Sustainable Development (SDG 2030) goals, the United Nations have placed the reduction of social inequalities at the center of [Read More]In the context of of the Sustainable Development (SDG 2030) goals, the United Nations have placed the reduction of social inequalities at the center of international development politics. Against this backdrop, altering social inequalities has gained importance in the field of German development cooperation. The evaluation of the impact of developmental projects on social inequalities requires a set of specific and valid indicators.
CEval has compiled a set of social inequality indicators and developed a systematic approach to categorize the indicators by vertical and horizontal dimensions of inequality, sectors and operational level. CEval has developed the adaption of relevant indicators to the context of german development cooperation and the projects of GIZ.
Management: Stefan SilvestriniMeta-Evaluation
Start of project: August 2018
End of project: December 2018
Meta-Evaluation of ADA Project and Programme Evaluations 2016-2018
Subject of the meta-evaluation were ADA evaluations of development cooperation projects and programmes implemented between 2016 and 2018. The objective was to assess the quality [Read More]Subject of the meta-evaluation were ADA evaluations of development cooperation projects and programmes implemented between 2016 and 2018. The objective was to assess the quality and usefulness of ADAs (internal and external) evaluations and to deliver recommendations on how to improve the ADA evaluation tool for the design and implementation of future evaluations.
A document analysis of the evaluation reports was at the heart of the meta-evaluation. Based on a detailed analysis protocol the quality of said reports was assessed. The use of such a protocol, which covered all relevant analysis dimensions, indicators and rating scales, guaranteed the transparency and reliability of the meta-evaluation results. In order to evaluate the usefulness for and use of the evaluations under study, their target audience (e.g. programme managers) were interviewed via guided individual and group interviews and in order to achieve the greatest possible representativeness of the results a semi-standardized online survey.
Management: Stefan Silvestrini | Co-Worker: Sandra Bäthge, Teresa BöttcherClient: Austrian Development Agency
Ex Post Evaluation
Start of project: September 2018
End of project: October 2018
Evaluation of hazard prevention and damage after the heavy rain events in Saarland in May/June 2018
In May and June 2018, Saarland was hit by heavy rainfalls which caused high damages in a number of municipalities. As a consequence, the Saarland [Read More]In May and June 2018, Saarland was hit by heavy rainfalls which caused high damages in a number of municipalities. As a consequence, the Saarland Ministry for the Environment and Consumer Protection (MUV) commissioned CEval GmbH to review the measures of hazard prevention and disaster management in the wake of these events. For this purpose, an analysis of strengths and weaknesses of hazard prevention and disaster management with regard to prevention, deployment and after-care was carried out in order to lay the foundation for identifying potential improvements and evidence-based recommendations. In addition, the damages caused by the heavy rainfalls should be assessed.
To analyse the strengths and weaknesses of hazard prevention and disaster management, 62 guided interviews were conducted with representatives of the Ministry of Interior, Construction and Sport, fire-fighting coordinators, local aid organisations and mayors of the most affected municipalities. The results of the content analysis of these interviews formed the basis for identifying potential for improvement and deriving concrete recommendations for action. The evaluation of damages in the area of municipal infrastructure, agriculture and private households was carried out quantitatively on the basis of the data provided by the client.
Management: Stefan Silvestrini | Co-Worker: Hansjörg Gaus, Fabian OttoStart of project: July 2018
End of project: September 2018
Accompanying evaluation of the support program of the AMASE Masters-program
Together with a consortium of three other European universities (Luleå Technical University, Sweden, Barcelona Tech, Spain and Université de Lorraine, France) the European School of [Read More]Together with a consortium of three other European universities (Luleå Technical University, Sweden, Barcelona Tech, Spain and Université de Lorraine, France) the European School of Materials (EUSMAT) at Saarland University offers the international Masters programme AMASE (Advanced Materials Science and Engineering). Its students have the opportunity to participate in an Integration Week in Saarbrücken right at the beginning of their studies, which aims at providing an orientation with regards to the course of the AMASE programme as well as strengthening the students? soft skills through a number of trainings. After the first year of study in the two-year AMASE programme, the students furthermore have the opportunity to take part in a Professional Summer School aiming at facilitating their career entry.
Both events are part of the AMASE support programme and were evaluated within the context of an accompanying evaluation. The main goal was to determine the use of those events for their respective target groups. In methodological terms two standardized surveys as well as a number of focus group discussions were conducted with participants of the two events.
Management: Stefan Silvestrini | Co-Worker: Sandra Bäthge, Cornelia RömlingClient: European School of Materials
Start of project: March 2018
End of project: August 2018
Analysis of the 2018 Forum Fairer Handel consumer survey
Since 2007 Forum Fairer Handel has repeatedly mandated the implementation of representative surveys on Fair Trade among the German population. In 2018 the most recent [Read More]Since 2007 Forum Fairer Handel has repeatedly mandated the implementation of representative surveys on Fair Trade among the German population. In 2018 the most recent survey has been conducted. Apart from various aspects of Fair Trade consumption the survey also covered questions on the attitudes of the interviewees concerning the political claims of Fair Trade.
Forum Fairer Handel has mandated CEval GmbH with the analysis of the survey data. Descriptive statistics have been applied during the analysis. Furthermore, in relation to some questions the differences in responses between groups have been checked. Whenever data from previous surveys was available, developments and trends over time have also been demonstrated.
Management: Stefan SilvestriniClient: Forum Fairer Handel
