Analysis of the 2018 Forum Fairer Handel consumer survey
Since 2007 Forum Fairer Handel has repeatedly mandated the implementation of representative surveys on Fair Trade among the German population. In 2018 the most recent survey has been conducted. Apart from various aspects of Fair Trade consumption the survey also covered questions on the attitudes of the interviewees concerning the political claims of Fair Trade.
Forum Fairer Handel has mandated CEval GmbH with the analysis of the survey data. Descriptive statistics have been applied during the analysis. Furthermore, in relation to some questions the differences in responses between groups have been checked. Whenever data from previous surveys was available, developments and trends over time have also been demonstrated.
Integration of the Programme “Service Agency Communities in One World” (SKEW) in the M&E-System of Engagement Global
Engagement Global gGmbH was established in 2012 as a central institution to promote and coordinate the development policy commitment of civil society, municipal and private sector actors as well as for development policy information and education work. SKEW a program of Engagement Global focuses in particular on the promotion of municipal development cooperation. In order to be able to monitor both the appropriate use of the funds and above all their effectiveness, Engagement Global has created a comprehensive monitoring and evaluation (M&E) system for all departments and programs. Since SKEW was excluded in the development of the monitoring system at that time, CEval and ISPO were commissioned to integrate SKEW into the organization-wide monitoring system.
As with the design of the company-wide M&E system, first the theory of change and indicators were reviewed or developed. Then the objectives and indicators were integrated into the strategical claims of Engagement Global. In order to complete the creation of a monitoring system for the SKEW, a data collection plan were created and the associated data collection instruments had been (further) developed.
Ex-post assessment of the Strategic Fairtrade Funding Programme (SFFP), Phase II (2010-2013)
One of the key objectives of the Swiss Department on Trade Promotion is to support fair trading conditions with developing countries to which private labels and standards substantially contribute. Accordingly, as part of a donor consortium, the State Secretariat for Economic Affairs (SECO) contributed to the Strategic Fairtrade Funding Programme (SFFP) in Phase I from 2008 to 2009 and Phase II from 2010 to 2013. Main purpose of the harmonized funding was to enable disadvantaged producers to establish stronger linkages to consumers, to gain improved market access and to be empowered for combating poverty.
In 2014/2015 an evaluation was already carried out which examined Phase II of the funding programme. Further research on Fairtrade conducted include, among others, a follow-up impact study at the producer-level, covering six product and context settings, implemented by the Center for Evaluation (CEval GmbH) in 2017/2018. Four years after the SFFP ended, it was now of interest for SECO to assess the level of achievement of the strategic indicators set in the Logical Framework of the SFFP as well as SECOs contribution to the same.
The objective of the ex-post assessment was hence threefold: (i) to valorize existing information on the effectiveness and impact of SECOs support to the Fairtrade system, (ii) to complement findings from previous studies with new data on relevance and sustainability of the program and (iii) to distill key findings, recommendations and lessons learnt in view of the SECOs future engagement in Fairtrade. To this end, CEval examined relevant secondary data and collected additional primary data from stakeholders at FT headquarters, three FT regional networks and FT USA.
Meta-evaluation of project and programme evaluations 2015-2017
The Ministry for Foreign Affairs of Finland commissioned CEval together with Particip and Indufor to conduct a meta-evaluation of their evaluations of development cooperation programmes and projects. The Finnish Ministry is regularly conducting meta-evaluations and subject to this meta-evaluation are the evaluations from 2015-2017.
The meta-evaluation has been initiated to improve evaluation reports, evaluation management practices and evaluation capacity development and furthermore to identify possible gaps in the Finnish evaluation practice. Furthermore, also the content of the evaluation reports was the focus e.g. the success in achieving the DAC-Criteria and further Finnish Cross Cutting Topics like gender equality or climate sustainability were evaluated.
A document analysis of the evaluation reports has been at the heart of the meta-evaluation to assess first the quality and afterwards the content of the evaluation reports. Different assessment tools were developed to provide a feasible and reliable assessment process. In the quality check the evaluation reports were reviewed for the existence of information essential to understand the evaluation object, process and results. Additionally, methodological requirements were assessed e.g. data and method triangulation or the logical flow from data over findings and conclusions to recommendations. Another tool was used for the content analysis in which the achievements with respect to the different evaluation criteria as well as conclusions and recommendations were analysed in detail in a semi-structured way.
University cooperation with the University of Costa Rica (UCR) and the Catholic Pontifical University of Ecuador (PUCE) supported through the DAAD funding programme „Subject-Related Partnerships with Institutions of Higher Education in Developing Countries
The Saarland University (UdS) and the University of Costa Rica (UCR) have been maintaining an intensive cooperation since 2008, which was financially supported by the DAAD programme Subject-Related Partnerships with Institutions of Higher Education in Developing Countries between 2012 and 2015. Through the DAAD University Advisory Program, the preceding collaboration had already received financial support. Particularly in the light of an ever-rising need for evaluations and competencies in the field of evaluation, both partners are zealous to continue the collaboration within the framework of the DAAD funding programme and to expand it sustainably by the integration of a second partnering institute of higher education. The Catholic Pontifical University of Ecuador (PUCE) could be won for this objective.
Crucial elements of this third phase of cooperation were supposed to be the implementation of a new blended learning Masters study program in evaluation at the PUCE, and the implementation of a further educational opportunity in the blended learning format, respectively, at the Center for Research and Training in Public Administration (CICAP) at the UCR, and quality assurance of the formation and further education in evaluation in Costa Rica, Ecuador, and Latin America (amelioration of formation and further education as well as research in evaluation). Moreover, the establishment of a Latin American university network in the discipline of evaluation (network formation and internationalization), the strengthening of practical orientation within the academic formation in evaluation and the contribution to evidence-based policy making (application of scientific evaluative research in political practice), and, eventually, the consolidation and institutional expansion of the cooperation between the universities involved and the research institute CICAP (triangular and South-South-cooperation) were on the agenda of the program. The respective cooperation was intended to last for four years (2017-2020).
Development of an operational guidebook for integrating and using impact evaluation-based M&E approaches in the field of climate change adaption
Climate change adaption projects require methodologically elaborated Monitoring and Evaluation (M&E) Systems that are not limited to measuring Project Outputs but also Need to include outcomes and further Impacts as well as their attribution to the measures undertaken.
GIZ engaged the Center for Evaluation (CEval) for technical support to develop an operationally viable impact evaluation and provided examples for their applicability in adaptation projects. Results were presented in a guidebook for integrating and using impact evaluation-based M&E approaches in the field of climate change Adaptation. Target Groups of the Framework are ministries, donors, and other related institutions that are looking for the underlying Drivers of Adaptation and evidence-based findings to improve the design of future climate change adaptation projects.
External Evaluation of the Netherlands Fellowship Programme (NFP II) and Netherlands Initiative for Capacity Development in Higher Education (NICHE II)
On behalf of the Dutch Ministry of Foreign Affairs (MFA) the CEval implemented an external Evaluation of the Dutch Fellowship Programme (NFP II) and Netherlands Initiative for Capacity Development in Higher Education (NICHE II). Financed by the MFA and administered by the Netherlands Organisation for International Cooperation in Higher Education ‘EP-Nuffic’, these programs offer capacity development in higher education for developing partner countries.
The overall aim of NFP and NICHE is to help reduce quantitative and qualitative shortage of trained professional staff in developing countries and to build sustainable capacity within the poverty reduction Framework. The NFP provides Fellowships to mid-career professionals to participate in Trainings and postgraduate courses in the Netherlands. NICHE is a capacity building programme to improve the quality, relevance and gender sensitiveness of post-secondary education in NICHE-countries.
The evaluation covered the period of 2002-2015 while a focus was on the latest project phase since 2012 which put an additional emphasis on economic diplomacy and closer alignment with the four priority themes of the Dutch development cooperation strategy (water, sexual reproductive health and rights, security and rule of law and food security). The purpose of the evaluation was to account for the Netherlands funding provided for the two programs under the current project cycle and to gain lessons for future policy development and implementation. Within the scope of the evaluation, both programs were analysed regarding the OECD/DAC criteria of relevance, efficiency, effectiveness, impact and sustainability. Recommendations for possible optimization of the program were developed on the basis of the findings. The evaluation was conducted in cooperation with ACE Europe, a Belgian consultancy.
Program Evaluation “Support in Strengthening Regional Cooperation and Integration in the Field of Quality Infrastructure (QI) in SAARC”
With this regional Project, PTB Germany aims to reduce the technical barriers to trade by fostering cooperation among the SAARC countries (India, Pakistan, Bangladesh, Nepal, Buthan, Maldives, Afghanistan and Sri Lanka) in the field of metrology, accreditation and standardization. The Project targeted professional and Management staff of national QI-institutions as well as ministries relevant to the QI-System. Amongst others, measures lige Trainings and awareness raising Events have been taken and consultancy has been provided.
The evaluation combined qualitative and quantitative methods (mixed method design). A document Analysis and qualitative interviews with different stakeholders from all SAARC countries during a field visit to Bangladesh, Bhutan and India, and an online Survey of former metrology and accreditation Training participants in awareness raising Events have been carried out.
By statistical data and qualitative Content Analysis, the evaluation aimed to provide empirical evidence about the relevance, effectiveness, efficiency, impact and sustainability of the Project (DAC criteria) and the success factores for the observed results and change processes (Capacity Works criteria: strategy, cooperation, steering structure, processes, learning and innovation).
Thereby, it contributed to the further development of the Project approach and ist revision in view of future activities by PTB (in the region). The evaluation results had also been used as a Basis to decide on a follow-up project and on potentially necessary adaptions of its design.
Evaluation of the Projects “Sustainable Municipal Development through Partnership Projects” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects”
As part of Engagement Global – Service für Entwicklungsinitiativen (Service for Development Initiatives) the Service Agency Communities in One World in Germany ist the competence Center for local development cooperation. Founded in 2001 the Service Agency offers a platform for dialogue for municipalities in Germany and the South and supports development activities of local actors by providing them with Training, Information and advisory Services.
The two programs “Sustainable Municipal Development through Partnership Projects – Nachhaltige Kommunalentwicklung durch Partnerschaftsprojekte (Nakopa)” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects – Förderprogramm für kommunale Klimaschutz- und Klimaanpassungsprojekte (FKKP)” are core activities of the Service Agency. While Nakopa supports existing and new municipal partnership Projects in selected relevant fields of development cooperation, FKKP explicitly addresses municipal activities in the field of climate protection and climate change adaptation.
After the first Project cycle of three years has been completed, the current Evaluation aims at evaluating These two programs with respect to processual aspects of project Administration as well as Impacts of both the funded partnership Projects and the Support programs Nakopa and FKKP. Leading questions of this evaluation are guided by the OECD-DAC evaluation criteria and had been answered using the approach of contribution Analysis and documentary and empirical data. Hence, the evaluation was based on Monitoring data and Project documents on the one Hand and empirical data gathered through extensive partly standardized interviews and partly standardized online surweys with participating communities on the other hand.
Improving the Monitoring and reporting System to implement the Action plan "Inclusion of Persons with disabilities" in the German development co-operation
In February 2013, the Federal Ministry for Economic Cooperation and Development (BMZ) published an action plan for the inclusion of persons with disabilities (PWDs). In doing so, it follows ist duties emanating from Article 32 of the UN Convention on the rights of PWDs. The action plan provides Information on strategic Goals and Actions to design development cooperation of the Federal Republic of Germany towards a better inclusion of PWDs.
To assess the progress and to steer the implementation of planned acitivities an appropriate Monitoring and reporting System is necessary. Hence, the Center of Evaluation (CEval) has been contracted to Support the sector Project “Inclusion of PWDs” of the German Agency for International Cooperation (GIZ) and the departement 300 of the BMZ with its methodological expertise to develop an Analysis grid to evaluate the implementation of the action plan.