Tool Evaluation
Start of project: October 2018
End of project: August 2021
Preparation of an AMPA overview for 20 multilateral organizations as well as two general overviews
The Annual Multilateral Performance Assessment (AMPA) is an instrument of the Swiss Agency for Development and Cooperation (SDC) for the annual performance assessment of its [Read More]The Annual Multilateral Performance Assessment (AMPA) is an instrument of the Swiss Agency for Development and Cooperation (SDC) for the annual performance assessment of its priority multilateral organizations. The aim is to assess these organizations as neutrally and independently as possible, with little resource input on the basis of existing and accessible information. AMPA is based on 1) MOPAN reports (external source), 2) the CCM Annual Reports (SDC assessment) and 3) self-assessments of multilateral organizations (e.g. scorecards).
As part of a consulting evaluation assignment, CEval has worked with the client to develop concrete measures to optimize the performance assessment.
Management: Reinhard StockmannEx Post Evaluation, Impact Analysis
Start of project: August 2019
End of project: October 2019
Evaluation of two school rehabilitationprojects in Nepal
During the two massive earthquakes in Nepal in 2015, several thousand schools got destroyed. Together with the consortium partners Helvetas and Swiss Red Cross, Caritas [Read More]During the two massive earthquakes in Nepal in 2015, several thousand schools got destroyed. Together with the consortium partners Helvetas and Swiss Red Cross, Caritas Switzerland built and rehabilitated 36 schools in the hard to reach district of Sindhupalchok. The Project encompassed trainings regarding disaster risk reduction, operation and maintainance, WASH (water, sanitation, hygiene) and social interaction with drop out students.
The final evaluation was based on three of the Core Humanitarian Standards (appropriateness and relevance, effectiveness and timeliness, preparedness and resilience) in order to draw lessons-learnt and give recommendations for further humanitarian outreaches.
The evaluation utilized different methods: next to interviews with the leadership of the project, representatives of national authorities, team members, head of schools and mayors; Focus Group Discussions gave voice to school management committees and teachers. A comprehensive survey among hundreds of students, parents and teachers enabled a in-depth quantitative analysis.
Management: Reinhard StockmannClient: Caritas Suisse
Start of project: January 2014
End of project: June 2017
Assessing the Impacts of Multinational Corporations on Global Development and Value Creation (Global Value)
The importance of multinational corporations for furthering global economic and social development is well established. Corporations invest in foreign countries, provide jobs, and thus generate [Read More]The importance of multinational corporations for furthering global economic and social development is well established. Corporations invest in foreign countries, provide jobs, and thus generate formalized labor relations and income. Furthermore, they pay taxes that contribute to the provision of public services. At the same time, multinational corporations particularly have received criticism by being accused of supporting exploitative employment conditions and human rights abuses, causing environmental deterioration, and failing to mainstream responsible conduct in developing countries.
The GLOBAL VALUE project developed an innovative framework for assessing impacts of multinational corporations on issues related to sustainable development, working conditions, human rights, transparency, and anti-corruption. It shed light on institutional arrangement; analysed systems of governance for responsible business practices; explored responsible competitiveness; assessed the complementarity of public and private sector activities; and derived recommendations for decision makers in business, policy and non-governmental organisations.
The project was carried out by universities and civil society organisations from Europe and ICPC countries. A toolkit for impact evaluation was developed and tested in close collaboration with leading multinational corporations (BATA (garment, Bangladesh), OLAM (food, Tanzania) and NOKIA (ICT, India)).
The main responsibility of CEval was the development of an indicator-set for the framework which is based on Millennium Development Goals, human rights, gender & diversity, and anti-corruption & transparency and takes corporate management approaches (supply chain management, life cycle analysis and base-of-pyramid) as well as systems of governance for responsible business practices, competitiveness, and complementarity with official development aid (ODA) into account. In addition, it carried out two research case studies focusing on the complementarity of ODA with multinational corporations´ activities.
Management: Reinhard StockmannClient: European Union
Accompanying
Start of project: April 2015
End of project: October 2016
Evaluation of the Initiative “Schulen – Partner der Zukunft” (PASCH)
The initiative Schools Partners of the future (PASCH), founded in 2008, is part of the central initiatives of the German foreign cultural and educational [Read More]The initiative Schools Partners of the future (PASCH), founded in 2008, is part of the central initiatives of the German foreign cultural and educational policy within the current legislature period. PASCH is a worldwide network currently including around 1800 partner schools, which are linked to Germany and especially teach German. The initiative concentrates diversified measures, which are supposed to contribute to establish lively and long-term relations to Germany, and encourage schools, teachers and students to exchange their knowledge and ideas as well as to work together. PASCH is coordinated by the German Federal Foreign Office and is implemented by the Central Office for Schools Abroad, the Goethe Institute, the Pedagogic Exchange Service, the Conference of Education Ministers and the German Academic Exchange Service.
Within the scope of the planned evaluation, PASCH will be analyzed regarding its relevance, efficiency, effectivity, impact and sustainability. It will also be analyzed to what extent the existing offers, processes and structures are suitable instruments for achieving the objectives. Recommendations for possible optimization of the measures, processes and structures and the creation of synergies will be derived on the basis of these results.
The evaluation is divided into two phases: During the first phase, document analyses as well as guided personal and telephone interviews with stakeholders in Germany and selected partner countries (Poland, Turkey and China) will be conducted. The focus of the second phase is then on quantitative online surveys with target groups in China, Turkey and Poland as well as on qualitative and quantitative data collection in another case study country.
Management: Reinhard StockmannClient: Federal Foreign Office
Start of project: December 2012
End of project: September 2016
Practice-Impact II: Development and Testing of a Concept for Documentation and Evaluation of Productive Interactions and Impacts on Practice and Society in Agricultural Research
Cooperation Partner: University of Kassel, Faculty of Organic Agricultural Sciences, Organic Farming and Cropping Systems Currently, established research evaluation procedures are largely based on peer [Read More]Cooperation Partner: University of Kassel, Faculty of Organic Agricultural Sciences, Organic Farming and Cropping Systems
Currently, established research evaluation procedures are largely based on peer review publications, citation based indicators and third-party funds. However, the focus on the "scientific impact" only partially captures to what extent research contributes to solving societal or ecological problems. An increasing number of scientists and initiatives point out the responsibility of (public funded) science to contribute appropriate solutions for problems like global warming, food safety and loss of ecosystem services and research evaluations should set incentives to do so. This project was aimed at contributing to the implementation of complementary assessment of activities and impact of research regarding the needs of practice and society.
Concepts for an evaluation beyond scientific impact do already exist for inter- and transdisciplinary research, applied agricultural research and broader/social/societal impact assessments and are partly used by funding agencies. They include criteria for productive interactions between research and society, which make an impact probable, as well as the impact on practice and society itself. Nevertheless, broader use of the existing concepts is hampered by the lack of reliable, easy to use data.
The main task of the project was the further development and testing of a documentation and evaluation concept for agricultural research. The concept was defined to be broad enough to include all productive interactions and impacts of different types of agricultural research and can be used for the evaluation of scientists, institutions, projects and programmes. This requires a structured, standardized documentation in a database-system that enables to filter, aggregate and analyse data in different ways.
In order not to increase the effort necessary for scientists to document their projects, the developed documentation system is connected to general documentation procedures in proposals and reports for research funding but also includes project independent research.
The project developed and tested the structured documentation system and an assessment framework. This included three perspectives: Scientists responsible for documentation, funding agencies and evaluation experts. Furthermore, the project engaged in the development of applicable and accepted pathways to implement such evaluation in alignment with agricultural research and other relevant stakeholders and experts.
Management: Reinhard StockmannStart of project: September 2013
End of project: April 2016
Accompanying Research and Acceptance Research on Current Issues of Grid Extension in Germany Scientific Monitoring of Planning Practice (Module III)
In order to reach a broad acceptance for grid extension within German society, information and communication actions on a large scale are necessary. First, Module [Read More]In order to reach a broad acceptance for grid extension within German society, information and communication actions on a large scale are necessary. First, Module III of the project provided an overview of existing and planned actions. Subsequently, the implementation of selected information and communication measures were analyzed to identify effects on attitude change towards grid extension, contributing factors and opportunities for optimization.
Management: Reinhard StockmannStart of project: January 2012
End of project: December 2015
University Cooperation with Universidad de Costa Rica (UCR), with Funding Provided by the DAAD Program Subject-Specific Partnerships with Universities in Developing Countries
Having completed a successful 3-year cooperation focusing on the further development of the postgraduate study programme Project and Programme Evaluation for Social Development established at [Read More]Having completed a successful 3-year cooperation focusing on the further development of the postgraduate study programme Project and Programme Evaluation for Social Development established at Universidad de Costa Rica (UCR) in 1995, the Center for Evaluation (CEval) and the Universidad de Costa Rica (UCR) identified and realised new areas of cooperation. Since 2008, the German Academic Exchange Service (DAAD) has supported this cooperation with its funding project for sending German academics to universities in developing countries. Since 2012, the DAAD has also provided funding through the programme Subject-Specific Partnerships with Universities in Developing Countries (2012-2015), financially ensuring the plans to enhance the partnership between the two universities in Germany and Costa Rica.
The goal of the DAAD programme was to improve the teaching situation and to set up sustainable teaching structures in developing countries by facilitating academic exchange. Therefore, the international exchange between students, PhD candidates and lecturers was one of the most important aspects in the context of this enhanced cooperation. The focus also laid on opening up the master programme to other Central-American areas (regionalisation), embedding the partnership into an existing, international network (internationalisation), and adapting teaching of the master programme at UCR to regional demands and ensuing educational needs (creating a distinct programme profile). The four year partnership helped establishing or expanding regional and international professional networks and developing additional capacities regarding teaching and research.
It is planned to continue with the partnership in 2017 and integrate another Latinamerican university into the cooperation.
Management: Reinhard Stockmann | Co-Worker: Sandra Bäthge, Susanne Johanna VäthClient: German Academic Exchange Service
Start of project: February 2014
End of project: December 2015
Development of an impact-orientated M&E-System in Germany for the Cooperation Platform of Latin America North (COPLAN)
Up to the present innovative environmental technologies are not sufficiently utilized in Mexico and Central America. One reason is the missing extension of modern environmental [Read More]Up to the present innovative environmental technologies are not sufficiently utilized in Mexico and Central America. One reason is the missing extension of modern environmental technologies in this region. That is the initial point of COPLAN (Cooperation Platform of Latin America North): The objective of COPLAN is the extension of (German) environmental technologies in Mexico and Central America to make a contribution to environment protection. In the last years the German environment economics has led the way worldwide and was responsible for many innovations in the area of application-oriented technologies. Three questions were the base of the survey, namely a) What is the export potential of the German industrial sector of environmental technologies? b) Correspond the German supply with the potential demand in Mexico and Central America? and c) Which export barriers exist and how they can be overcome?
Because of the intended project effects in Germany it was necessary to create another impact-orientated monitoring-system in Germany beside the local impact-orientated monitoring-system in Mexico. The central task was to create an impact-orientated Monitoring and Evaluation System for measuring the intended outcome of COPLAN in Germany. For this purpose the main task was to develop an extensive indicator raster and an investigation plan for implementing two panels measuring the changes of the two target groups, the German enterprises of environmental technologies on the one hand, and central multipliers of the German environment economics on the other hand. Following the development of the M&E system, the CEval was tasked with implementing the first wave of the panel design. This included quantitative as well as qualitative data collection methods (standardized survey with participating companies, a comparison group and multiplicators, analysis of documents and secondary data, guided expert interviews).
Management: Reinhard StockmannStart of project: November 2014
End of project: November 2015
Support for the compilation of possible methodological approaches to evaluating medium-term and long-term impacts of (policy) consulting in the specific context of the ECLAC project
In July 2014, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) started the consultancy project Structural change for sustainable and inclusive development in Latin America and [Read More]In July 2014, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) started the consultancy project Structural change for sustainable and inclusive development in Latin America and the Caribbean together with the United Nations Economic Commission for Latin America and the Caribbean (ECLAC). Currently, the project utilizes a monitoring system which allows tracking activities and consultancies carried out by ECLAC (inputs) and their utilization (outputs). Long-term impacts are not considered yet in this approach.
The Hertie-School of Governance (HSoG) was commissioned to elaborate possible methodological approaches to evaluating medium-term and long-term impacts of (policy) consulting in the specific context of the ECLAC project. Besides experience in policy consulting, this process also requires expertise in evaluation research and practice; therefore, the HSoG approached the Center for Evaluation (CEVAL) for collaboration.
The consultancy comprises three work packages:
A general review of theoretical approaches to the evaluation of policy consulting (considering qualitative methods of empirical social research);
A review of practical experiences from policy consulting, if possible in similar contexts;
An outline of a methodological approach to measuring medium-term and long-term impacts based on the results of the first two work packages and adapted to the specific context of the collaboration with ECLAC.
CEval primarily contributed to the third work package and developed a methodological approach which combines a overarching impact and program specific outcome M&E. The approach proposed comprises a mixture of methods such as document analysis, standardized (online) surveys and guided interviews with a panel of experts.
Management: Reinhard StockmannClient: Hertie School
Start of project: July 2011
End of project: July 2015
Development and Implementation of a M&E-System for the Plan Honduras Project "Prevention, Protection and rights Restitution for Girls and Boys survivors of sexual violence"
The overarching objective of the Plan Honduras project was that girls and boys are protected from sexual violence and had access to child friendly and [Read More]The overarching objective of the Plan Honduras project was that girls and boys are protected from sexual violence and had access to child friendly and competent prevention and attention mechanisms from local to national level. In order to achieve this objective, Plan Honduras developed an action programme that was to strengthen prevention, protection and attention capabilities for at-risk girls and boys and survivors of sexual violence in the family, community, municipality, departments and at the national level.
The task of the Center for Evaluation (CEval) in this framework was to assist and consult the process of developing and implementing an impact oriented gender sensitive M&E-system that could be used throughout the project cycle and beyond. Therefore the CEval adviced and trained project executives and partners in order to enable them to systematically monitor and evaluate project related activities, to identify the most relevant aspects on all impact levels, to formulate impact oriented qualitative indicators for the project contributing to gender equality, to design a data collection plan and instruments and to solve occurring challenges during the data collection and analysis process.
The knowledge transfer was facilitated through a combination of continuous consultancy, training measures and practical implementation of the consultancy and training content within an accompanied baseline study including a gender analysis as well as an accompanied interim and a final project evaluation.
Management: Reinhard StockmannClient: Plan International Germany
