Accompanying and consulting of the impact assessment/baseline update of HEKS Bangladesh country program
HEKS/EPER’s Bangladesh 2013-1016 country programme aimed to promote the social inclusion of marginalized and disenfranchised minorities in north-western Bangladesh. The various interventions primarily targeted urban and rural Dalit and lowland Adibashi communities, who suffer from complex forms of discrimination. The exclusion of both minorities is a key cause of conflict between minorities and mainstream society. HEKS wanted to promote the social inclusion of the two groups with a human rights approach and a value chain approach.
HEKS wanted to measure the impact of the last project phase and at the same time collect valid baseline data for the next program phase. The CEval supported HEKS in this. The impact of the program was to be recorded with the help of household surveys and the qualitative method of contribution analysis. CEval mainly supported HEKS in the preparation of the quantitative data collection and the quantitative data analysis. CEval prepared the sampling strategy, helped with the development of the survey instruments and with the statistical analysis of the data. In the latter, it was particularly important to combine the previously collected data with the newly collected data in such a way that valid conclusions could be drawn from the statistical analysis.
Monitoring & Evaluation of slum rehabilitation project in Brazil
Jointly with a local partner, Knorr-Bremse Global Care e.V. (KBGC) supports a training and rehabilitation project in a community in Grajaú, a slum district in Sao Paolo, Brasil. The project aims at improving the living conditions of the community by qualifying young people for self-sustained income-generation and rehabilitating the area by building wooden houses. Main activities include training of youth in civil construction and set up of sustainable housing by residents of the slum and the trained youth.
KBGC commissioned CEval to establish a Monitoring & Evaluation framework to gather feedback on the training activities and trace quality of life outcomes over time to assess whether the programs objectives are being achieved. In order to identify the outcomes of the training on the qualification of the participants and their application of knowledge gained, a time-series design was applied, implying several phases of data collection with the help of standardized survey questionnaires. Outcomes of students were traced every six months over a period of three years, with baseline, monitoring and end line reports were drafted after each data collection to support organizational learning and improvement on project design.
Evaluation and Consultancy Service: The QUIZ-Compass under Critical Review
In 2006, the department of International Development Coorperation of Friedrich-Ebert-Foundation (FES) developed a document to improve Quality Management in international development cooperation, called the QUIZ-Compass. After several pilots and revisions the document became mandatory for all country projects in 2010. About the same time the Federal German Ministry for Economic Cooperation and Development (BMZ) began to work on an improved reporting and porposal System. Thus, in 2012 BMZ introduced new formats which became mandatory for all political foundations.
Given thes developments requirements by the QUIZ-Compass and by the BMZ regulations turned out to be partly overlapping and contradictory. Therefore FES decided to critically Review the QUIZ-Compass. The aim of the evaluation and consultancy Services was first to check whether it is still line with state of the art insights by the impact debate in development cooperation. Secondly, the practicability of the tool was tested. Finally, recommendations to enhance the QUIZ-Compass were derived.
Compliation of methodological approaches and experiences for impact Evaluations of Training and capacity building measures in the public sector
In cooperation with the Costa Rican institute CICAP, CEval counselled the GIZ program CADESAN in the Andean region. The regional program supports the development of capacity building measures on decentralization and modernization in the public sector.
The consultancy aimed at providing GIZ and its partners with an analysis and compilation of possible methodological approaches to evaluate the impacts of capacity building and training measures. To that end, various approaches and evaluation designs, their applicability in the respective realm as well as international experiences and practices were analyzed. The results were discussed with GIZ and its partners during a training designed for that purpose.
Development of an impact-orientated M&E-System in Germany for the Cooperation Platform of Latin America North (COPLAN)
Up to the present innovative environmental technologies are not sufficiently utilized in Mexico and Central America. One reason is the missing extension of modern environmental technologies in this region. That is the initial point of COPLAN (Cooperation Platform of Latin America North): The objective of COPLAN is the extension of (German) environmental technologies in Mexico and Central America to make a contribution to environment protection. In the last years the German environment economics has led the way worldwide and was responsible for many innovations in the area of application-oriented technologies. Three questions were the base of the survey, namely a) What is the export potential of the German industrial sector of environmental technologies? b) Correspond the German supply with the potential demand in Mexico and Central America? and c) Which export barriers exist and how they can be overcome?
Because of the intended project effects in Germany it was necessary to create another impact-orientated monitoring-system in Germany beside the local impact-orientated monitoring-system in Mexico. The central task was to create an impact-orientated Monitoring and Evaluation System for measuring the intended outcome of COPLAN in Germany. For this purpose the main task was to develop an extensive indicator raster and an investigation plan for implementing two panels measuring the changes of the two target groups, the German enterprises of environmental technologies on the one hand, and central multipliers of the German environment economics on the other hand. Following the development of the M&E system, the CEval was tasked with implementing the first wave of the panel design. This included quantitative as well as qualitative data collection methods (standardized survey with participating companies, a comparison group and multiplicators, analysis of documents and secondary data, guided expert interviews).
Energising Development (EnDev) stove assessment study in Ethiopia
The project Energising Development (EnDev), implemented by the GIZ, is a joint initiative of the governments of Australia, Germany, the Netherlands, Norway and the United Kingdom. Focus of this stove assessment study is the Energy Coordination Office (ECO), also known as EnDev Ethiopia. EnDev Ethiopia supports the development of a commercial stove sector and sustainable stove market not only by improving the business and technical skills of stove producers but also by strengthening the demand side of households.
The objective of this stove assessment study was therefore
- to analyze and assess the project intervention in three regions in Ethiopia by using the EnDev internal sustainability criteria and the HERA Sustainability Assessment Framework (HERA Poverty-Oriented Basic Energy Services);
- to generate recommendations for future steering of EnDevs project implementation.
Following a mixed methods approach, the study relied on standardised and qualitative methods of data collection in addition to the document analysis of relevant project and sector papers. CEval contributed to the study by providing its methodological expertise in data collection and analysis.
Meta-evaluation of 34 evaluation reports on the Area Development Programmes of World Vision Germany
World Vision Germany (WVG) had engaged CEval to conduct a meta-evaluation of 34 evaluation reports of its Area Development Programmes. The aim of the evaluation was to analyze the reports with regard to their quality and validity. Beyond this assessment, CEval provided recommendations to enhance the design and implementation of future evaluation.
Based on the evaluation criteria relevance, effectiveness, efficiency, impact and sustainability, CEval implemented a two-tier analysis method. According to WVGs quality criteria (i.e. voice and inclusion, transparency, appropriateness of evaluation methods, methodology, triangulation, and contribution) it had been first assessed to which extent the evaluation reports comply with various sub-criteria within one criterion. In a second step, findings had been consolidated and deviated one aggregated rating per criterion.
Moreover, CEval supported WVG in the production of their impact reports by critical commenting and methodological counselling.
Evaluation of the German Egyptian Research Fund
On behalf of the Federal Ministry for Education and Research the Project Management Agency of the German Aerospace Centre (DLR) implements the programme activities of the German-Egyptian Research Fund (GERF). The programme activities that had been evaluated comprise two rounds of funding (GERF I: 2009-2011, GERF II: 2011-2013/14) supporting German-Egyptian research projects in the fields of Biotechnology, Energy Efficiency, Renewable Energies, Water Desalination, Health, and Natural Resource Management.
CEval was contracted to conduct both, an ex-post evaluation of GERF I and a mid-term evaluation of GERF II. Accordingly, the objectives and downstream effects achieved by GERF I and their sustainability had to be evaluated, and the implementation process of the ongoing GERF II program had to be examined with regard to its design and efficiency, and recommendations for the continuation of GERF II and the design of future funding cycles had to be developed on the basis of the findings.
External Evaluation of Four Different Projects in the Field of Awareness Raising and a Synthesis
As international Christian development organisation, the Christian Blind Mission (CBM) aims at improving the human rights situation and the inclusion of people with disabilities. Awareness raising is one important component to achieve this target successfully. To learn how awareness raising activities are implemented, how they contribute to CBMs overall aim and how project and country specifics produce different results, CBM Germany has engaged CEval to evaluate four projects and to compile findings and recommendations in a synthesis report.
With co-funding, either by the European Union or by the German Federal Ministry for Economic Cooperation and Development, CBM Germany implements two projects addressing the needs of people with hearing impairment in the Palestine Territories (Gaza and Gaza Strip), one project to improve eye care and rehabilitation in India, and one project to strengthen disability rights and inclusion in Swaziland. Given the diversity of these projects, issues of awareness raising can be studied from various perspectives. Thus, analysing the different underlying assumptions and needs which in turn lead to various approaches and expected outcomes, will contribute to a better understanding of the overall topic of awareness raising. While the projects in Gaza and India require a mid-term evaluation as they are still on-going, the projects in Gaza Strip and Swaziland are close to their end and will be finally evaluatedDevelopment of a Monitoring and Evaluation Framework for the Plan project “Adolescent Boys: Champions of Change on Gender Equality and Girls Rights”
Plan International is a non-religious, non-political and non-governmental development organization working to bring lasting improvements in childrens lives in Africa, Asia and Latin America.
In order to strengthen the promotion of gender equality and to engage boys and men in this work, Plan International Germany and Plan International Finland together with four Plan country offices from the region of the Americas implemented the pilot project ’Adolescent Boys: Champions of Change on Gender Equality and Girls Rights’. The project built the capacity of male youth as peer educators for gender equality and was carried out in five countries: El Salvador, Guatemala, Honduras, Dominican Republic and Germany.
The Center for Evaluation (CEval) evaluated the pilot project using quantitative and qualitative methods, identified lessons learnt and designed a monitoring and evaluation framework for the future implementation and upscaling of the project.
