Evaluation of the Funding Activities of the Foundation Environment and Development North Rhine-Westphalia
The Foundation Environment and Development North Rhine-Westphalia was founded 2001 and promotes mainly volunteer environmental and development work of civil society groups. The objective of the foundation is to support the environmental and development work in North Rhine-Westphalia (NRW) in order to establish this work in the consciousness and in the practical involvement of the population permanently and at the state of the art.
For this purpose about 1,100 projects of non-profit organisations in NRW were supported with a volume of EUR 47.5 million so far. In addition, the Foundation carries out its own projects, to also promote the implementation of Agenda 21 of the United Nations Conference on Environment and Development in Rio de Janeiro in in 1994.
Within the evaluation, the results and effects of the funding activities of the Foundation were recorded and assessed. Furthermore, the evaluation was intended to find out information on potential inhibitory and / or promoting factors in the funding activities of the Foundation with respect to the foundation goals. Based on these findings recommendations on priorities of the Foundation with regard to content, actors, target groups and the methods and formats will be derived.
CEval applied a mixed method approach to collect relevant data. Therefore, relevant documents and accessible data of the foundation and of funded projects will be analysed. This will be combined with data based on qualitative guideline-based interviews with several stakeholders and standardised online surveys of current and completed projects.
External Evaluation of Four Different Projects in the Field of Awareness Raising and a Synthesis
As international Christian development organisation, the Christian Blind Mission (CBM) aims at improving the human rights situation and the inclusion of people with disabilities. Awareness raising is one important component to achieve this target successfully. To learn how awareness raising activities are implemented, how they contribute to CBMs overall aim and how project and country specifics produce different results, CBM Germany has engaged CEval to evaluate four projects and to compile findings and recommendations in a synthesis report.
With co-funding, either by the European Union or by the German Federal Ministry for Economic Cooperation and Development, CBM Germany implements two projects addressing the needs of people with hearing impairment in the Palestine Territories (Gaza and Gaza Strip), one project to improve eye care and rehabilitation in India, and one project to strengthen disability rights and inclusion in Swaziland. Given the diversity of these projects, issues of awareness raising can be studied from various perspectives. Thus, analysing the different underlying assumptions and needs which in turn lead to various approaches and expected outcomes, will contribute to a better understanding of the overall topic of awareness raising. While the projects in Gaza and India require a mid-term evaluation as they are still on-going, the projects in Gaza Strip and Swaziland are close to their end and will be finally evaluatedDevelopment of a Monitoring and Evaluation Framework for the Plan project “Adolescent Boys: Champions of Change on Gender Equality and Girls Rights”
Plan International is a non-religious, non-political and non-governmental development organization working to bring lasting improvements in childrens lives in Africa, Asia and Latin America.
In order to strengthen the promotion of gender equality and to engage boys and men in this work, Plan International Germany and Plan International Finland together with four Plan country offices from the region of the Americas implemented the pilot project ’Adolescent Boys: Champions of Change on Gender Equality and Girls Rights’. The project built the capacity of male youth as peer educators for gender equality and was carried out in five countries: El Salvador, Guatemala, Honduras, Dominican Republic and Germany.
The Center for Evaluation (CEval) evaluated the pilot project using quantitative and qualitative methods, identified lessons learnt and designed a monitoring and evaluation framework for the future implementation and upscaling of the project.
Evaluation of CALIDENA A participative instrument to support the improvement of quality infrastructures
CEval had been mandated by Physikalisch Technische Bundesanstalt (PTB) to conduct the evaluation of CALIDENA, a participative approach to systematically and sustainably support the improvement of national quality infrastructures in developing countries. The evaluation systematized the experiences that have been made with the instrument so far and, on that basis, aimed at delivering relevant information for the enhancement of the instrument and its transfer to and application in new contexts.
CEval employed a mixed-method approach. Data had been collected using qualitative expert interviews, quantitative surveys, participant observation and group discussions. Once the analysis of the data thus gathered had been finalized, the evaluation results had been presented in an evaluation report and nurture the overall evaluation synthesis of various PTB instruments in the area of quality infrastructure development, which was also being implemented by CEval.
Synthesis of three PTB technical cooperation instrument evaluations
CEval has been mandated by Physikalisch Technische Bundesanstalt (PTB) to synthesize the results of three PTB instrument evaluations. In 2014 PTB had commissioned separate evaluations of three of its technical cooperation instruments: CALIDENA, QuISP and NMI-Metrology User Relations, of which the first two evaluations have been implemented by CEval. Their findings had been aggregated, compared and contrasted in order to extract overarching success factors and lessons learnt and generate relevant recommendations, among others, with regard to the future incorporation of the three instruments into PTB technical cooperation.
To that end, CEval qualitatively analyzed the information contained in the evaluation reports and collected complementary information by conducting a number of qualitative expert interviews. The synthesis results had been presented in an evaluation report and a workshop had been held to present them to PTB management and staff.
Accompanying evaluation of the project ‘Media in Libya – Stability through Structure"
The „Media in Libya- Stability through Structure“ project had the main goal of supporting the introduction of modern media legislation in Libya and to strengthen the countrys media institutions. In addition, the project should contribute to strengthen independent and professional journalism in the provinces of Libya. The project had a total duration of 30 months and was funded by the European Commission within the SPRING program. The project was controlled and implemented by the Deutsche Welle Academy (Department of Media Development, Team Africa) in cooperation with the Institute for War and Peace Reporting (IWPR). The activities of the project were divided into four activity clusters, of which DW Academy three realised, while IWPR edited one. CEval evaluated in this project clusters 1 to 3, which were in responsibility of the DW Academy.
The evaluation of the project was divided into a baseline, an intermediate and a final evaluation at the end of the implementation phase. CEval followed a mixed-methods approach for data mining. An in-depth analysis of already existing documents was carried out as well as own surveys (standardized online surveys of the participants of the trainings and qualitative guideline interviews with external experts and officials DW Academy).
Conception, realization and analysis of an expert panel for the assessment of qualitative factors of the renewable energy sector in Chile
The Deutsche Gesellschaft für internationale Zusammenarbeit (GIZ), on behalf of the Federal Ministry for the Environment, Nature Conservation, Building and Nuclear Safety (BMUB), supports the Chilean Energy Ministry in the implementation of measures for the diversification of the countries energy supply. One component of this program is a project to promote the development of solar energy use. At the beginning of the project, the quantitative and qualitative factors influencing the effectivity and sustainability and therefore the achievement of the projects objectives needed to be identified. Particularly for the documentation of qualitative factors it was necessary to gather empirical data through an expert panel, i.e. structured interviews with energy experts.
In the context of this assignment, the CEval GmbH rendered the following services to GIZ:
– Concept development: Identification of experts for the panel in coordination with GIZ, development of structured questionnaires for each group of stakeholders, development of a concept for future periodic panel interviews, development of a methodological concept for the long-term continuation of the expert Panel
- Realization of the first data collection round
- Analysis and reporting of the first data collection round
- Development of a concept for the identification of quantitative impacts using the qualitative data, applying Stockmanns funnel approach
Evaluation of the project “Set-up of a demand-oriented and regionally harmonized quality infrastructure in the Andean Region”
CEval was mandated by Physikalisch Technische Bundesanstalt (PTB) to evaluate the project Set-up of a demand-oriented and regionally harmonized quality infrastructure in the Andean Region.
The project involved the countries of Bolivia, Ecuador, Colombia and Peru and aimed at improving the services offered by local institutions of quality infrastructure (QI). The evaluation addressed the criteria developed by DAC (relevance, effectiveness, impact, efficiency, sustainability) and the components of the management model Capacity Works (strategy, cooperation, steering structure, processes, learning and innovation). Specific evaluation questions were:
– How successful was the approach to develop regional QI-services?
– To what extent do the partners consider the approach as appropriate?
– Which success factors and lessons learnt can be identified?
– How successful was the approach to develop sector-specific supply strategies in each country?
– Which success factors and lessons learnt can be identified?
First, the evaluation consulted project documents only to assess the possibility to conduct evaluations based solely on documents. After a first assessment of the results, these were complemented by interviews with project staff of the PTB and its partner institutions. >From this, recommendations for future projects and evaluations based solely on documents were derived.
Meta-Evaluation: Project Evaluations of international development co-operation of the Friedrich-Ebert-Foundation (FES)
The International Development Cooperation Department (IEZ) of the Friedrich-Ebert-Stiftung (FES) commissions around ten external evaluations per year in order to continuously improve its own work, to initiate learning processes as an organization based on experience gained and to fulfil its accountability to the donor.
As part of a meta-evaluation, the Center for Evaluation (CEval) was commissioned to analyze 18 evaluation reports that had been prepared between 2012 and 2014 during a phase in which IEZ’s quality management had undergone significant development. The aim of the meta-evaluation was to identify systematic strengths and weaknesses of current evaluation practice, to derive findings for improving the quality of future evaluations and thus to increase their usefulness for project managers and the responsible units.
While the quality of the evaluation reports was analyzed in a desk study, in-depth expert interviews and focus group discussions were also held within the FES to assess the usefulness of the evaluations.
Improving the monitoring and reporting system to implement the action plan Inclusion of Persons with disabilities in the German development co-operation
In February 2013, the Federal Ministry for Economic Cooperation and Development (BMZ) published an action plan for the inclusion of persons with disabilities (PWDs). In doing so, it follows its duties emanating from Article 32 of the UN Convention on the rights of PWDs. The action plan provides information on strategic goals and actions to design development cooperation of the Federal Republic of Germany towards a better inclusion of PWDs.
To asses the Progress and to steer the implementation of planned an appropriate Monitoring and reporting system was necessary. Hence, the Center for Evaluation (CEval) had been contracted to Support the sector project “Inclusion of PWDs” of the German Agency for International Cooperation (GIZ) and the department 300 of the BMZ with its methodological Expertise to develop an Analysis grid to evaluate the implementation of the action plan.
