Projekt-Kategorien: Development Cooperation

Synthesis of three PTB technical cooperation instrument evaluations

CEval has been mandated by Physikalisch Technische Bundesanstalt (PTB) to synthesize the results of three PTB instrument evaluations. In 2014 PTB had commissioned separate evaluations of three of its technical cooperation instruments: CALIDENA, QuISP and NMI-Metrology User Relations, of which the first two evaluations have been implemented by CEval. Their findings had been aggregated, compared and contrasted in order to extract overarching success factors and lessons learnt and generate relevant recommendations, among others, with regard to the future incorporation of the three instruments into PTB technical cooperation.

To that end, CEval qualitatively analyzed the information contained in the evaluation reports and collected complementary information by conducting a number of qualitative expert interviews. The synthesis results had been presented in an evaluation report and a workshop had been held to present them to PTB management and staff.

Accompanying evaluation of the project ‘Media in Libya – Stability through Structure"

The „Media in Libya- Stability through Structure“ project had the main goal of supporting the introduction of modern media legislation in Libya and to strengthen the country’s media institutions. In addition, the project should contribute to strengthen independent and professional journalism in the provinces of Libya. The project had a total duration of 30 months and was funded by the European Commission within the SPRING program. The project was controlled and implemented by the Deutsche Welle Academy (Department of Media Development, Team Africa) in cooperation with the Institute for War and Peace Reporting (IWPR). The activities of the project were divided into four activity clusters, of which DW Academy three realised, while IWPR edited one. CEval evaluated in this project clusters 1 to 3, which were in responsibility of the DW Academy.
The evaluation of the project was divided into a baseline, an intermediate and a final evaluation at the end of the implementation phase. CEval followed a mixed-methods approach for data mining. An in-depth analysis of already existing documents was carried out as well as own surveys (standardized online surveys of the participants of the trainings and qualitative guideline interviews with external experts and officials DW Academy).

Conception, realization and analysis of an expert panel for the assessment of qualitative factors of the renewable energy sector in Chile

The Deutsche Gesellschaft für internationale Zusammenarbeit (GIZ), on behalf of the Federal Ministry for the Environment, Nature Conservation, Building and Nuclear Safety (BMUB), supports the Chilean Energy Ministry in the implementation of measures for the diversification of the countries energy supply. One component of this program is a project to promote the development of solar energy use. At the beginning of the project, the quantitative and qualitative factors influencing the effectivity and sustainability and therefore the achievement of the project‘s objectives needed to be identified. Particularly for the documentation of qualitative factors it was necessary to gather empirical data through an expert panel, i.e. structured interviews with energy experts.

In the context of this assignment, the CEval GmbH rendered the following services to GIZ:
– Concept development: Identification of experts for the panel in coordination with GIZ, development of structured questionnaires for each group of stakeholders, development of a concept for future periodic panel interviews, development of a methodological concept for the long-term continuation of the expert Panel
•- Realization of the first data collection round
•- Analysis and reporting of the first data collection round
•- Development of a concept for the identification of quantitative impacts using the qualitative data, applying Stockmann‘s funnel approach”

Impact evaluation of the Hanns Seidel Foundation activities in the area of ‘civic education’ in Kenya

The Hanns Seidel Foundation (HSF) has been committed to development cooperation for more than 35 years (as of 2013) and has been working in Kenya since 1987. In 2013, the Foundation set out to conduct an impact evaluation of its activities in the area of civic education in Kenya for the first time. These activities first and foremost comprise civic education workshops.

This impact evaluation aimed at providing empirically and methodologically sound evidence of the HSF’s civic education workshops in Kenya and making an estimation of the workshops’ expected sustainability.

To answer the evaluation questions an elaborate quasi-experimental design was chosen that was characterized by having several measurement times and by including two comparison groups. The different measurement times allow observing direct effects of the interventions as well as a (potential) change in these effects over time. The comparison group approach enabled us to control external effects on the target groups on the local and national levels. Data collection was implemented by a team of students of the University of Nairobi who have been trained for this specific task by the CEval.

Multi-Country Evaluation of Regional Knowledge and Leadership Areas: Area 4: Inclusion of all Out of School Children in Quality Learning in CEE/CIS

The purpose of the evaluation was (i) to assess the extent to which UNICEF’s contributions to program interventions addressed major child rights violations in education, reduced equity gaps and led to impact results for children; and (ii) to draw from learning practices, innovations and models for future UNICEF country programming processes.
The objective is, therefore:

• to document and report on impact results in terms of changes in children’s life
• to assess and demonstrate how such results were made possible through system changes; and
• to document the contribution of UNICEF to these system changes.

In addition, the evaluation ensures that mechanisms for the assessment of impact results embrace issues of:

• contextual adequacy (regarding tailoring programming approaches to the qualitative differences between inequities);
• coordination (with other stakeholders);
• coherence (across policies and other supporting interventions); and
• protection (in terms of adequacy of the response of the respective education systems to protecting vulnerable groups).

Ultimately, the evaluation made a contribution towards improving future programming by adapting it to the specific groups of children out of school.

Sustainable Economic Development in Myanmar

Together with the German Institute for Development Evaluation (DEval), CEval supported three projects on sustainable economic development in Myanmar, aiming to create the preconditions for identifying and assessing their impacts. The support focussed on three projects in the fields of vocational training, private sector development and financial sector reforms, and they should provide the basis for the development of a coherent programme on sustainable economic development.

Accordingly, it was the task of CEval, together with DEval:

1. contribute to quality, relevance and completeness of the project conceptions and to the implementation of a coherent goal and impact system
2. establish the preconditions for identifying and evaluating project impacts
3. contribute to programme designs
4. conduct an impact evaluation.

To that end, a multi-level analysis and consultation concept had been developed, which consists of

1. developing an encompassing impact model,
2. defining an encompassing table of indicators,
3. implementing a baseline study, and
4. developing an impact-oriented monitoring system.

Independent Evaluations in the Sector Rural Development: Sustainable Management and Use of Natural Resources and Capacity Development in Central America

In this cycle of independent evaluations, there were three international development programs (all executed by the GIZ) to be evaluated by CEval:

• Promoting sustainable use of resources and local economic development in Honduras (PRORENA), Honduras
• Sustainable resource management and promotion of entrepreneurial competencies (MASRENACE), Nicaragua
• Developing and strengthening competencies and capacities to manage natural resources in Central America (Alianza de Aprendizaje)

The evaluation aimed at studying all emerged (positive, as well as negative, intended as well as non-intended) changes in order to define the project results. Based on the logic of the “funnel approach”, first all sector relevant changes were detected and then the causal link to the program interventions were analyzed (–>result attribution).

To attain the relevant information and to construct the counterfactual situation, CEval used a mixed-methods approach. Concretely, CEval performed the following services:

• Developing an overarching results model of the three interventions based on the project documents for assessing the project results,
• Conducting qualitative interviews with key actors and focus group interviews with beneficiaries,
• Administering a quantitative survey on the level of beneficiaries,
• Bio-technical analyses of soil samples and biodiversity factors (e.g. investigating the basal area and the number of tree species).

As a final step, the cross-cutting analyses of all three programs were merged in a synthesis report.

Assessing the Impacts of Multinational Corporations on Global Development and Value Creation (Global Value)

The importance of multinational corporations for furthering global economic and social development is well established. Corporations invest in foreign countries, provide jobs, and thus generate formalized labor relations and income. Furthermore, they pay taxes that contribute to the provision of public services. At the same time, multinational corporations particularly have received criticism by being accused of supporting exploitative employment conditions and human rights abuses, causing environmental deterioration, and failing to mainstream responsible conduct in developing countries.

The GLOBAL VALUE project developed an innovative framework for assessing impacts of multinational corporations on issues related to sustainable development, working conditions, human rights, transparency, and anti-corruption. It shed light on institutional arrangement; analysed systems of governance for responsible business practices; explored responsible competitiveness; assessed the complementarity of public and private sector activities; and derived recommendations for decision makers in business, policy and non-governmental organisations.

The project was carried out by universities and civil society organisations from Europe and ICPC countries. A toolkit for impact evaluation was developed and tested in close collaboration with leading multinational corporations (BATA (garment, Bangladesh), OLAM (food, Tanzania) and NOKIA (ICT, India)).
The main responsibility of CEval was the development of an indicator-set for the framework which is based on Millennium Development Goals, human rights, gender & diversity, and anti-corruption & transparency and takes corporate management approaches (supply chain management, life cycle analysis and base-of-pyramid) as well as systems of governance for responsible business practices, competitiveness, and complementarity with official development aid (ODA) into account. In addition, it carried out two research case studies focusing on the complementarity of ODA with multinational corporations´ activities.

Evaluation of the PTB blended-learning course “Quality Infrastructure for Sustainable Development” (QuISP)

The Center for Evaluation (CEval) had been mandated to conduct an evaluation of the blended-learning course ‘Quality Infrastructure for Sustainable Development’ (QuISP) which was developed and implemented by the Physikalisch-Technische Bundesanstalt (PTB). The course aims at improving the Quality Infrastructure (QI) in developing countries by building up individual and institutional capacities particularly with regard to standardization and related disciplines.

The objective of the evaluation was to provide relevant information for the improvement of the QuISP training course with regard to its future implementation and potential scaling up.

Final Evaluation Evaluation of SCAMPIS Scaling up micro-irrigation systems in India, Madagascar and Guatemala

Since 1977, IFAD has worked to raise productivity and income of rural poor populations to overcome poverty and improve the quality of their lives. In its long history, the Fund has developed and implemented various programs and projects all over the world. Scaling up micro-irrigation systems in India, Madagascar and Guatemala (SCAMPIS) is one of the IFAD’’s projects, financed by COOPERNIC with a budget of 3 million Euro. In the period of 2008-2012, it has aimed at improving livelihoods and food security of 30.000 smallholder households in the three pilot countries of India, Madagascar and Guatemala by providing them with micro irrigation systems (MIS) and liquid organic fertilizer systems (LOF).
Before the program phased out in December 2012, an external evaluation of SCAMPIS was conducted in order to assess the achievement of program objectives and outcomes. In particular, according to the Terms of Reference, the evaluation had pursued three objectives:

• The review assessed the achievement of the programme objectives and outcomes with regard to its components and corresponding target groups.
• Lessons learnt and recommendations were provided for IFAD with regard to improvement of design and implementation of future programs. Replicability and scaling up were core concepts in this context.
• Special attention was put on the sustainability especially with regard to any future activities to be implemented by the different stakeholders along the water value chain of MIS/LOF.

The CEval evaluation team visited each project country for 12 days to conduct interviews with main stakeholders in the implementing organizations, key persons along the MIS value chain and the primary beneficiaries. In addition, a quantitative survey had been conducted in each of the three project countries to learn more about the impact MIS has on food security in different country contexts. Finally, IFAD’s perspective was taken into account by using data from interviews with most important representatives for the SCAMPIS project.