Strengthening evaluation capacities within government and civil society in Uganda (ECD Uganda): Development of a blended-learning Master of Evaluation (2012-2015)
Evaluation as a scientific means to describe and assess interventions is gaining relevance worldwide. The needs for qualified evaluation professionals are increasing in Uganda, too. However, at the moment these needs cannot be satisfied due to lack of training- and evaluation programs in this region. It is for this reason that the BMZ together with CEval and GIZ initiated a program for developing evaluation capacities in Uganda.
The most important component of this program is the development and implementation of a master course in evaluation in the blended-learning format at Uganda Technology and Management University (UTAMU) in Kampala, Uganda. Blended-learning is an innovative concept which is gaining popularity worldwide and is now the default means of teaching at UTAMU. Blended-learning unites traditional face-to-face classes with modern forms of e-learning in a didactical coherent way. An advantage of this concept is, that it is possible not only for Ugandan students to attend classes but also for all students of English-speaking African countries.
The Master of Evaluation of Saarland University in cooperation with HTW Saarland serves as a template for the Ugandan Master, which will be introduced in fall 2015. The contents of the master program have been developed in cooperation with CEval GmbH, the teachers of the German master as well as Ugandan contact persons of Makarere University and have been adapted to the Ugandan context. The Ugandan Master of Monitoring and Evaluation will comprise 15 modules in total, seven of which are in compliance with the basic teaching program of evaluation at Saarland universities. This is to ensure that all topics relevant to acquire profound knowledge in the field of evaluation are covered. UTAMU has complemented this program with further modules regarding project management, monitoring as well as business start-up consulting. The modules are roughly comprised of web-based trainings, e-tutoring and face-to-face seminars. The teaching concepts have also been developed in cooperation with all stakeholders. The GIZ is responsible for the technical implementation of the e-learning concepts.
Two modules of the master of evaluation have been tested in an experimental phase together with UTAMU. During this experimental phase e-tutoring as well as the face-to-face seminars have been supervised in the tandem-principle by teachers of UTAMU and Saarland University/HTW. The hosting of the online study course has been undertaken by the GIZ platform GC 21. In a next step, the seven modules have been integrated into the already existing UTAMU master and the new course of study has been submitted to the responsible committees. The course of study is now aggregated and will be introduced experimentally. A decision about whether the course of study will be continued and which modifications would be necessary for this procedure, will be made in December 2016.
Strengthening of evaluation capacity in Central America (FOCEVAL)
The project “Strengthening of evaluation capacity in Central America (FOCEVAL)” of the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH aimed to strengthen the role of evaluation in the policy-making process and the control of policies in Costa Rica and selected countries in Central America. The measure was designed as a regional project, whose political sponsorship lies with the Costa Rican Ministry of National Planning and Economic Policy (MIDEPLAN).
Within the framework of the ECD-project, the task of the CEval GmbH was the implementation of a number of training courses on monitoring and evaluation (M&E) at different aspiration levels for the partner structures. The contents of the already tested and repeatedly conducted training courses reflect the different needs among the target groups. In addition, the CEval GmbH offered the regional project a continuous scientific support, which ensures a permanent connection to the current international discussion on evaluation, and maintains the link to scientific organizations and networks. It furthermore comprised a range of advisory services, particularly in the strategy and concept development, which was based on the latest scientific findings of monitoring and evaluation research.
Impact evaluation of the Hanns Seidel Foundation activities in the area of ‘civic education’ in Kenya
The Hanns Seidel Foundation (HSF) has been committed to development cooperation for more than 35 years (as of 2013) and has been working in Kenya since 1987. In 2013, the Foundation set out to conduct an impact evaluation of its activities in the area of civic education in Kenya for the first time. These activities first and foremost comprise civic education workshops.
This impact evaluation aimed at providing empirically and methodologically sound evidence of the HSFs civic education workshops in Kenya and making an estimation of the workshops expected sustainability.
To answer the evaluation questions an elaborate quasi-experimental design was chosen that was characterized by having several measurement times and by including two comparison groups. The different measurement times allow observing direct effects of the interventions as well as a (potential) change in these effects over time. The comparison group approach enabled us to control external effects on the target groups on the local and national levels. Data collection was implemented by a team of students of the University of Nairobi who have been trained for this specific task by the CEval.
Multi-Country Evaluation of Regional Knowledge and Leadership Areas: Area 4: Inclusion of all Out of School Children in Quality Learning in CEE/CIS
The purpose of the evaluation was (i) to assess the extent to which UNICEFs contributions to program interventions addressed major child rights violations in education, reduced equity gaps and led to impact results for children; and (ii) to draw from learning practices, innovations and models for future UNICEF country programming processes.
The objective is, therefore:
- to document and report on impact results in terms of changes in childrens life
- to assess and demonstrate how such results were made possible through system changes; and
- to document the contribution of UNICEF to these system changes.
In addition, the evaluation ensures that mechanisms for the assessment of impact results embrace issues of:
- contextual adequacy (regarding tailoring programming approaches to the qualitative differences between inequities);
- coordination (with other stakeholders);
- coherence (across policies and other supporting interventions); and
- protection (in terms of adequacy of the response of the respective education systems to protecting vulnerable groups).
Ultimately, the evaluation made a contribution towards improving future programming by adapting it to the specific groups of children out of school.
Sustainable Economic Development in Myanmar
Together with the German Institute for Development Evaluation (DEval), CEval supported three projects on sustainable economic development in Myanmar, aiming to create the preconditions for identifying and assessing their impacts. The support focussed on three projects in the fields of vocational training, private sector development and financial sector reforms, and they should provide the basis for the development of a coherent programme on sustainable economic development.
Accordingly, it was the task of CEval, together with DEval:- contribute to quality, relevance and completeness of the project conceptions and to the implementation of a coherent goal and impact system
- establish the preconditions for identifying and evaluating project impacts
- contribute to programme designs
- conduct an impact evaluation.
- developing an encompassing impact model,
- defining an encompassing table of indicators,
- implementing a baseline study, and
- developing an impact-oriented monitoring system.
Independent Evaluations in the Sector Rural Development: Sustainable Management and Use of Natural Resources and Capacity Development in Central America
In this cycle of independent evaluations, there were three international development programs (all executed by the GIZ) to be evaluated by CEval:
- Promoting sustainable use of resources and local economic development in Honduras (PRORENA), Honduras
- Sustainable resource management and promotion of entrepreneurial competencies (MASRENACE), Nicaragua
- Developing and strengthening competencies and capacities to manage natural resources in Central America (Alianza de Aprendizaje)
The evaluation aimed at studying all emerged (positive, as well as negative, intended as well as non-intended) changes in order to define the project results. Based on the logic of the funnel approach, first all sector relevant changes were detected and then the causal link to the program interventions were analyzed (–>result attribution).
To attain the relevant information and to construct the counterfactual situation, CEval used a mixed-methods approach. Concretely, CEval performed the following services:
- Developing an overarching results model of the three interventions based on the project documents for assessing the project results,
- Conducting qualitative interviews with key actors and focus group interviews with beneficiaries,
- Administering a quantitative survey on the level of beneficiaries,
- Bio-technical analyses of soil samples and biodiversity factors (e.g. investigating the basal area and the number of tree species).
As a final step, the cross-cutting analyses of all three programs were merged in a synthesis report.
Assessing the Impacts of Multinational Corporations on Global Development and Value Creation (Global Value)
The importance of multinational corporations for furthering global economic and social development is well established. Corporations invest in foreign countries, provide jobs, and thus generate formalized labor relations and income. Furthermore, they pay taxes that contribute to the provision of public services. At the same time, multinational corporations particularly have received criticism by being accused of supporting exploitative employment conditions and human rights abuses, causing environmental deterioration, and failing to mainstream responsible conduct in developing countries.
The GLOBAL VALUE project developed an innovative framework for assessing impacts of multinational corporations on issues related to sustainable development, working conditions, human rights, transparency, and anti-corruption. It shed light on institutional arrangement; analysed systems of governance for responsible business practices; explored responsible competitiveness; assessed the complementarity of public and private sector activities; and derived recommendations for decision makers in business, policy and non-governmental organisations.
The project was carried out by universities and civil society organisations from Europe and ICPC countries. A toolkit for impact evaluation was developed and tested in close collaboration with leading multinational corporations (BATA (garment, Bangladesh), OLAM (food, Tanzania) and NOKIA (ICT, India)).
The main responsibility of CEval was the development of an indicator-set for the framework which is based on Millennium Development Goals, human rights, gender & diversity, and anti-corruption & transparency and takes corporate management approaches (supply chain management, life cycle analysis and base-of-pyramid) as well as systems of governance for responsible business practices, competitiveness, and complementarity with official development aid (ODA) into account. In addition, it carried out two research case studies focusing on the complementarity of ODA with multinational corporations´ activities.
Evaluation of the PTB blended-learning course “Quality Infrastructure for Sustainable Development” (QuISP)
The Center for Evaluation (CEval) had been mandated to conduct an evaluation of the blended-learning course Quality Infrastructure for Sustainable Development (QuISP) which was developed and implemented by the Physikalisch-Technische Bundesanstalt (PTB). The course aims at improving the Quality Infrastructure (QI) in developing countries by building up individual and institutional capacities particularly with regard to standardization and related disciplines.
The objective of the evaluation was to provide relevant information for the improvement of the QuISP training course with regard to its future implementation and potential scaling up.
Development and Implementation of a M&E-System for the Plan Honduras Project "Prevention, Protection and rights Restitution for Girls and Boys survivors of sexual violence"
The overarching objective of the Plan Honduras project was that girls and boys are protected from sexual violence and had access to child friendly and competent prevention and attention mechanisms from local to national level. In order to achieve this objective, Plan Honduras developed an action programme that was to strengthen prevention, protection and attention capabilities for at-risk girls and boys and survivors of sexual violence in the family, community, municipality, departments and at the national level.
The task of the Center for Evaluation (CEval) in this framework was to assist and consult the process of developing and implementing an impact oriented gender sensitive M&E-system that could be used throughout the project cycle and beyond. Therefore the CEval adviced and trained project executives and partners in order to enable them to systematically monitor and evaluate project related activities, to identify the most relevant aspects on all impact levels, to formulate impact oriented qualitative indicators for the project contributing to gender equality, to design a data collection plan and instruments and to solve occurring challenges during the data collection and analysis process.
The knowledge transfer was facilitated through a combination of continuous consultancy, training measures and practical implementation of the consultancy and training content within an accompanied baseline study including a gender analysis as well as an accompanied interim and a final project evaluation.
Impact Evaluation of WIENCO Ghana Ltd. Out-grower Schemes
The overall objective of the impact evaluation was to measure and evaluate the investment of the Agricultural Trade and Investment Fund (AATIF) in two of WIENCO’s out-grower schemes in Ghana. In particular, two out-grower arrangements, the Cocoa Abrabopa Association for cocoa farmers and the Masara N’Arziki for maize farmers, were under investigation. The main interest lied in identifying the causal link between participating in WIENCO’s out-grower arrangements and the following outcomes and their development over time:
- Productivity Levels
- Employment opportunities
- Farm and overall household income
- Living and working conditions
- Risk exposure regarding total household income; and
- Farm-households’ risk coping strategies and mechanisms.
This shall provide AATIF with evidence on the effectiveness of funding out-grower schemes. In addition a simplified tool allowing WIENCO to collect data on the key outcomes shall be designed.
The 5-year evaluation will be conducted by a consortium of the Center for Evaluation (CEval) and the German Institute for Global and Area Studies (GIGA), together with Ghanaian partners. A mixed-method approach had been embedded in a profound contextual analysis. The evaluation combined a quantitative analysis of household and village surveys with qualitative insight into WIENCO’s impacts derived from focus group discussions (FGDs) and expert interviews. The reports provided the counterpart with robust evaluation results while also presenting practical recommendations to improve WIENCO’s activities.