Evaluation of the EndOCSEA@Europe Project
Funded by: Safe Online
In response to the alarming rise of Online Child Sexual Exploitation and Abuse (OCSEA), the Council of Europe has commissioned the EndOCSEA@Europe project, implemented in two phases, to combat OCSEA across Council of Europe member states. Phase I (2018-2021) focused on mapping legislation, policies, and practices on OCSEA, strengthening laws, enhancing law enforcement capacities, and raising awareness in countries such as Albania, Armenia, and Ukraine. Phase II (2023-2025), focuses on further strengthening actions, particularly in Georgia, the Republic of Moldova, and Montenegro. The project promotes multi-stakeholder collaboration, legislative reforms, capacity-building, and awareness campaigns, integrating recommendations from Phase I, with synergies sought with other Council of Europe initiatives. Managed by the Children’s Rights Division, it involves key stakeholders, including government authorities, law enforcement, NGOs, and professionals working with children.
The evaluation of the EndOCSEA@Europe project aims to assess its effectiveness, impact, efficiency, and sustainability across both phases, focusing on legislative improvements, cross-border cooperation, capacity-building, and implementation challenges. Using a non-experimental, theory-based, mixed-methods approach with extensive stakeholder participation, the evaluation will examine the project’s Theory of Change, impact, and scalability at political, institutional, community, and individual levels. Data will be gathered through interviews, case studies, and surveys, analyzed using triangulation for validity, and following the OECD-DAC criteria with a focus on age- and gender-sensitive methodologies.
Study on double counting of GIZ standard indicators at country level
Funded by: German Federal Ministry for Economic Cooperation and Development (BMZ)
In German development cooperation, several projects usually work towards a common goal in a country, a region, and/or a sector, but report their results individually and often without comparing them with each other.
2022 the BMZ has introduced the “BMZ 2030” reform concept and thus decided that GIZ projects should report on 44 so-called standard indicators along defined core topics. This laid the foundation for cross-project reporting of results, which can also be used for political communication. However, due to the multiple collection of data, there is a risk that the aggregation of these indicators at the country level will overestimate the results and impacts achieved. The aim of the study was to develop a method for identifying the frequency and magnitude of these overlaps, to apply this method, and, based on the findings, to draw up recommendations on how double counting can be avoided in the future.
To this end, a quantitative “hot spot” analysis was carried out on the basis of an online survey conducted by GIZ in order to identify the potential risk of overlaps. For the subsequent validation of the actual occurrence of overlaps, empirical data was collected from interviews with portfolio managers and cluster coordinators. Based on this data, the magnitude of overlaps was estimated, and a methodologically robust solution approach was developed.
Evaluation of the “Knauf Training Initiative Phase I and II”
DEG Impulse uses the develoPPP Classic program to support measures by well-established enterprises that wish to operate in developing and emerging countries on a long-term basis and thereby shape their business involvement in a sustainable manner. One of the projects under develoPPP Classic is the “Knauf Training Initiative” (KTI) in its phase I (2018-2023) and phase II (2021-2026). Knauf is one of Europe’s leading manufacturer of gypsum-based construction materials and also specialized in production of isolating materials and cement products.
The main objective of the KTI is the establishment of training centers (TCs) to improve the technical qualification and employability of young people, students and skilled workers in the field of drywall construction and related products. Through the TCs, both practical and theoretical modules imparting comprehensive knowledge regarding the application of drywall construction techniques are provided to installers, architects/civil engineers, academic staff and unemployed people. The KTI is to provide people with the possibility of completing a training, which in turn should lead in the long-term to sustainable career prospects.
CEval GmbH was commissioned by DEG Impulse to evaluate the KTI. The evaluation assessed the relevance, coherence, effectiveness, efficiency, impact and sustainability of the KTI, identified methodological lessons learned and conclusions resulting from the findings, and provided evidence-based, actionable recommendations for DEG Impulse and Knauf. Furthermore, the evaluation integrated a gender perspective, in order to address the initiative’s effects on gender mainstreaming and women empowerment. Eventually, the evaluation investigated the additionality of the initiative, i.e., the extent to which further and/or more impacts were achieved through the financial support of DEG Impulse.
The evaluation followed a mixed-methods case-study design, collecting primary data through individual and group interviews in Germany and three target countries (i.e., Egypt, Algeria and Mexico), as well as a semi-standardized online survey among TC staff in countries not visited. Secondary data, including monitoring reports, training materials, and TC survey data, were analyzed through a desk study. The evaluation team conducted interviews with TC staff, stakeholders, and beneficiaries, ensuring appropriate representation of diverse target groups.
The data analysis involved qualitative content analysis using the software MaxQDA, as well as a descriptive statistical analysis of quantitative data from the surveys, with disaggregation by gender and country where possible.
#eSkills4Girls Initiative
The #eSkills4Girls initiative aims to close the digital gender divide by promoting the access of women and girls to opportunities to help improve their digital skills. The initiative was launched in 2017 by the BMZ under Germany’s G20 presidency and managed by the GIZ sector programme education (SP Education). Since then, the initiative established partnerships with private and public sector actors and joined EQUALS, the global multi-stakeholder partnership promoting gender equality in the digital age. To improve the digital skills of women and girls, the initiative established the #eSkills4Girls Fund, which provided small grants to 27 existing grassroots initiatives through EQUALS. Additionally, the initiative served as community manager for 9 bilateral projects financed by the BMZ under the #eSkills4Girls banner. Other activities implemented by the initiative aimed at raising the awareness of policymakers, showcasing female role models, and fostering sharing and learning among projects.
CEval GmbH was commissioned by GIZ SP Education to conduct an impact evaluation of the #eSkills4Girls initiative. The goal of the evaluation was to understand how and to which extent the different activities achieved their respective goals. Based on the evaluation questions, the initiative was assessed against the OECD-DAC criterion ‘effectiveness’. To fill in missing information and data after an initial review of existing secondary documents, online-based questionnaires for the sub-grantees of the #eSkills4Girls Fund were disseminated. Furthermore, project reports and monitoring data by bilateral project partners were reviewed. In addition, focus group discussions and interviews with representatives of the sub-grantees and bilateral projects were conducted to complete the qualitative data collection.
Project Evaluation: Promotion of Adult Education Uganda
The Institute for International Cooperation of the German Adult Education Association (DVV International) is supporting the Government of Uganda with the implementation of the Integrated Community Learning for Wealth Creation (ICOLEW) programme. As part of the programme, DVV has initiated the introduction of Community Learning Centres (CLC) together with government partners in six districts covering all major areas of Uganda. CLC provides a range of learning opportunities and services to youth and adults from surrounding communities on topics such as functional literacy/numeracy, other forms of non-formal livelihoods/vocational skills, health and sanitation, as well as business skills.
DVV International commissioned the CEval GmbH to evaluate the success of the CLC approach. The evaluation team assesses to what extent the introduction of CLC is functional and has achieved its objectives. The evaluation is based on the OECD-DAC criteria and is structured by an evaluation matrix developed be the CEval team. During a one-week on-site mission in Uganda, the evaluation team will visit CLCs to collect qualitative data by conducting interviews and focus group discussions. Data will be assessed applying qualitative content analysis. Findings will be presented and discussed in a comprehensive evaluation report.
UNICEF Thailand: Representative Study on Childcare Services
CEval supports PROMAN in conducting a national mixed methods study commissioned by UNICEF Thailand that will examine the supply and demand of under three childcare services in Thailand across the public-private continuum and assess the cost of inaction.
CEval is assigned to develop and implement a sampling strategy that provides the basis for a nationwide, representative study. The sampling strategy enables subsequent qualitative and quantitative data collection. CEval is also responsible for supporting instrument development, including household and provider surveys, hosting the survey instruments, guiding local partners’ data collection, and performing data quality management. Moreover, CEval supports its cooperation partner(s) by conducting appropriate descriptive and inferential statistical analysis.
This assignment will forecast the demand for services to make an investment case for developing a national policy strategy for under three childcare services in Thailand.
Evaluability Assessment of UNICEF Jordan Country Program 2023-2027 commissioned by the UNICEF JCO
The purpose of the evaluability assessment of the UNICEF Jordan Country Programme (JCO) was to ensure that medium- and long-term results can ultimately be demonstrated. The scope of the evaluability assessment covered JCO Country Programme Document (CPD) 2023-2027, including its six programme sections (health & nutrition, education, child protection, adolescent & youth, WASH, social policy & protection), monitoring systems, annual planning, and evaluation plans and the cross-cutting priorities (gender equality, disability inclusion, climate change, social equity, social behaviour change (SBC) and protection from sexual exploitation and abuse (PSEA)).
The objectives of the evaluability assessment were (1) to provide an independent assessment of how well JCO is prepared for a country programme or strategic framework evaluation and (2) to provide recommendations to enhance the evaluability of the CPD. The aim was to examine whether JCO has planned to effectively manage key risks, to establish mechanisms for timely implementation of evaluation recommendations, and whether it possesses the essential resources and infrastructure required to prove results achievement. The EA aimed to enhance JCO’s preparedness for the upcoming evaluation of the CPD.
The evaluability assessment employed a theory-based approach to analyse the methodological preconditions necessary for assessing effectiveness, impacts, and sustainability of UNICEF’s Country Programme. These preconditions included clear definitions of program results, baseline data availability, and timely target data. The assessment also aimed to establish meaningful hypotheses to understand the causal relationship between the JCO and observed development results. Contextual factors were considered crucial for a valid analysis, with an emphasis on identifying and monitoring risks and framework conditions. A detailed evaluability matrix guided the systematic analysis and structured data collection. A mixed-methods approach was adopted, using primary and secondary data sources as well as qualitative and quantitative methods to ensure validity and reliability. Stakeholder participation was integral, involving interviews with heads of JCO units and sections, consultations, and feedback loops to incorporate diverse perspectives.
Development of an M&E concept for the BBK’s international project work
The International Affairs Division at the Federal Office of Civil Protection and Disaster Assistance (BBK) develops projects abroad as part of Germany’s international engagement. These are based on expertise in civil protection and disaster relief and are implemented in cooperation with national and local authorities abroad.
The BBK commissioned CEval GmbH to support the development of a needs-based monitoring and evaluation (M&E) system for international project work. The objectives of the assignment were: (i) the development of an M&E concept for the BBK’s international project work, taking into account existing project documents, the (political) framework conditions and the BBK’s strategic orientation as well as staffing capacities; (ii) the further development of existing instruments and the development of additional instruments to make the impact-oriented monitoring of international projects more efficient; (iii) the presentation of the concept and the instruments at a workshop for the BBK’s project staff and project managers.
In developing the M&E system for the BBK’s International Affairs Division, CEval GmbH pursued a theory-led approach in order to create the foundations for an impact-orientated M&E system. In addition, the team pursued a participatory approach with the close involvement of the client in order to best align the M&E system with the client’s needs. The team advised and supported BBK in the introduction of SMART indicators at various levels for a methodologically sound M&E system and organized an M&E workshop for project staff and project managers for the projects in Jordan, Ukraine and Tunisia.
Evaluation of the Deval Funding Programme Rigorous Impact Evaluation
The RIE funding programme supported a matchmaking between actors from science and development cooperation, which led to funding of nine rigorous impact evaluations (RIE) to be implemented between March 2023 and September 2025. Scientific institutions in Germany were eligible to apply together with a development project funded by the Federal German Ministry for Economic Cooperation and Development (BMZ). The programme aims to remove barriers to the implementation of RIE and to support systematic anchoring of RIE in German development cooperation.
CEval GmbH was commissioned with the evaluation of the RIE funding programme by the German Institute for Development Evaluation (DEval). The evaluation is carried out according to the OECD-DAC criteria and is based on a comprehensive analysis grid. A mix of methods is applied: The RIE project proposals and interim reports are assessed by means of a systematic review, which is structured by an assessment grid. Guideline based interviews are conducted with all relevant stakeholder groups of the programme at two points in time. In addition, a standardised online survey is implemented, which addresses all scientists and representatives of the development organisations as well as local scientists in the partner countries who are directly involved in the implementation of the nine RIE.
In addition, a Delphi survey is implemented as innovative method for looking to the future. For the first Delphi round, international experts will be recruited and their perspective on the future role of RIE in German development cooperation will be examined by thematic-focussed interviews. The results will be assessed and returned to the panel participants in a second round of inquiry in the form of an online survey in order to identify, weight and prioritise converging and diverging perspectives and views of the experts.
Central Project Evaluation: Agribusiness Facilities for Africa
The project Agribusiness Facilities for Africa (ABF) serves as a pan-African knowledge and advisory hub on agribusiness trainings and has worked with about 30 countries across the continent. With the objective of improving the effectiveness of private and public agribusiness support measures and supporting organisations for micro, small and medium-sized enterprises in Africa, ABF develops training concepts for farmers and processors to meet their partners’ specific needs. The Farmer Business School (FBS) training teaches farmers and producers principles of agriculture as a business and promises to strengthen the management skills of MSMEs for better and diversified incomes and nutrition. Other training modules approaches offered by ABF include, Processor Business School (PBS), Cooperative Business School (CBS) and the more recently developed Aquaculture Business School (ABS ).
The evaluation was conducted according to the specifications and guidelines for central project evaluations of GIZ, which include a standardized analysis grid (evaluation matrix) based on the OECD-DAC criteria (relevance, coherence, efficiency, effectiveness, impact, sustainability). The evaluation team applies qualitative methods for data collection and analysis. Semi-structured interviews and focus groups discussions (FGDs) are conducted for primary data collection. During a 3 week field study in Uganda, Zambia and Malawi, the evaluation team will visit partner projects of ABF to assess the implementation and the impact of FBS, CBS and ABS teaching approaches on the ground. Qualitative content analysis will be applied based on written transcripts of the interviews, FGDs and the vast body of project documents. Causal relations between project activities and potential impacts, are assessed following a contribution analysis approach.
