International consultancy to conduct mapping of Early Childhood Care Development and Education (ECCDE) Centres in Eswatini
The purpose of this assignment was to support the Ministry of Education and Training in Eswatini to conduct a mapping of all early childhood care, development, and education programs (ECCDE). The mapping had been conducted on all centres that provide ECCDE services (formal and informal), including detailing critical information on location, enrollment, infrastructure, and the type of services provided. Results from the ECCDE mapping are expected to inform the development of the Education Sector Strategic Plan 2022 – 2034, including to promote the improvement of access and quality of ECCDE services across the country.
CEval is responsible for the digitalization of the questionnaire with the help of the software Kobocollect, data cleaning and statistical analysis.
Mid-Term Evaluation of the „Palestinian-German Science Bridge” (PGSB)
The program Palestinian-German Science Bridge is a pilot project financed by the German Federal Ministry of Education and Research (BMBF), which is jointly implemented by Forschungszentrum Jülich and the Palestinian Academy for Science and Technology (PALAST). It aims at establishing a joint education and research cooperation between Forschungszentrum Jülich and the Palestinian partner universities. To this end, the program combines measures focusing on student and researcher mobility, scientific exchange among faculty staff, the support of research collaborations and the reintegration of young Palestinian researchers into the Palestinian academic sector.
Forschungszentrum Jülich mandated CEval GmbH to counsel the Forschungszentrum with regard to the conceptualization of a mid-term evaluation of the program and to assist the Forschungszentrum in implementing it. The purpose of the evaluation was to assess the short- and medium-term results of the program as well as the prospects for long-term goal achievement. The objective, further, was to identify potentials for improvement in relation to the program.
Methodologically, the evaluation was based on a mix of qualitative and quantitative data collection and analysis methods. Data collection combines a document and secondary data analysis with guided interviews and semi-standardized online surveys among the project stakeholders. During data analysis, descriptive statistics and qualitative content analysis were applied. Due to the COVID-19 pandemic, the evaluation was planned as a remote evaluation.
Alumni survey of the European School of Materials (EUSMAT)
The European School of Materials (EUSMAT) at Saarland University offers various international study programs at Bachelors, Masters and PhD level. In order to stay in touch with the alumni of these programs, EUSMAT is currently expanding its alumni activities.
To verify how well the current alumni activities fit the needs and to identify expectations of EUSMAT alumni, the EUSMAT had mandated CEval GmbH with the conceptualization and implementation of an alumni survey. The objective was to generate a data base for a further enhancement of the EUSMAT alumni work.
Methodologically, a semi-standardized survey as well as a number of focus group discussions had been designed and conducted among EUSMAT alumni.
Joint departmental strategic evaluation of the AA and BMZ engagement in Iraq
Within the framework of its crisis engagement in Iraq, the Federal Government intends to further embed monitoring and evaluation as instruments of learning and quality assurance, accountability and strategic steering. In this context, a joint evaluation of the Iraq engagement of the Federal Foreign Office (AA) and the Federal Ministry for Economic Cooperation and Development (BMZ) was conducted. This was intended to serve as a pilot project and, if possible, as a model for joint evaluations of both ministries in relevant fields of action and at their interfaces. The main objective of this joint evaluation was joint institutional learning. The AA component of the evaluation was realised by a consortium under the leadership of the GFA Consulting Group, in which CEval GmbH was involved; the BMZ component by the German Evaluation Institute (DEval). Both evaluation teams worked closely together in a so called Arbeitsgemeinschaft.
The aim of the evaluation was to conduct an independent and comprehensive analysis and assessment of the engagement of the two ministries in order to gain and use insights for (joint) steering. The evaluation provided insights into the relevance, effectiveness and overarching impacts, efficiency, sustainability and connectivity as well as coordination, coherence and complementarity of the Iraq engagement of both ministries. Furthermore, the evaluation provided recommendations for optimising the overall engagement and individual measures, structures and processes in order to adjust the current portfolio if necessary. Finally, learning experiences from the AA-BMZ engagement in Iraq were identified for similar engagement in other crisis contexts. The evaluation also served as an accompanying or final performance review of the country portfolio in accordance with the Federal Budget Code (Bundeshaushaltsordnung).
Methodologically, the evaluation was based on a broad mix of qualitative and quantitative data collection and evaluation methods. Due to pandemic-related travel restrictions, the data collection was realised or managed remotely by the consortium in both Germany and Iraq.
Development of results-based framework “Resilience in Strategic Communication”
The digital transformation has also greatly changed the demands on foreign policy and its communication. Whereas it used to be primarily the responsibility of the foreign minister and the global network of foreign missions, foreign policy today has many state and non-state actors. At the same time, global competition for attention has reached a new dimension due to digitalisation as well as numerous new players and alliances. But it has also changed due to a massive shift of communication to social media. Strategic communication today is therefore not only a different way of communicating in other social spaces, but at the same time a confrontation with challenges such as disinformation, propaganda or hate crime and a changed concept of the public, of freedom of expression and its threats.
The aim of the assignment was to provide a decision-making basis for project selection based on clear and predefined criteria on how well a proposed project fits into the overall logic of resilience promotion, as well as to create a basis for evaluating German engagement in the resilience sector. This took the form of developing a results-based framework with associated impact-oriented indicators, which were documented in an accompanying report and embedded in terms of communication theory.
Pilot Project Strategic Evaluation of the Transformation Partnerships with Jordan, Morocco and Tunisia
Together with the GFA Consulting Group, CEval GmbH conducted a strategic evaluation of the transformation partnerships of the Federal Foreign Office (AA), which have been in place since 2011 as part of the German government’s foreign policy to support democratisation processes in Arab countries. The focus of the evaluation was on the areas of rule of law promotion, civil society promotion and foreign cultural and education policy, as well as on the three countries Jordan, Morocco and Tunisia.
The pilot project of the strategic evaluation was in the context of the longer-term goal of expanding the use of strategic evaluation as a steering instrument and anchoring institutional learning in the AA. In addition to examining the procedures and methods actually used to set strategic priorities, the study was therefore also intended to include possibilities for expanding strategic management and improving the link between project implementation and evaluation.
As a starting point for the evaluation, the evaluation team developed results-based framework for the overall strategy of the transformation partnerships, in which both the objectives intended by the AA and possible contextual factors were listed. In addition, results-based frameworks were developed for the priority areas (promotion of the rule of law, promotion of civil society), which were integrated into the impact structure of the overall strategy. Methodologically, both quantitative and qualitative methods were used to collect data in Germany and in the field. The data analysis combined descriptive statistics with qualitative content analysis.
International Program for Development Evaluation Training (IPDET)
The International Program for Development Evaluation Training (IPDET) is an executive training program in development evaluation. From 2018 to 2024, it was managed jointly by the Center for Continuing Education at the University of Bern (Switzerland), the Center for Evaluation at Saarland University (Germany), and the Independent Evaluation Group of the World Bank. Since 2021, IPDET has been a key implementing partner of the Global Evaluation Initiative (GEI).
IPDET aims to provide managers and practitioners in the field of evaluation with the tools required to evaluate policies, programs and projects at local, regional, national, and global levels, as well as to commission, manage, and especially use those evaluations for decision-making. Further information on IPDET can be found on the website: https://ipdet.org/.
Pilot project for the joint match day follow-up in German professional football
The Daniel Nivel Foundation aims to improve the relationship between football fans and the police. To that end, the foundation cooperates with national football associations and in 2018 has released funds for two pilot projects, one with the French Football Federation (FFF) and one with the German Football Association (DFB). On the German side, the following idea was developed within the so called future workshops: to establish a joint match day follow-up, to create a framework for dialogue between football fans and police. In this way, conflicts would be dealt with after match days and solutions for future events could be developed.
The DFB had commissioned CEval GmbH to provide scientific support for this project. The tasks of the CEval GmbH ranged from the conception of the project (follow-up process, selection of pilot locations, etc.) to the development of suitable instruments for fan surveys and suiting formats for match day follow-ups, as well as the support of the actors in the practical implementation and moderation of the follow-up process. Therefore, different procedures and formats for the match day follow-ups had been tested, in order to present an applicable concept after the pilot phase. All steps had been carried out in close cooperation with the DFB project team. There was also a regular, project related exchange with the Fan Projects Coordination Office (KOS) and the German Football League (DFL). The project was scheduled to run from January 2019 to May 2020. In the first half of 2019, a detailed project conception and elaboration took place, including workshops with all participating actors. In the second half of the year, during the first half of the 2019/2020 season, the developed concept for the match day follow-up had been implemented and tested in cooperation with selected pilot locations from professional football. Central partners had been the Fan Projects of the respective locations. This was followed by an evaluation and, if necessary, adaptation of the matchday follow-up concept and documentation of the project in the form of a final report.
Evaluation of the project “Digital Leadership” of the Führungsakademie Baden-Württemberg
Since March 2018, the Führungsakademie Baden-Württemberg (FüAK) has been offering the “Digital Leadership” programme. The programme comprises various advanced training formats to promote digital skills for managers in the administrative bodies of the federal state of Baden-Württemberg. The aim is to prepare these managers for the requirements of digital shift, to network them with one another and to equip them with the necessary skills for driving and shaping digital change in their institutions. The program’s formats include professional training (e.g. programming, design thinking), networking events, learning journeys to innovative companies and topic-specific focus groups. The evaluation focused on the different training formats of the programme. These were evaluated in regard to their effects. The focus was on the outcome level, especially on the short- and medium-term effects of the different events. In the sense of the participatory evaluation approach of CEval GmbH, the theory of change was developed together with the project team. The effects were assessed within the evaluation using a multi-method approach. On the one hand, participant questionnaires were developed and used at the end of an event and after a defined impact phase. On the other hand, interviews were conducted with different stakeholder groups in order to obtain more in-depth assessments of relevant aspects of the programme. In this way, the advantages of both data collection methods were combined and used to evaluate the training formats.
Evaluation of refugee projects supported by the Representative of the Federal Government for Migration, Refugees and Integration
In 2019, 25 refugee projects were supported by the Representative of the Federal Government for Migration, Refugees and Integration in three areas, i.e. strengthening volunteers in refugee assistance, empowerment and prevention of violence, and political and social participation. The funding aimed at contributing to an improved integration of refugees in Germany. The goal of the evaluation was to assess the goal achievement, efficacy and efficiency of the funded projects and to develop recommendations for future funding of such projects. At the same time the evaluation was to support the projects in setting up and implementing a results-based monitoring system. The evaluation was designed as an accompanying evaluation with formative and summative elements. It also involved a capacity building component. In terms of methodology, the evaluation relied on a mix of qualitative and quantitative data collection and analysis methods. Among others, a document review, guided interviews, focus group discussions as well as an online survey were conducted.