Impact Analysis
Start of project: November 2023
End of project: November 2024
Tracer study of alumni of the Max Weber Foundation’s institutes abroad 2005-2022
The Max Weber Foundation (MWS) is one of the leading organisations supporting German research in humanities and social sciences abroad. Therefore, the foundation maintains 11 [Read More]The Max Weber Foundation (MWS) is one of the leading organisations supporting German research in humanities and social sciences abroad. Therefore, the foundation maintains 11 institutes located around the world, which have a bridging function between the host countries and Germany. The institutes abroad play a particularly important role in the professional development and support of young academics in an international and interdisciplinary environment.
CEval GmbH has been contracted by the MWS to carry out a tracer study. The study will analyse the professional careers of former employees who worked at one of the MWS's institutes abroad in a qualification position or a long-term scholarship and who left between 2005 and 2022. The aim of the study is to evaluate the success and thus the competitiveness of the promotion of young academics at the institutes.
The tracer study follows a mixed-methods approach combining qualitative and quantitative instruments for data collection and analysis. An online survey, semi-structured interviews and a secondary data analysis will be applied.
Management: Sandra Bäthge | Co-Worker: Vera Hennefeld, Sabrina GöbelClient: Max Weber Foundation
Accompanying
Start of project: October 2023
End of project: June 2025
Data collection for the certificate “Fan project in accordance with the National Plan for Safety in Sports” (German: „Fanprojekt nach dem Nationalen Konzept Sport und Sicherheit“)
In Germany, there is a dense network of social work institutions in the field of adolescent soccer fans (“fan projects”) that work in accordance with [Read More]In Germany, there is a dense network of social work institutions in the field of adolescent soccer fans (“fan projects”) that work in accordance with the National Plan for Safety in Sports (German: Nationales Konzept Sport und Sicherheit). A certification of those fan projects has been launched in 2010. Offering fan projects an instrument of quality control, the certification is intended to pursue multiple goals: Establishing a common standard for fan projects, increasing confidence in the work of fan projects, and improving the working context of fan projects. The certificate is awarded on the basis of a recommendation by the Quality Management Group of the Centre for Fan Project Coordination of the German Sports Youth (German: Koordinationsstelle Fanprojekte bei der Deutschen Sportjugend). The Quality Management Group includes representatives from soccer, the highest state and family authorities, science, and fan projects themselves. After the certification was revised and tested by CEval GmbH in two previous projects and subsequently rolled out nationwide by collecting the information required for the certification in all fan projects, CEval GmbH has now been commissioned with the data collection for the third phase of the certification process for the current 71 fan projects.
Management: Vera Hennefeld | Co-Worker: Katharina Hary, Sandra Bäthge, Sabrina GöbelStrategic Evaluation
Start of project: September 2023
End of project: May 2024
Evaluation of the admission procedures to scholarship programs of the Hans Böckler Foundation (HBS)
The Hans Böckler Foundation (HBS) awards scholarships to secondary school leavers, artists and students as well as to doctoral candidates in all academic fields. To [Read More]The Hans Böckler Foundation (HBS) awards scholarships to secondary school leavers, artists and students as well as to doctoral candidates in all academic fields. To obtain a scholarship, applicants go through a multi-stage and meanwhile digitalized admission procedure in which their appropriateness is assessed on the basis of a series of criteria. CEval GmbH was commissioned by the HBS to evaluate the application procedures. In particular, the CEval GmbH examines the appropriateness, purposefulness and efficiency of the procedures as well as the fairness and comprehensibility of the test criteria.
Therefore, the CEval team orientates itself on the OECD-DAC criteria (relevance, coherence, efficiency, effectiveness, impact, sustainability), which are used to systematically develop an analytical matrix. The evaluation follows a mixed-methods approach combining qualitative and quantitative instruments for data collection and analysis. Semi-structured guided interviews, qualitative short interviews, an online survey and a secondary data analysis will be applied.
Management: Stefan Silvestrini | Co-Worker: Sabrina GöbelClient: Hans Böckler Foundation
Ex Post Evaluation, Impact Analysis
Start of project: September 2023
End of project: June 2024
Central Project Evaluation: Agribusiness Facilities for Africa
The project Agribusiness Facilities for Africa (ABF) serves as a pan-African knowledge and advisory hub on agribusiness trainings and has worked with about 30 countries [Read More]The project Agribusiness Facilities for Africa (ABF) serves as a pan-African knowledge and advisory hub on agribusiness trainings and has worked with about 30 countries across the continent. With the objective of improving the effectiveness of private and public agribusiness support measures and supporting organisations for micro, small and medium-sized enterprises in Africa, ABF develops training concepts for farmers and processors to meet their partners’ specific needs. The Farmer Business School (FBS) training teaches farmers and producers principles of agriculture as a business and promises to strengthen the management skills of MSMEs for better and diversified incomes and nutrition. Other training modules approaches offered by ABF include, Processor Business School (PBS), Cooperative Business School (CBS) and the more recently developed Aquaculture Business School (ABS ).
The evaluation was conducted according to the specifications and guidelines for central project evaluations of GIZ, which include a standardized analysis grid (evaluation matrix) based on the OECD-DAC criteria (relevance, coherence, efficiency, effectiveness, impact, sustainability). The evaluation team applies qualitative methods for data collection and analysis. Semi-structured interviews and focus groups discussions (FGDs) are conducted for primary data collection. During a 3 week field study in Uganda, Zambia and Malawi, the evaluation team will visit partner projects of ABF to assess the implementation and the impact of FBS, CBS and ABS teaching approaches on the ground. Qualitative content analysis will be applied based on written transcripts of the interviews, FGDs and the vast body of project documents. Causal relations between project activities and potential impacts, are assessed following a contribution analysis approach.
Management: Stefan Silvestrini | Co-Worker: Janis Wicke, Nicolle Comafay-HeinrichEx Post Evaluation, Program Evaluation
Start of project: August 2023
End of project: April 2024
Evaluation of the Diakonie Katastrophenhilfe Programme Indonesia 2018-2023
The DKH Indonesia programme was launched in the immediate aftermath of three large scale natural disasters occurring in Lombok (earthquake), Central Sulawesi (earthquake and tsunami) [Read More]The DKH Indonesia programme was launched in the immediate aftermath of three large scale natural disasters occurring in Lombok (earthquake), Central Sulawesi (earthquake and tsunami) and the Sunda Strait (tsunami) in 2018. The programme was composed of eight individual projects implemented by two Indonesian partner organisations in five provinces and featured immediate emergency response measures, recovery assistance and Disaster Risk Reduction activities with a focus on inclusion of people with disability and prioritisation of most vulnerable groups. CEval was commissioned to provide an independent assessment of the programme based on the OECD-DAC criteria.
The evaluation was based on extensive stakeholder participation as well as on a mixed-methods and theory-based evaluation approach. The evaluation team developed a comprehensive Theory of Change (ToC) at programme level and a detailed evaluation matrix. A desk-based systematic review of project documents was complemented by a three-week on-site study in Indonesia. The systematic review was based on a standardised assessment grid developed from the evaluation matrix and applied to aggregate monitoring data and relevant information from project documents. During the on-site study, the evaluation team conducted 29 interviews and 29 focus group discussions with programme and project staff, local authorities, and beneficiaries. Primary data was analysed applying qualitative content analysis. Findings were presented in a comprehensive evaluation report and during several workshops.
Management: Susanne Johanna Väth | Co-Worker: Janis Wicke, Nicolle Comafay-HeinrichAccompanying, Ex Post Evaluation, Program Evaluation
Start of project: July 2023
End of project: September 2024
Evaluation of the EU Home Affairs Funds: Mid-term evaluations of the Internal Security Fund (ISF) and the Instrument for Financial Support for Border Management and Visa Policy (BMVI)
Under the responsibility of the Directorate-General for Migration and Home Affairs (DG HOME), the European Commission (EU COM) provides the Internal Security Fund (ISF; managed [Read More]Under the responsibility of the Directorate-General for Migration and Home Affairs (DG HOME), the European Commission (EU COM) provides the Internal Security Fund (ISF; managed by the Federal Criminal Police Office (BKA)), the Border Management and Visa Instrument (BMVI, managed by the Federal Police) and the Asylum, Migration and Integration Fund (AMIF, managed by the Federal Office for Migration and Refugees (BAMF)). Together, these form the so-called "Home Affairs Funds".
The member states are obliged to prepare mid-term and final evaluations for the EU Commission on the implementation of the measures financed by the individual funds and the degree to which the objectives of the national programmes supported by the Home Affairs Funds have been achieved. The main users of the evaluations are the EU (DG Home), the fund-managing authorities BKA, BPOL and BAMF and their monitoring committees.
The aim of the evaluations is to assess the results and outcomes of the programmes and measures co-financed by the Home Affairs Fund based on the criteria of effectiveness, efficiency, relevance, coherence and EU added value. The coherence between the individual funds is of particular interest. In addition to assessing success, the evaluations should also develop recommendations for improving the quality of the design and implementation of the national programmes and analyse the coherence and use of synergy potential between the individual funds.
After the Gesellschaft für Internationale Zusammenarbeit (GIZ) carried out the mid-term evaluation for the ISF on behalf of the BKA in 2017, it has now been commissioned again to carry out the final evaluation of the ISF for the 2014-2020 funding period and the mid-term evaluation of the ISF, the BMVI and the AMIF for the 2021-2027 funding period.
In the course of this, GIZ commissioned CEval GmbH to carry out the
> Final evaluation of the ISF, security sub-sector and border sub-sector, 2014-2021 funding period,
> Mid-term evaluation of the ISF, current funding period 2021-2027 and
> Mid-term evaluation of the BMVI, current funding period 2021-2027
to be carried out.
Various survey procedures are used to triangulate sources and methods:
> Research of all relevant programme and project documents as well as monitoring data,
> Semi-standardised online survey with project participants and their implementation partners as well as other potentially eligible applicants and
> Guided qualitative interviews with programme and project managers.
Management: Stefan Silvestrini | Co-Worker: Vera Hennefeld, Sandra Bäthge, Laszlo SzentmarjayAccompanying, Ex Post Evaluation
Start of project: May 2023
End of project: November 2024
Perspective study on the DAAD program “HAW.International” (2023/2024)
The program 'Securing Talents - Shaping the Future' of the Cusanuswerk - Bischöfliche Studienförderung aims to promote the next generation of female leaders. To this [Read More]The program 'Securing Talents - Shaping the Future' of the Cusanuswerk - Bischöfliche Studienförderung aims to promote the next generation of female leaders. To this end, the program relies on mentoring elements, i.e. tutorial support for female scholarship holders of German scholarship programs for the gifted by academics with professional experience, training and support measures geared to specific professions, and the establishment of a peer network. Since the start of the career development program in 2007, CEval has conducted external evaluations to ensure the quality of the program. An accompanying evaluation will also take place in the program run in 2023/2024. The central task of this evaluation is to provide the Cusanuswerk staff responsible for implementation with timely information relevant to management. In accordance with the program's objectives, questions regarding satisfaction with the events offered, the quality of the tutorial relationships, and the professional and personal benefits of the relationship for both sides are of particular interest. Methodologically, the evaluation is based on standardized baseline and final online surveys of mentees and mentors.
Management: Vera HennefeldAccompanying, Impact Analysis
Start of project: April 2023
End of project: September 2023
Perspective study on the DAAD program “HAW.International”
The German Academic Exchange Service (DAAD) is the world's largest funding organization for the international exchange of students and scholars. In addition to awarding scholarships, [Read More]The German Academic Exchange Service (DAAD) is the world's largest funding organization for the international exchange of students and scholars. In addition to awarding scholarships, the DAAD promotes the internationalization of German universities, strengthens German studies and the German language abroad, supports developing countries in establishing efficient universities, and advises decision-makers in cultural, educational, and development policy.
The DAAD funding program "HAW.International" has been offered since 2019 and is aimed exclusively at universities of applied sciences (HAW). Within the framework of the program, students and teachers at HAW are supported with individual scholarships as well as structure-building and model and cooperation projects. In the long term, the program is intended to bring about a comprehensive and sustainable internationalization of HAW and promote its strategic positioning as an innovative university model for practice-oriented academic education worldwide. The program is intended to promote the internationalization of all HAWs both broadly and structurally. This also means that the internationalization process should be strategically anchored and sustainably implemented at all levels of higher education and across all groups of people (students, teachers, researchers, and also administrative staff and university management). To this end, "HAW.International" offers four thematic support modules that can be used by universities and students individually or complementarily.
In preparation for a further funding phase starting in 2025, the perspective study was intended to summarize the results of the funding to date and to examine the need for further support at HAW to strengthen their internationality and international visibility. The focus of the study was on identifying success factors, hurdles, and future needs. The results provide the DAAD with a basis for planning and designing the second generation of HAW.International program.
From a methodological point of view, the first step was the analysis of monitoring data. The insights gained in this way had been supplemented in the further course of the study by qualitative data from individual and group interviews with program stakeholders.
Management: Vera HennefeldClient: German Academic Exchange Service
Accompanying, Capacity Development, Development M&E-System
Start of project: April 2023
End of project: April 2027
Framework contract for the further development and implementation of systematic results-based approaches through monitoring and evaluation for DAAD programs (2023)
The DAAD understands itself as a modern science organization that strives to continuously improve its effectiveness through continuous learning and the willingness to change its [Read More]The DAAD understands itself as a modern science organization that strives to continuously improve its effectiveness through continuous learning and the willingness to change its work. This approach is formulated in the DAAD's Integrated Monitoring and Evaluation Concept, which was adopted in 2019. The concept identifies institutional learning, quality assurance, results-based orientation, transparency in processes and legitimacy as important dimensions to which the various monitoring and evaluation activities are geared. Doing so, from an overarching perspective, the strategic orientation of the DAAD and its programs is also to be supported on an evidence-based basis.
The framework agreement focuses on impact-oriented program planning, impact-oriented monitoring and target achievement and thus on further development and learning processes as well as on reporting obligations within the framework of performance monitoring. However, it also focuses on the effective and efficient use of the data generated by the Results-oriented Monitoring (RoM) in the context of monitoring-based analyses and evaluations and can thus make a significant contribution to strengthening and increasing the benefits of the RoM.
For this purpose, services are provided in the following areas:
- Development of results frameworks and indicator catalogues
- Development of program-specific monitoring instruments and revision of the program documents according to the requirements of the monitoring concept
- Data processing and analysis
- Monitoring based analyses and evaluations
- Internal and external workshops for the implementation of RoM
- Support in the context of monitoring visits to project partners
- Consultations on further approaches and methods in the field of monitoring and evaluation
- Further consulting and training services
Management: Vera Hennefeld | Co-Worker: Sandra Bäthge, Laszlo SzentmarjayClient: German Academic Exchange Service
Accompanying
Start of project: March 2023
End of project: September 2023
Evaluation of the publication of the newsletter of the City of Saarbrücken
The state capital of Saarbrücken is required by law to publish public notices. In 2021, the city council of the state capital Saarbrücken decided to [Read More]The state capital of Saarbrücken is required by law to publish public notices. In 2021, the city council of the state capital Saarbrücken decided to introduce the newsletter of the state capital Saarbrücken. The newsletter contains information on public announcements and tenders as well as other information from the administration. The newsletter is distributed free of charge to all accessible households (approx. 90,000) in the city of Saarbrücken.
The publication of the newsletter had been evaluated two years after its introduction. Among other things, it had been examined how well known the newsletter is among the citizens, whether the newsletter reaches the citizens' households and by which means (print or online edition) this is done. For this purpose, about 800 citizens living in the city of Saarbrücken had been interviewed in short personal surveys in pedestrian zones and other busy places in various parts of Saarbrücken.
Management: Vera Hennefeld | Co-Worker: Sabrina GöbelClient: State capital of Saarbrücken
