Programme Evaluation of APPEAR III
CEval GmbH is currently conducting the programme evaluation of the third phase of the Austrian Partnership Programme in Higher Education and Research for Development (APPEAR III) on behalf of the Austrian Agency for Education and Internationalisation (OeAD). APPEAR is a programme of the Austrian Development Cooperation (ADC) conceived, guided, and financed by the Austrian Development Agency (ADA) to implement ADC’s strategy for support of Higher Education and Research for Development on an academic institutional level in ADC’s priority regions. Implemented by OeAD, the programme currently is in its 15th year of implementation as part of which at least 107 projects (Component 1) and 183 scholarships (Component 2) have been funded through different funding instruments. In the programme phase that is being evaluated, participating institutions are based in the following countries (ADA’s “priority countries”): Albania, Armenia, Austria, Bhutan, Burkina Faso, Ethiopia, Georgia, Kosovo, Moldova, Mozambique, Palestinian Territories, Uganda, and further (“non-priority”) countries.
In comparison with previous programme phases, APPEAR has become increasingly complex in terms of its funding provisions. In response to this complexity, the evaluation team applies a theory-based and participatory approach in which both qualitative and quantitative evaluation methods are employed. While up to 42 interviews and focus group discussions constitute the qualitative data base of the evaluation and allow for an in-depth understanding of causal mechanisms and personal views on the programme, two online surveys targeting 179 programme beneficiaries generate quantitative data that allow for the identification of patterns and relationships that may not emerge from the qualitative data. Furthermore, on-site data collection in Armenia – as part of six case studies – provides close-up insights into programme implementation in one of APPEAR’s most recent partner countries.
The programme evaluation is being conducted along the OECD DAC evaluation criteria relevance, effectiveness, and efficiency, and aligns with ADA’s guiding evaluation principles. Its main purpose is to inform a potential fourth phase of the APPEAR programme.
Analysis of customer satisfaction with GATE-Germany marketing services
GATE-Germany is financed by the German Federal Ministry of Research, Technology and Space (BMFTR). Since its establishment in 2001, the DAAD has used these funds to address the increasing demand from German universities for expertise and practical support in their international recruitment and marketing activities. GATE-Germany offers a wide range of marketing tools and other services. The customer satisfaction analysis focuses on the services offered under the umbrella term ‘Marketing Services’ (MarS).
This analysis aims to gain insight into how universities use MarS and how satisfied they are with it over time. Based on the findings, recommendations for the further development and optimisation of MarS will be developed.
In terms of methodology, a combination of qualitative and quantitative survey and evaluation methods will be employed. This will include a comprehensive document and secondary data analysis, guided interviews and group discussions, as well as an online survey.
Employee survey in the rückenwind³ project: (Self-)leadership with FLOW – freedom for solutions, organisation and change
The Caritasverband Saar-Hochwald e.V. is a welfare organisation of the Catholic Church responsible for the Saarland districts of Merzig-Wadern and Saarlouis. As part of the ESF-funded programme “rückenwind³ – für Vielfalt, Wandel und Zukunftsfähigkeit in der Sozialwirtschaft” (tailwind³ – for diversity, change and sustainability in the social economy), the association is implementing the project “(Selbst)Führung mit FLOW – Freiraum für Lösungen, Organisation und Wandel” (self-management with FLOW – freedom for solutions, organisation and change). The aim is to further develop the management and corporate culture of the Caritasverband with the involvement of its employees.
As part of the project, CEval GmbH is conducting an employee survey to provide a validated analysis of the baseline situation with regard to leadership culture and cooperation. The study follows a participatory, theory-based approach and is based on a mix of methods. The core of the study is a partially standardised online survey of all Caritas Association employees. In addition, documents are analysed and exploratory interviews are conducted with various stakeholder groups of the rückenwind³ project.
The online questionnaire is designed in such a way that it can be used for follow-up surveys during the course of the project in order to identify changes brought about by the rückenwind³ project on the basis of time series data. The results of the baseline survey will also be incorporated into the design of the rückenwind³ project.
Accompanying evaluation of the project “Strong through play” – Fit for school with exercise and healthy eating
The project ‘Spielend stark’ – Mit Bewegung & gesunder Ernährung fit in die Schule (Strong through play – Fit for school through exercise and healthy eating) – carried out by the Saarländische Krebsgesellschaft e.V. (Saarland Cancer Society) in cooperation with the Techniker Krankenkasse health insurance fund – aims to sustainably embed health awareness in the everyday lives of children in the form of regular exercise and a balanced diet, thereby strengthening a healthy lifestyle and health resources among pre-school children in the long term. Long-term, these habits can help prevent cancer. The project is aimed at kindergartens and includes preschoolers, teachers, parents, caregivers, and other family members as key target groups.
CEval GmbH provides scientific support for the project in the form of an accompanying evaluation. The aim of the evaluation is to optimise the quality of the project design and implementation in the pilot phase and to gain insights into the achievement of objectives and effectiveness in the subsequent rollout phase. The evaluation follows a participatory and theory-based approach and is based on a mix of quantitative and qualitative methods. The basis for this is an evaluation concept tailored to the specifics of the project. The concrete operationalisation, data sources and methodological instruments are developed in parallel with the project design during the course of the project. This ensures that the evaluation remains flexible and is tailored precisely to the project implementation.
Evaluation of the funding programme “Nachbarschaftsgespräche”
The Allianz für Beteiligung (“Alliance for Participation”) is a network that is committed to strengthening citizen participation and civil society activities in the Federal State of Baden-Württemberg. Since 2018, the Alliance for Participation has been implementing the funding programme “Nachbarschaftsgespräche” (“Neighbourhood Talks”) financed by the Baden-Württemberg State Ministry. It supports civil society initiatives in implementing dialogue formats in small social spaces in Baden-Württemberg; since 2018, around 100 projects have been funded under the programme.
In order to learn from the implementation to date and to be able to further develop the programme accordingly, the Alliance for Participation commissioned CEval GmbH to carry out an interim evaluation. This was based on the evaluation of 60 funded projects from the second and third funding phases (2020-2025) and focused on the aspects of quality, effectiveness, sustainability and opportunities for further development of the programme.
The evaluation followed a theory-based approach and included both qualitative and quantitative methods of data collection and analysis: document analyses, guided individual interviews, focus group discussions and an online survey. Based on the findings, opportunities for further development of the funding programme were identified and recommendations for action were derived.
Evaluation of the project “Time for Children’s Rights”
“Zeit für Kinderrechte” (“Time for Children’s Rights”) is a project organised by the Kinderschutzbund Landesverband Schleswig-Holstein (Child Protection Association Schleswig-Holstein), with the goal of strengthening pupils in their knowledge of children’s rights and their sense of self-efficacy in order to enhance their protection. In a two-day school workshop, children in 3rd and 4th grade are informed about their rights and learn what they can do if their rights are not respected. The project is running since 2019 and is funded by the Schleswig-Holstein Ministry of Education since 2020, supported by the Sparkassen- und Giroverband Schleswig-Holstein.
CEval GmbH has been commissioned by the Child Protection Association Schleswig-Holstein to carry out an interim evaluation of the project. The aim was to examine the quality and effectiveness of the project and to identify potential development needs.
The evaluation followed a participatory, theory-based approach and was based on a mix of methods that combined qualitative and quantitative instruments for data collection and analysis. This included document and secondary data analyses, an online survey and semi-structured group-interviews of teachers and school social workers as well as interviews with parents, the Ministry of Education Schleswig-Holstein and the project staff. Additionally, a child-friendly, standardised questionnaire was used to survey all pupils taking part in the workshop during the evaluation period.
Study “Music Industry of Saarland: Saarland as a Location for National and International Music Industries—Potentials and Requirements”
The subject of the study is the music industry of Saarland, with the aim of providing a more comprehensive picture of the industry. The focus is on identifying relevant stakeholders, existing structures and formats for promotion, visibility, education and training, as well as regional, national and international networking within the Saarland music industry. In addition, economic potential and infrastructure deficiencies are being assessed.
Methodologically, the study is based on the secondary analysis of existing data and studies, qualitative interviews, focus groups, and a semi-standardised online survey of professional groups working in the music industry. The study is being conducted in close consultation with the client. The results will serve as the basis for a strategy paper and a practice-oriented guide for the sustainable development of the music industry in Saarland.
Evaluation of the national competition for German-speaking acting students
The national competition for German-speaking acting students, which has been held annually since 1990, is funded by Germany’s Federal Ministry of Education and Research (BMBF) and aims to honor outstanding artistic achievements, to intensify the exchange between the German-speaking drama academies of Germany, Austria and Switzerland and to facilitate the transition to artistic practice for acting students.
In order to review the achievement of the objectives, efficiency and cost-effectiveness of the national competition in the sense of monitoring success in accordance with the Federal Budget Code and to derive recommendations for optimizing the competition, the organizer – the Europäische Theaterakademie GmbH “Konrad Ekhof” Hamburg – has commissioned CEval GmbH to evaluate the national competitions for the years 2015 to 2025.
The evaluation is structured according to the OECD-DAC criteria of relevance, effectiveness, and efficiency. It follows a theory-based approach and is based on a mix of methods. The evaluation is primarily based on an online survey of the participating students and teachers. This quantitative database is supplemented by quantitative and qualitative secondary data and primary data from guideline-based (group) interviews, as well as participant observation of the national competition 2025 in Salzburg.
Evaluation of the KIWI-Program of CARE Deutschland e. V.
The KIWI program of CARE Deutschland e.V. has the primary goal of closing educational gaps and promoting a more diversity-sensitive and inclusive society. The program is aimed in particular at the target group of pupils with an international background who experience discrimination, as well as educational professionals. The program supports social and transcultural learning through materials, further training for teachers, workshops for pupils, digital exchanges, and systemic process consulting for schools.
The aim of the evaluation of the KIWI program was to analyze the project design and the impact of the program in the 2023-2025 funding period. In particular, the focus was on examining the impact on the target groups, especially pupils with an international history. In addition, recommendations for the further development of the project design were to be developed in order to increase the effectiveness and impact of the program and ensure its suitability for the target groups.
The evaluation was based on a qualitative case study design and a mix of quantitative and qualitative data collection and analysis methods. It included the analysis of program documents and secondary data as well as the collection of primary data through guided expert interviews, group discussions with school coaches and pupils, and a partially standardized online survey of teachers. The data was collected in accordance with a participatory evaluation approach in close consultation with the commissioning parties.
Evaluation of the program “Quartiersimpulse”
The evaluation focused on the funding program “Quartiersimpulse” of the Allianz für Beteiligung e.V. and the Ministry of Social Affairs, Health and Integration of Baden-Wuerttemberg. The programme has been part of the state strategy “Quartier 2030 – Gemeinsam.Gestalten” since 2018 and is aimed at municipalities in Baden-Wuerttemberg. It promotes projects with a focus on age- and generation-appropriate development in neighborhoods through citizen participation, which is supported by external consulting. A prerequisite for funding is collaboration with both civil society partners and the municipal political level.
The goal of the mid-term evaluation was to take stock and identify optimization potentials for the further development of the program. In this context, the evaluation aimed to assess the quality of program implementation, as well as the goals and outcomes achieved so far, and to capture the networking of the program and the resulting synergy effects in the context of the state strategy. Based on these findings, recommendations for the further development of the program were derived.
The evaluation was based on a multi-method approach that combined document analysis and a quantitative online survey with qualitative surveys conducted during on-site visits and virtual interviews.