Development of an Evaluation Framework for the Smart City-Projekt LAND L(I)EBEN – digital.gemeinsam.vorOrt
Through the “LAND L(i)EBEN – digital.gemeinsam.vorOrt” Smart City pilot project, the Kusel district is working together with local residents to make the region more sustainable and future-ready. Digital, innovative and locally tailored solutions aim to improve mobility, healthcare, education, social participation and quality of life in rural areas. The project will run until the end of 2027 and is funded by the German Federal Ministry for Housing, Urban Development and Building and the KfW under the “Smart Cities Pilot Projects” programme.
CEval is advising the project team on the development of a results model, an appropriate evaluation framework and suitable data collection tools for self-evaluation. The support provided by CEval is defined as the project progresses, in response to emerging needs and priorities.
Study: Requirements for the successful participation in computer science competitions from the perspective of teachers
The National Computer Science Competitions (BWINF) are an initiative designed to promote young computer science talents in Germany. The program is aimed at children and youths from elementary school to age 21 and ranges from the beginner-friendly online competition “Informatik-Biber” to the “Youth Computer Science Competition” and the advanced “National Computer Science Competition.” The latter serves as a qualifier for the International Olympiad in Informatics and offers national winners excellent opportunities to receive scholarships from the German Academic Scholarship Foundation. The National Computer Science Competitions are organized by BWINF gGmbH, with support from the German Informatics Society, the Fraunhofer ICT Group, and the Max Planck Institute for Informatics. BWINF receives significant funding from the German Federal Ministry of Education, Family Affairs, Senior Citizens, Women, and Youth. In addition, the competitions are recommended by the state ministries of education and are held under the patronage of the Federal President.
On behalf of BWINF gGmbH, CEval is conducting a study on the conditions for successful participation in the computer science competitions from the perspective of teachers. The goal is to gain insights into the visibility, accessibility, and conceptual suitability of the competitions and to derive recommendations for action from these findings.
The study employs a mixed-methods approach that combines qualitative and quantitative instruments and analytical methods. Data collection takes place in two sequential phases: During the conceptual phase, document analysis and structured interviews are used to refine the methodological framework; this is followed by a partially standardized online survey of computer science teachers, which forms the primary source of data collection.
Programme Evaluation of APPEAR III
CEval GmbH conducted the programme evaluation of the third phase of the Austrian Partnership Programme in Higher Education and Research for Development (APPEAR III) on behalf of the Austrian Agency for Education and Internationalisation (OeAD). APPEAR is a programme of International Partnerships Austria conceived, guided, and financed by the Austrian Development Agency (ADA) to implement International Partnerships Austria’s strategy for support of Higher Education and Research for Development on an academic institutional level in Austria’s priority regions for development cooperation. Implemented by OeAD, the programme was in its 15th year of implementation when the evaluation was commissioned and has funded at least 107 projects (Component 1) and 183 scholarships (Component 2) through different funding instruments since its first phase. In the evaluated programme phase, participating institutions were based in the following countries (ADA’s “priority countries”): Albania, Armenia, Austria, Bhutan, Burkina Faso, Ethiopia, Georgia, Kosovo, Moldova, Mozambique, Palestinian Territories, Uganda, and further (“non-priority”) countries.
According to the client, APPEAR has become increasingly complex in terms of its funding provisions compared to previous programme phases. In response to this complexity, the evaluation team applied a theory-based and participatory approach in which both qualitative and quantitative evaluation methods were employed. While data from 46 interviews and 19 focus group discussions constitute the qualitative data base of the evaluation, allowing for an in-depth understanding of causal mechanisms and personal views on the programme, two online surveys targeting 280 (former) programme beneficiaries generated quantitative data that allowed for the identification of patterns and relationships that did not emerge from the qualitative data. Furthermore, on-site data collection in Armenia – as part of six case studies – provided close-up insights into programme implementation in one of APPEAR’s most recent partner countries.
The programme evaluation was conducted along the OECD DAC evaluation criteria relevance, effectiveness, and efficiency, and in accordance with ADA’s Project and Programme Evaluation Guidelines. Its main purpose was to inform a potential fourth phase of the APPEAR programme.
Analysis of customer satisfaction with GATE-Germany marketing services
GATE-Germany is financed by the German Federal Ministry of Research, Technology and Space (BMFTR). Since its establishment in 2001, the DAAD has used these funds to address the increasing demand from German universities for expertise and practical support in their international recruitment and marketing activities. GATE-Germany offers a wide range of marketing tools and other services. The customer satisfaction analysis focuses on the services offered under the umbrella term ‘Marketing Services’ (MarS).
This analysis aims to gain insight into how universities use MarS and how satisfied they are with it over time. Based on the findings, recommendations for the further development and optimisation of MarS will be developed.
In terms of methodology, a combination of qualitative and quantitative survey and evaluation methods will be employed. This will include a comprehensive document and secondary data analysis, guided interviews and group discussions, as well as an online survey.
Employee survey in the rückenwind³ project: (Self-)leadership with FLOW – freedom for solutions, organisation and change
The Caritasverband Saar-Hochwald e.V. is a welfare organisation of the Catholic Church responsible for the Saarland districts of Merzig-Wadern and Saarlouis. As part of the ESF-funded programme “rückenwind³ – für Vielfalt, Wandel und Zukunftsfähigkeit in der Sozialwirtschaft” (tailwind³ – for diversity, change and sustainability in the social economy), the association is implementing the project “(Selbst)Führung mit FLOW – Freiraum für Lösungen, Organisation und Wandel” (self-management with FLOW – freedom for solutions, organisation and change). The aim is to further develop the management and corporate culture of the Caritasverband with the involvement of its employees.
As part of the project, CEval GmbH is conducting an employee survey to provide a validated analysis of the baseline situation with regard to leadership culture and cooperation. The study follows a participatory, theory-based approach and is based on a mix of methods. The core of the study is a partially standardised online survey of all Caritas Association employees. In addition, documents are analysed and exploratory interviews are conducted with various stakeholder groups of the rückenwind³ project.
The online questionnaire is designed in such a way that it can be used for follow-up surveys during the course of the project in order to identify changes brought about by the rückenwind³ project on the basis of time series data. The results of the baseline survey will also be incorporated into the design of the rückenwind³ project.
Accompanying evaluation of the project “Strong through play” – Fit for school with exercise and healthy eating
The project ‘Spielend stark’ – Mit Bewegung & gesunder Ernährung fit in die Schule (Strong through play – Fit for school through exercise and healthy eating) – carried out by the Saarländische Krebsgesellschaft e.V. (Saarland Cancer Society) in cooperation with the Techniker Krankenkasse health insurance fund – aims to sustainably embed health awareness in the everyday lives of children in the form of regular exercise and a balanced diet, thereby strengthening a healthy lifestyle and health resources among pre-school children in the long term. Long-term, these habits can help prevent cancer. The project is aimed at kindergartens and includes preschoolers, teachers, parents, caregivers, and other family members as key target groups.
CEval GmbH provides scientific support for the project in the form of an accompanying evaluation. The aim of the evaluation is to optimise the quality of the project design and implementation in the pilot phase and to gain insights into the achievement of objectives and effectiveness in the subsequent rollout phase. The evaluation follows a participatory and theory-based approach and is based on a mix of quantitative and qualitative methods. The basis for this is an evaluation concept tailored to the specifics of the project. The concrete operationalisation, data sources and methodological instruments are developed in parallel with the project design during the course of the project. This ensures that the evaluation remains flexible and is tailored precisely to the project implementation.
Evaluation of the funding programme “Nachbarschaftsgespräche”
The Allianz für Beteiligung (“Alliance for Participation”) is a network that is committed to strengthening citizen participation and civil society activities in the Federal State of Baden-Württemberg. Since 2018, the Alliance for Participation has been implementing the funding programme “Nachbarschaftsgespräche” (“Neighbourhood Talks”) financed by the Baden-Württemberg State Ministry. It supports civil society initiatives in implementing dialogue formats in small social spaces in Baden-Württemberg; since 2018, around 100 projects have been funded under the programme.
In order to learn from the implementation to date and to be able to further develop the programme accordingly, the Alliance for Participation commissioned CEval GmbH to carry out an interim evaluation. This was based on the evaluation of 60 funded projects from the second and third funding phases (2020-2025) and focused on the aspects of quality, effectiveness, sustainability and opportunities for further development of the programme.
The evaluation followed a theory-based approach and included both qualitative and quantitative methods of data collection and analysis: document analyses, guided individual interviews, focus group discussions and an online survey. Based on the findings, opportunities for further development of the funding programme were identified and recommendations for action were derived.
Evaluation of the project “Time for Children’s Rights”
“Zeit für Kinderrechte” (“Time for Children’s Rights”) is a project organised by the Kinderschutzbund Landesverband Schleswig-Holstein (Child Protection Association Schleswig-Holstein), with the goal of strengthening pupils in their knowledge of children’s rights and their sense of self-efficacy in order to enhance their protection. In a two-day school workshop, children in 3rd and 4th grade are informed about their rights and learn what they can do if their rights are not respected. The project is running since 2019 and is funded by the Schleswig-Holstein Ministry of Education since 2020, supported by the Sparkassen- und Giroverband Schleswig-Holstein.
CEval GmbH has been commissioned by the Child Protection Association Schleswig-Holstein to carry out an interim evaluation of the project. The aim was to examine the quality and effectiveness of the project and to identify potential development needs.
The evaluation followed a participatory, theory-based approach and was based on a mix of methods that combined qualitative and quantitative instruments for data collection and analysis. This included document and secondary data analyses, an online survey and semi-structured group-interviews of teachers and school social workers as well as interviews with parents, the Ministry of Education Schleswig-Holstein and the project staff. Additionally, a child-friendly, standardised questionnaire was used to survey all pupils taking part in the workshop during the evaluation period.
Study “Music Industry of Saarland: Saarland as a Location for National and International Music Industries—Potentials and Requirements”
The subject of the study is the music industry of Saarland, with the aim of providing a more comprehensive picture of the industry. The focus is on identifying relevant stakeholders, existing structures and formats for promotion, visibility, education and training, as well as regional, national and international networking within the Saarland music industry. In addition, economic potential and infrastructure deficiencies are being assessed.
Methodologically, the study is based on the secondary analysis of existing data and studies, qualitative interviews, focus groups, and a semi-standardised online survey of professional groups working in the music industry. The study is being conducted in close consultation with the client. The results will serve as the basis for a strategy paper and a practice-oriented guide for the sustainable development of the music industry in Saarland.
Evaluation of the national competition for German-speaking acting students
The national competition for German-speaking acting students, which has been held annually since 1990, is funded by Germany’s Federal Ministry for Education, Family Affairs, Senior Citizens, Women and Youth (BMBFSFJ) and aims to honor outstanding artistic achievements, to intensify the exchange between the German-speaking drama academies of Germany, Austria and Switzerland and to facilitate the transition to artistic practice for acting students.
In order to assess the achievement of the objectives as well as the efficiency and cost-effectiveness of the national competition in compliance with the Federal Budget Code and to derive recommendations for optimizing the competition, CEval GmbH evaluated the national competitions for the years 2015 to 2025 as commissioned by the organizer – the Europäische Theaterakademie GmbH “Konrad Ekhof” Hamburg.
The evaluation followed a theory-based approach and was structured along the OECD-DAC criteria relevance, effectiveness and efficiency. Based on a mix of methods, the evaluation was primarily based on online surveys of the students and teachers that have participated in the competition. This quantitative database was supplemented by quantitative and qualitative secondary data and primary data from guideline-based (group) interviews as well as participant observation of the national competition 2025 in Salzburg.