Accompanying evaluation of the support programme “Female Producer Collective“ (2023)
The program “Female Producer Collective” of GOFORMUSIC gGmbH is a special support project for female producers, who receive coaching in the areas of “Creativity and Idea Generation”, “Recording & Sound Design”, “Mixing & Mastering” and “Business & Marketing”. The aim of the project is to specifically promote female producers, to counteract the low quota of women in the production sector and to inspire female musicians to produce. CEval GmbH was already commissioned with the accompanying evaluation of the funding project in the first funding round (2022). In 2023, the programme “Female Producer Collective” started with a second round, whereby CEval GmbH had again conducted an accompanying evaluation. From a methodological point of view, focus group interviews with participants and coaches of the programme as well as an online survey via a standardised questionnaire had been conducted.
Data collection for the certificate “Fan project in accordance with the National Plan for Safety in Sports” (German: „Fanprojekt nach dem Nationalen Konzept Sport und Sicherheit“)
In Germany, there is a dense network of social work institutions in the field of adolescent soccer fans (“fan projects”) that work in accordance with the National Plan for Safety in Sports (German: Nationales Konzept Sport und Sicherheit). A certification of those fan projects has been launched in 2010. Offering fan projects an instrument of quality control, the certification is intended to pursue multiple goals: Establishing a common standard for fan projects, increasing confidence in the work of fan projects, and improving the working context of fan projects. The certificate is awarded on the basis of a recommendation by the Quality Management Group of the Centre for Fan Project Coordination of the German Sports Youth (German: Koordinationsstelle Fanprojekte bei der Deutschen Sportjugend). The Quality Management Group includes representatives from soccer, the highest state and family authorities, science, and fan projects themselves. After the certification was revised and tested by CEval GmbH in two previous projects and subsequently rolled out nationwide by collecting the information required for the certification in all fan projects, CEval GmbH has now been commissioned with the data collection for the third phase of the certification process for the current 71 fan projects.
Evaluation of a digital Senior Citizens Portal “digital.vital”
The senior citizens’ portal “digital.vital” is a model project of the Amt Hüttener Berge, the district of Germersheim and the city of Püttlingen. The aim of the portal is to make a contribution to meeting the challenges associated with demographic change and advancing digitalisation: Through needs-based offers, people are to be supported in the areas of nutrition, exercise and leisure activities/social participation (incl. housing, mobility, support) in achieving and maintaining a high quality of life in old age. At the same time, the portal also aims to strengthen the digital participation of older people. The project started in July 2020 and from autumn 2022 various services and information will be made available on the portal.
The Amt Hüttener Berge has commissioned CEval GmbH to evaluate the senior citizens’ portal in order to gain insights into the quality of the information and services offered and to identify possible potential for optimising the user experience in terms of portal functionality and content. Methodologically, a total of four participant observations will be carried out for the structured evaluation of the user experience among the pilot users in the context of so-called digital workshops at various locations. The participant observations will be supplemented by follow-up interviews with selected participants a few days after their visit to the digital workshops in order to collect data on the actual benefits of the workshops and the use of the website by the participants. In addition, a standardised online survey will be conducted among senior citizens’ organisations and actors in the partner municipalities on their perceptions and professional assessments of the senior citizens’ portal, its services and potential for further development.
Evaluation Lectureships
As the world’s largest funding organization for the exchange of students and academics, the DAAD promotes teaching stays for academics at universities abroad with its Lectureships Abroad programs. With its lectureships abroad, the DAAD pursues foreign cultural policy, development policy, higher education policy and program policy objectives. Depending on the host country, the host institution and the nature of the lectureship, these objectives vary in their focus.
The evaluation focused on the long-term, short-term, and Herder lectureships in the period under review from 2012 to 2022.
The evaluation pursued three central objectives: To examine whether the programs in their current design are up-to-date and needs-oriented, to support the accountability of the DAAD to the Federal Foreign Office (Auswärtiges Amt), and to support the strategic positioning of the programs through their further development.
In terms of methodology, a mix of qualitative and quantitative survey and evaluation methods were combined. Among other things, a comprehensive document and secondary data analysis, virtual site visits/field study, guided interviews, group discussions and online surveys were carried out. Examination and revision of the impact model as well as indicator development were part of the evaluation.
Evaluation of the German Colombian Peace Institute – CAPAZ
The German Colombian Peace Institute (CAPAZ) is one of five Centers of Excellence for Research and Teaching abroad, which has been financed by the German Federal Foreign Office since 2016. The institute is considered an academic platform for cooperation which facilitates the exchange of knowledge and experience in the field of peace-related research between research, teaching, civil society and state institutions, and, in doing so, intends to contribute to the establishment of a sustainable peace order in Colombia. The fields of activity of the CAPAZ Institute encompass teaching, research, policy advice, and knowledge transfer.
With the current funding period coming to an end and in view of an envisaged next funding period, the German Academic Exchange Service (DAAD) mandated CEval GmbH with an evaluation of the CAPAZ Institute. The mission of the evaluation was to review the project’s progress and results and, on that basis, put forth recommendations allowing to further develop and possibly improve the project and its funding. In that respect, the evaluation was based on the following evaluation criteria: relevance, effectiveness, efficiency, sustainability, as well as coordination, complementarity, and coherence.
The methodological approach chosen for this evaluation is theory-based and participatory. Furthermore, it is based on a combination of primarily qualitative methods of data collection and analysis, including an analysis of documents and secondary data, exploratory interviews as well as in-depth guided interviews and focus group discussions.
Evaluation of the ASG Mentoring Programme 2022/2023
The overriding goal of the non-profit Agrarsoziale Gesellschaft e.V. is to work for the improvement of living conditions in rural areas and in agriculture. Qualified and committed people are a central resource for the positive development of rural areas. Against this backdrop, an ASG mentoring programme was created in which students and graduates who would like to become professionally involved in and for rural areas are supported on their career path over a period of 18 months. The ASG Mentoring Programme is based on the three classic mentoring components: The tutorial support of mentees through the formation of mentoring tandems with leaders from science, administration, politics, non-governmental organisations and business, the acquisition of competencies by the mentees within the framework of the accompanying seminars of the programme, ASG conferences/seminars and the development of personal contacts and networks through the professional exchange with all stakeholder groups within the programme as well as in the context of the ASG conferences and seminars. CEval GmbH was commissioned with the accompanying evaluation of the mentoring programme. The central task of this evaluation is to provide the ASG staff who are responsible for the implementation of the project with timely and relevant information. Of particular interest are questions about satisfaction with the seminars offered, the quality of the tutorial relationships, and the professional and personal benefits of the relationship for both sides. Methodologically, the evaluation is based on standardised online surveys of the mentees and/or mentors at six points in time and on qualitative interviews of the programme leaders and mentors as well as on (focus) group discussions with the mentees.
Implementation of an evaluation system for courses at the August Everding Theatre Academy
The August Everding Theatre Academy offers various artistic study programmes at Bachelor’s and Master’s level. In order to determine the students’ satisfaction with individual courses and to identify potential for improvement, the August Everding Theatre Academy has commissioned CEval GmbH to implement an evaluation system for courses at the August Everding Theatre Academy. The aim is to create a regular database for the further development and optimisation of the courses. Methodologically, semi-standardised surveys will be designed for various teaching formats, which can be used for the evaluations of different courses (practically oriented and theoretically oriented courses as well as individual lessons).
Synthesis and Meta-Evaluation of Project and Program Evaluations of the Centers for German and European Studies (CGES)
In view of its long-term goal to contribute to the internationalization of the science and innovation systems in Germany, Europe and worldwide and to shaping education and research in relation to Germany and Europe, the DAAD supports interdisciplinary Centers for German and European Studies (CGES) at universities abroad. The Centers train future experts in the field of German and European Studies, among others, by offering respective study programs. They also engage in a discourse on topics related to Germany and Europe with stakeholders from science, politics, and the interested public.
To continuously improve the effectiveness of its work, the DAAD had mandated CEval GmbH with the implementation of a meta-evaluation and synthesis regarding the CGES. The object of the study were the project and program evaluations on the CGES implemented between 2015 and 2020. The task entailed a criteria-based synthesis of the findings documented in the evaluation reports as well as an assessment of the quality of the evaluations themselves (meta-evaluation). The objective of the study was, among others, to assess the effectiveness and goal achievement in relation to the CGES from a birds-eye-view, to identify potentials for improvement and to develop recommendations for the future support of the CGES as well as in view of any future CGES evaluations.
Methodologically, the study was desk-based. For the meta-evaluation, the evaluation team developed an assessment scheme as the primary data collection tool. On that basis, the evaluation reports were systematically analyzed and assessed according to a set of previously defined quality criteria and in relation to six different dimensions of analysis. Similarly, an analysis grid was developed for the purpose of the synthesis. It served to derive a coding scheme and, on that basis, implement a qualitative content analysis of the evaluation reports, which then allowed for a synthesize of the evaluation findings.
Evaluation of the German-Argentinean University Center (DAHZ-CUAA)
The German-Argentinean University Center (DAHZ-CUAA) is a binational university center that aims at promoting and strengthening the institutional, academic, and scientific cooperation between Argentina and Germany. The main objective is the creation of innovative, binational study programs with double degrees. In order to achieve this objective, there are three program lines to support universities in both countries, i.e., the German-Argentinean program to promote binational study programs with double degrees (main program line), the I.DEAR program (Engineers Germany-Argentina), and the program to promote accompanying research projects.
To gain insights into the successes, potentials for improvement, and the progress made in the three program lines and to be able to further develop the programs and to make them even more effective on that basis, the German office of the center mandated CEval GmbH in cooperation with smep consult e.U. with an evaluation of the center’s work. The mission of the evaluation was to assess the relevance, effectiveness (i.e., goal achievement and impact), efficiency, and sustainability of the center and its program lines and to put forth recommendations that allow further development and possibly improving the program lines in the future.
Methodologically, the evaluation was based on a combination of qualitative and quantitative methods of data collection and analysis. It included the analysis of documents and secondary data, site visits in Germany and Argentina, guided interviews, group discussions and online surveys.
Employee survey on working time and mobile work
The TU Darmstadt (TUD) pursues the goal of developing working time models and regulations for mobile work that are oriented towards the needs of employees. In this way, the satisfaction of employees with the organisation of working time and place of work is to be promoted. To this end, the TUD Executive Board commissioned CEval GmbH to conduct an employee survey on the topics of working hours and mobile work.
The surveys were thematically oriented in such a way that, on the one hand, the lived practice of organising working time and using mobile work was recorded, and on the other hand, the respondents’ satisfaction with this practice as well as suggestions and wishes for the future. Without this being foreseeable when the study was initiated, the Corona pandemic led to considerable adjustments in the design of working conditions during the survey period, especially with regard to mobile work. During the fine-conception of the study, these changes were taken into account by also including questions on the (new) experiences regarding mobile working during the pandemic.
The data collection combined a document analysis with a partially standardised online survey of all employee groups at the TUD as well as guideline-based interviews. The data analysis combined descriptive statistics and qualitative content analysis.
