Evaluation Lectureships
As the world’s largest funding organization for the exchange of students and academics, the DAAD promotes teaching stays for academics at universities abroad with its Lectureships Abroad programs. With its lectureships abroad, the DAAD pursues foreign cultural policy, development policy, higher education policy and program policy objectives. Depending on the host country, the host institution and the nature of the lectureship, these objectives vary in their focus.
The evaluation focused on the long-term, short-term, and Herder lectureships in the period under review from 2012 to 2022.
The evaluation pursued three central objectives: To examine whether the programs in their current design are up-to-date and needs-oriented, to support the accountability of the DAAD to the Federal Foreign Office (Auswärtiges Amt), and to support the strategic positioning of the programs through their further development.
In terms of methodology, a mix of qualitative and quantitative survey and evaluation methods were combined. Among other things, a comprehensive document and secondary data analysis, virtual site visits/field study, guided interviews, group discussions and online surveys were carried out. Examination and revision of the impact model as well as indicator development were part of the evaluation.
Evaluation of the German Colombian Peace Institute – CAPAZ
The German Colombian Peace Institute (CAPAZ) is one of five Centers of Excellence for Research and Teaching abroad, which has been financed by the German Federal Foreign Office since 2016. The institute is considered an academic platform for cooperation which facilitates the exchange of knowledge and experience in the field of peace-related research between research, teaching, civil society and state institutions, and, in doing so, intends to contribute to the establishment of a sustainable peace order in Colombia. The fields of activity of the CAPAZ Institute encompass teaching, research, policy advice, and knowledge transfer.
With the current funding period coming to an end and in view of an envisaged next funding period, the German Academic Exchange Service (DAAD) mandated CEval GmbH with an evaluation of the CAPAZ Institute. The mission of the evaluation was to review the project’s progress and results and, on that basis, put forth recommendations allowing to further develop and possibly improve the project and its funding. In that respect, the evaluation was based on the following evaluation criteria: relevance, effectiveness, efficiency, sustainability, as well as coordination, complementarity, and coherence.
The methodological approach chosen for this evaluation is theory-based and participatory. Furthermore, it is based on a combination of primarily qualitative methods of data collection and analysis, including an analysis of documents and secondary data, exploratory interviews as well as in-depth guided interviews and focus group discussions.
Evaluation of the ASG Mentoring Programme 2022/2023
The overriding goal of the non-profit Agrarsoziale Gesellschaft e.V. is to work for the improvement of living conditions in rural areas and in agriculture. Qualified and committed people are a central resource for the positive development of rural areas. Against this backdrop, an ASG mentoring programme was created in which students and graduates who would like to become professionally involved in and for rural areas are supported on their career path over a period of 18 months. The ASG Mentoring Programme is based on the three classic mentoring components: The tutorial support of mentees through the formation of mentoring tandems with leaders from science, administration, politics, non-governmental organisations and business, the acquisition of competencies by the mentees within the framework of the accompanying seminars of the programme, ASG conferences/seminars and the development of personal contacts and networks through the professional exchange with all stakeholder groups within the programme as well as in the context of the ASG conferences and seminars. CEval GmbH was commissioned with the accompanying evaluation of the mentoring programme. The central task of this evaluation is to provide the ASG staff who are responsible for the implementation of the project with timely and relevant information. Of particular interest are questions about satisfaction with the seminars offered, the quality of the tutorial relationships, and the professional and personal benefits of the relationship for both sides. Methodologically, the evaluation is based on standardised online surveys of the mentees and/or mentors at six points in time and on qualitative interviews of the programme leaders and mentors as well as on (focus) group discussions with the mentees.
Implementation of an evaluation system for courses at the August Everding Theatre Academy
The August Everding Theatre Academy offers various artistic study programmes at Bachelor’s and Master’s level. In order to determine the students’ satisfaction with individual courses and to identify potential for improvement, the August Everding Theatre Academy has commissioned CEval GmbH to implement an evaluation system for courses at the August Everding Theatre Academy. The aim is to create a regular database for the further development and optimisation of the courses. Methodologically, semi-standardised surveys will be designed for various teaching formats, which can be used for the evaluations of different courses (practically oriented and theoretically oriented courses as well as individual lessons).
Synthesis and Meta-Evaluation of Project and Program Evaluations of the Centers for German and European Studies (CGES)
In view of its long-term goal to contribute to the internationalization of the science and innovation systems in Germany, Europe and worldwide and to shaping education and research in relation to Germany and Europe, the DAAD supports interdisciplinary Centers for German and European Studies (CGES) at universities abroad. The Centers train future experts in the field of German and European Studies, among others, by offering respective study programs. They also engage in a discourse on topics related to Germany and Europe with stakeholders from science, politics, and the interested public.
To continuously improve the effectiveness of its work, the DAAD had mandated CEval GmbH with the implementation of a meta-evaluation and synthesis regarding the CGES. The object of the study were the project and program evaluations on the CGES implemented between 2015 and 2020. The task entailed a criteria-based synthesis of the findings documented in the evaluation reports as well as an assessment of the quality of the evaluations themselves (meta-evaluation). The objective of the study was, among others, to assess the effectiveness and goal achievement in relation to the CGES from a birds-eye-view, to identify potentials for improvement and to develop recommendations for the future support of the CGES as well as in view of any future CGES evaluations.
Methodologically, the study was desk-based. For the meta-evaluation, the evaluation team developed an assessment scheme as the primary data collection tool. On that basis, the evaluation reports were systematically analyzed and assessed according to a set of previously defined quality criteria and in relation to six different dimensions of analysis. Similarly, an analysis grid was developed for the purpose of the synthesis. It served to derive a coding scheme and, on that basis, implement a qualitative content analysis of the evaluation reports, which then allowed for a synthesize of the evaluation findings.
Evaluation of the German-Argentinean University Center (DAHZ-CUAA)
The German-Argentinean University Center (DAHZ-CUAA) is a binational university center that aims at promoting and strengthening the institutional, academic, and scientific cooperation between Argentina and Germany. The main objective is the creation of innovative, binational study programs with double degrees. In order to achieve this objective, there are three program lines to support universities in both countries, i.e., the German-Argentinean program to promote binational study programs with double degrees (main program line), the I.DEAR program (Engineers Germany-Argentina), and the program to promote accompanying research projects.
To gain insights into the successes, potentials for improvement, and the progress made in the three program lines and to be able to further develop the programs and to make them even more effective on that basis, the German office of the center mandated CEval GmbH in cooperation with smep consult e.U. with an evaluation of the center’s work. The mission of the evaluation was to assess the relevance, effectiveness (i.e., goal achievement and impact), efficiency, and sustainability of the center and its program lines and to put forth recommendations that allow further development and possibly improving the program lines in the future.
Methodologically, the evaluation was based on a combination of qualitative and quantitative methods of data collection and analysis. It included the analysis of documents and secondary data, site visits in Germany and Argentina, guided interviews, group discussions and online surveys.
Framework contract for the further development and implementation of systematic results-based approaches through monitoring and evaluation for DAAD programs (2020)
The DAAD understands itself as a modern science organization that strives to continuously improve its effectiveness through continuous learning and the willingness to change its work. This approach is formulated in the DAAD’s Integrated Monitoring and Evaluation Concept, which was adopted in 2019. The concept identifies institutional learning, quality assurance, results-based orientation, transparency in processes and legitimacy as important dimensions to which the various monitoring and evaluation activities are geared. Doing so, from an overarching perspective, the strategic orientation of the DAAD and its programs is also to be supported on an evidence-based basis.
The framework contract with CEval GmbH aims to meet the increasing demands for results-based orientation and control in the various funding areas of the DAAD even better by supporting the implementation of results-based orientation and control for existing and new programmes in the AA- and BMBF-funding areas and by accompanying the results-based orientation of newly designed BMZ programmes. To this end, services are provided in the following areas:
Evaluation of the Alexander von Humboldt Foundation’s Research Fellowship Programmes for Researchers from Germany (FLF, JSPS, MOST)
The Alexander von Humboldt Foundation (AvH) enables highly qualified academics from all over the world to spend longer periods of research in Germany and supports the resulting academic and cultural links between Germany and abroad. The Foundation’s network now comprises over 29,000 researchers from all disciplines in more than 140 countries.
To strengthen the Humboldt Network, the AvH’s programme portfolio also includes three research fellowship programmes that support researchers from Germany to work at research institutions abroad. The BMBF-funded Feodor Lynen Research Fellowship Programme (FLF) enables German researchers to spend a longer period of time researching abroad. In this way, the existing Humboldt Network abroad is strengthened and the potential of Humboldt alumni abroad is tapped in their function as academic advisors and “door openers” for (junior) researchers from Germany. For the target countries Japan and Taiwan, the Humboldt Foundation cooperates with the Japan Society for the Promotion of Science (JSPS) and the Ministry of Science and Technology, Taiwan (MOST) within the framework of the JSPS Programme and the MOST Programme. The AvH also offers support services for alumni from all three scholarship programmes from the BMBF funding. These include reintegration assistance, return scholarships, renewed research stays, institute partnerships as well as annual conferences and networking events.
The evaluation is intended to fulfil two functions: On the one hand, it should provide an evidence-based foundation for the further development of the programmes in the sense of a formative evaluation. On the other hand, it is to be used to account to the funding agencies and the public for the implementation and achievement of the goals of the scholarship programmes. The evaluation focuses on the extent to which the Humboldt Network has benefited from the three programmes and whether suitable research fellows have been selected with regard to the programme goals.
The evaluation is characterised by a participatory and multi-method approach. The evaluation combines document and secondary analyses with qualitative (guided intensive interviews) and quantitative survey procedures (partially standardised online surveys).
International consultancy to conduct mapping of Early Childhood Care Development and Education (ECCDE) Centres in Eswatini
The purpose of this assignment was to support the Ministry of Education and Training in Eswatini to conduct a mapping of all early childhood care, development, and education programs (ECCDE). The mapping had been conducted on all centres that provide ECCDE services (formal and informal), including detailing critical information on location, enrollment, infrastructure, and the type of services provided. Results from the ECCDE mapping are expected to inform the development of the Education Sector Strategic Plan 2022 – 2034, including to promote the improvement of access and quality of ECCDE services across the country.
CEval is responsible for the digitalization of the questionnaire with the help of the software Kobocollect, data cleaning and statistical analysis.
Mid-Term Evaluation of the „Palestinian-German Science Bridge” (PGSB)
The program Palestinian-German Science Bridge is a pilot project financed by the German Federal Ministry of Education and Research (BMBF), which is jointly implemented by Forschungszentrum Jülich and the Palestinian Academy for Science and Technology (PALAST). It aims at establishing a joint education and research cooperation between Forschungszentrum Jülich and the Palestinian partner universities. To this end, the program combines measures focusing on student and researcher mobility, scientific exchange among faculty staff, the support of research collaborations and the reintegration of young Palestinian researchers into the Palestinian academic sector.
Forschungszentrum Jülich mandated CEval GmbH to counsel the Forschungszentrum with regard to the conceptualization of a mid-term evaluation of the program and to assist the Forschungszentrum in implementing it. The purpose of the evaluation was to assess the short- and medium-term results of the program as well as the prospects for long-term goal achievement. The objective, further, was to identify potentials for improvement in relation to the program.
Methodologically, the evaluation was based on a mix of qualitative and quantitative data collection and analysis methods. Data collection combines a document and secondary data analysis with guided interviews and semi-standardized online surveys among the project stakeholders. During data analysis, descriptive statistics and qualitative content analysis were applied. Due to the COVID-19 pandemic, the evaluation was planned as a remote evaluation.