Tracer Study of the Young Talent Scholarship Program of the Heinrich-Böll-Foundation
The Heinrich-Boell-Foundation granted since 1990 approximately 3000 scholarships to students and doctoral students of all subjects and nationalities per year. These stipend holders pursued their degree at universities, universities of applied sciences (‘Fachhochschulen’), or universities of the arts (‘Kunsthochschulen’). It is expected by the Heinrich-Boell-Foundation that scholarship recipients have excellent academic records, are socially and politically engaged, and have an active interest in the basic values of the foundation, basically ecology and sustainability, democracy and human rights, self-determination and justice.
The CEval conducted a tracer study of the promotion of young talent scholarship programme in order to evaluate its long-term impact. The impact of the scholarship programme for the target population was analyzed with a special focus on gender, diversity and the different sponsorship and conveying efforts of the foundation. The CEval followed a “mixed methods” approach with regard to data collection. As a result, existing data and documents were analyzed in detail and data derived from a quantitative online-survey with all previous stipend holders of the hbs was assessed. Qualitative semi-structured interviews with alumni and project managers of the hbs were conducted and form the qualitative part of the analysis.
Accompanying evaluation of the career support program ‘Securing talents shaping the future’ (2015-2016)
The program Securing talents shaping the future by the Cusanuswerk aims to support future female leaders. The program focuses on mentoring elements that is tutorial support for scholarship holders by professionally experienced academics. It also focuses on occupationally specific training and support measures as well as on building a peer-network.
The Center for Evaluation (CEval) has been conducting accompanying external evaluations to ensure quality management since the start of the career support program in 2007.
An accompanying evaluation will also take place during the fifth program run in 2015/16. The main task of this evaluation is to provide the responsible employees with management-related information. In this context, questions regarding satisfaction with the offered events, quality of tutorial relationships as well as questions regarding professional and personal use of the relationships for both sides are especially relevant.
The evaluation concept is based on Stufflebeams CIPP model and comparability of evaluation results regarding different program runs since the start of the career support program has been ensured. In regards to methods, the evaluation covers standardized questionnaires of mentees and mentors compared at three points in time. We will also conduct an alumni-survey of mentees that have been supported thus far.
Evaluation of the Baden-Württemberg-STIPENDIUM for University Students BWS plus
The Baden-Württemberg Stiftung has awarded the Baden-Württemberg-STIPENDIUM for more than ten years. Each year, up to 1,400 students of all nationalities receive a scholarship for Placements abroad. With the Baden-Württemberg-STIPENDIUM for University Students – BWS plus, the Baden-Württemberg Stiftung supports innovative cooperation of Baden-Württemberg universities. The aim is to establish and expand new relationships with foreign universities and the Lasting intensification of existing contacts.
CEval is commissioned to assess the degree to which the programme fulfils stated objectives. It distinguishes between objectives in term of the programme (e.g. expanding the international network, promoting innovative ideas of students) and in Terms of structure (e.g. improving the awareness of the scholarship programs and of the Baden-Württemberg Stiftung).
CEval employs a mixed-methods approach with regard to data collection. Therefore, relevant documents and accessible data will be analysed. Data will be collected using a standardized online Survey. In addition, a qualitative approach will be carried out by conducting semi-structured interviews with project participants.
Process Evaluation of the Career Support Program ’Talente sichern & Zukunft gestalten’ (Securing Talents Shaping the Future)
Talente sichern Zukunft gestalten (Securing Talents – Shaping the Future) is a leadership development programme for young women run by Cusanuswerk – scholarship body of the Catholic Church in Germany. Mentoring and coaching by experienced academics and professionals as well as building a peer network constitute the main elements of this program.
Since its launch in 2007, the quality of the program had been assessed annually by means of external process evaluations conducted by the Center for Evaluation (CEval), and a similar evaluation had been carried out in the programs fourth year 2013-2014. This evaluation was designed to provide responsible members of staff at Cusanuswerk with information relevant to secure a successful performance adapted to the participants specific needs. Therefore, questions about overall satisfaction with offered courses and the quality as well as the professional and personal benefits of mentoring activities for both mentees and mentors were of particular interest.
The content of the existing and proven evaluation concept was sharpened as part of the evaluation by applying the CIPP (Context, Input, Process, Product) model, while ensuring comparability of evaluation results since the start of the career development program. Methodologically, the evaluation process was based on standardized online questionnaires, which mentees and mentors completed at three different stages of the program, the aim being to identify potential difficulties and appropriate solutions sufficiently early in the process.
Member Survey of IMMOEBS e.V.
IMMOEBS e.V. is the alumni and sponsor association of post graduate and master programs of real estate economics at the European Business School and the University of Regensburg. IMMOEBS was founded in 1991 and has about 2.600 members at the moment. It is the major alumni network of real estate economics in Germany.
CEval was commissioned by IMMOEBS with the conduction of a member survey. The general purpose of this online survey was the assessment of IMMOEBS and its activities by its members as well as the identification of potentials for development and adaptation requirements.
East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.Talking about migrants lives in Russia
Subject of Evaluation was the EU-funded project Talking about migrants lives in Russia of the DW Akademie and the Foundation for Independent Radio Broadcasting. The project aimed to increase awareness about the situation of migrants within Russian society by strengthening cooperation between the Russian media and civil society organizations (CSOs) who are working on migrant issues. In order to achieve this goal, training activities for radio journalists and CSO activists were provided. The workshops aimed to introduce the CSO activists to agenda-setting mechanism of the media and to equip the journalists with storytelling skills and in-the-field reporting skills in order to do genuine social reporting on migration issues. Following the training activities the workshop participants took part in panel discussions where they presented their media productions and exchanged experiences on the reporting on migrant issues. A major medium-term outcome of those activities was the establishment of a sustainable network between CSO activists and radio journalists.
Goal of the summative ex-post evaluation was to analyze and assess the short- and medium-term outcomes of the training workshops. For this purpose the evaluation additionally analyzed to what extent the projects web platform, the panel discussions and organizational aspects in general influenced the expected outcomes. For data acquisition, CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews, document as well as secondary analysis of module evaluation surveys. Additionally a standardized online survey in Russian of all workshop participants was carried out.Strengthening evaluation capacities within government and civil society in Uganda (ECD Uganda): Development of a blended-learning Master of Evaluation (2012-2015)
Evaluation as a scientific means to describe and assess interventions is gaining relevance worldwide. The needs for qualified evaluation professionals are increasing in Uganda, too. However, at the moment these needs cannot be satisfied due to lack of training- and evaluation programs in this region. It is for this reason that the BMZ together with CEval and GIZ initiated a program for developing evaluation capacities in Uganda.
The most important component of this program is the development and implementation of a master course in evaluation in the blended-learning format at Uganda Technology and Management University (UTAMU) in Kampala, Uganda. Blended-learning is an innovative concept which is gaining popularity worldwide and is now the default means of teaching at UTAMU. Blended-learning unites traditional face-to-face classes with modern forms of e-learning in a didactical coherent way. An advantage of this concept is, that it is possible not only for Ugandan students to attend classes but also for all students of English-speaking African countries.
The Master of Evaluation of Saarland University in cooperation with HTW Saarland serves as a template for the Ugandan Master, which will be introduced in fall 2015. The contents of the master program have been developed in cooperation with CEval GmbH, the teachers of the German master as well as Ugandan contact persons of Makarere University and have been adapted to the Ugandan context. The Ugandan Master of Monitoring and Evaluation will comprise 15 modules in total, seven of which are in compliance with the basic teaching program of evaluation at Saarland universities. This is to ensure that all topics relevant to acquire profound knowledge in the field of evaluation are covered. UTAMU has complemented this program with further modules regarding project management, monitoring as well as business start-up consulting. The modules are roughly comprised of web-based trainings, e-tutoring and face-to-face seminars. The teaching concepts have also been developed in cooperation with all stakeholders. The GIZ is responsible for the technical implementation of the e-learning concepts.
Two modules of the master of evaluation have been tested in an experimental phase together with UTAMU. During this experimental phase e-tutoring as well as the face-to-face seminars have been supervised in the tandem-principle by teachers of UTAMU and Saarland University/HTW. The hosting of the online study course has been undertaken by the GIZ platform GC 21. In a next step, the seven modules have been integrated into the already existing UTAMU master and the new course of study has been submitted to the responsible committees. The course of study is now aggregated and will be introduced experimentally. A decision about whether the course of study will be continued and which modifications would be necessary for this procedure, will be made in December 2016.
Constraints Faced by Graduates in the Labour Market of the Greater Region
The Unit for co-operation between Science and the Working World (KoWa) in cooperation with Arbeitskammer (Chamber of Labour) Saarland commissioned the research project Labour Market for Graduates in the Greater Region (Saarland and Rhineland-Palatinate in Germany, the Grand Duchy of Luxembourg, the French region of Lorraine and Belgian Wallonia). The intended use of this study was to provide findings about graduates labour market opportunities in Saarland, as well as about favorable and constraining conditions for graduates who wish to enter the border-crossing or the Saarland labour market. Thanks to previous research, it was known that graduates face considerable difficulties when trying to gain access to the border-crossing labour market, despite a number of specialized placement services or study programs which both involve several universities in the Greater Region.
Yet, an encompassing, representative survey among graduates of higher education institutions in Saarland, focusing on motives and backgrounds of the decision if or if not to search for jobs in neighbouring foreign countries during their transition into employment, has not yet been conducted. Also, there was hardly any information about those graduates from Lorraine, Luxembourg and Wallonia who search for jobs in Saarland. This study aimed at providing the lacking data by conducting a graduate survey among all students who graduated from Saarland University during the last eight years. Via email and via postcards sent to their home addresses, graduates had been invited to participate in an online survey. Additional graduate surveys at different universities of the Greater Region were being planned.
Evaluation of Staff/Management Talks at Saarstahl AG
In 2013, Saarstahl AG Management introduced Performance appraisals between Supervisors and their Managers (‘Mitarbeitergespräche’, short: MAGs) to the company’s human resource management. These annual talks take place in private, Independent of any particular incident or Event and on a non-hierarchical Basis. The aim of the MAGs at Saarstahl is to improve orientation towards the company’s Targets and its corporate culture, to promote employee Training and development and to improve cooperation within the company.
The evaluation of the 2013 MAGs was commissioned in order to Analyse if the implementation process had been successful. A special Focus wa on the following aspects: preparation of employees (seminars, information material); implementation and communication of target Agreements as well as employee support and development strategies; General manageability of this tool regarding Content and structure.
In close cooperation with Saarstahl AG, CEval developed and conducted a standardised online Survey among employees at all levels of the company. In preparation, guideline-based interviews were conducted with those members of staff responsible for the implementation process as well as with selected employees at different levels of the company.
