Tracer Study of the Young Talent Scholarship Program of the Heinrich-Böll-Foundation
The Heinrich-Boell-Foundation granted since 1990 approximately 3000 scholarships to students and doctoral students of all subjects and nationalities per year. These stipend holders pursued their degree at universities, universities of applied sciences (‘Fachhochschulen’), or universities of the arts (‘Kunsthochschulen’). It is expected by the Heinrich-Boell-Foundation that scholarship recipients have excellent academic records, are socially and politically engaged, and have an active interest in the basic values of the foundation, basically ecology and sustainability, democracy and human rights, self-determination and justice.
The CEval conducted a tracer study of the promotion of young talent scholarship programme in order to evaluate its long-term impact. The impact of the scholarship programme for the target population was analyzed with a special focus on gender, diversity and the different sponsorship and conveying efforts of the foundation. The CEval followed a “mixed methods” approach with regard to data collection. As a result, existing data and documents were analyzed in detail and data derived from a quantitative online-survey with all previous stipend holders of the hbs was assessed. Qualitative semi-structured interviews with alumni and project managers of the hbs were conducted and form the qualitative part of the analysis.
Accompanying evaluation of the career support program ‘Securing talents shaping the future’ (2015-2016)
The program Securing talents shaping the future by the Cusanuswerk aims to support future female leaders. The program focuses on mentoring elements that is tutorial support for scholarship holders by professionally experienced academics. It also focuses on occupationally specific training and support measures as well as on building a peer-network.
The Center for Evaluation (CEval) has been conducting accompanying external evaluations to ensure quality management since the start of the career support program in 2007.
An accompanying evaluation will also take place during the fifth program run in 2015/16. The main task of this evaluation is to provide the responsible employees with management-related information. In this context, questions regarding satisfaction with the offered events, quality of tutorial relationships as well as questions regarding professional and personal use of the relationships for both sides are especially relevant.
The evaluation concept is based on Stufflebeams CIPP model and comparability of evaluation results regarding different program runs since the start of the career support program has been ensured. In regards to methods, the evaluation covers standardized questionnaires of mentees and mentors compared at three points in time. We will also conduct an alumni-survey of mentees that have been supported thus far.
Evaluation of the Baden-Württemberg-STIPENDIUM for University Students BWS plus
The Baden-Württemberg Stiftung has awarded the Baden-Württemberg-STIPENDIUM for more than ten years. Each year, up to 1,400 students of all nationalities receive a scholarship for Placements abroad. With the Baden-Württemberg-STIPENDIUM for University Students – BWS plus, the Baden-Württemberg Stiftung supports innovative cooperation of Baden-Württemberg universities. The aim is to establish and expand new relationships with foreign universities and the Lasting intensification of existing contacts.
CEval is commissioned to assess the degree to which the programme fulfils stated objectives. It distinguishes between objectives in term of the programme (e.g. expanding the international network, promoting innovative ideas of students) and in Terms of structure (e.g. improving the awareness of the scholarship programs and of the Baden-Württemberg Stiftung).
CEval employs a mixed-methods approach with regard to data collection. Therefore, relevant documents and accessible data will be analysed. Data will be collected using a standardized online Survey. In addition, a qualitative approach will be carried out by conducting semi-structured interviews with project participants.
Evaluation of Staff/Management Talks at Saarstahl AG
In 2013, Saarstahl AG Management introduced Performance appraisals between Supervisors and their Managers (‘Mitarbeitergespräche’, short: MAGs) to the company’s human resource management. These annual talks take place in private, Independent of any particular incident or Event and on a non-hierarchical Basis. The aim of the MAGs at Saarstahl is to improve orientation towards the company’s Targets and its corporate culture, to promote employee Training and development and to improve cooperation within the company.
The evaluation of the 2013 MAGs was commissioned in order to Analyse if the implementation process had been successful. A special Focus wa on the following aspects: preparation of employees (seminars, information material); implementation and communication of target Agreements as well as employee support and development strategies; General manageability of this tool regarding Content and structure.
In close cooperation with Saarstahl AG, CEval developed and conducted a standardised online Survey among employees at all levels of the company. In preparation, guideline-based interviews were conducted with those members of staff responsible for the implementation process as well as with selected employees at different levels of the company.
Quality Management Support at Saarland Institute for Vocational Training and Further Education in Psychotherapy (SIAP)
The Saarland Institute for Vocational Training and Further Education in Psychotherapy (SIAP) was founded in 1995 and is part of Saarland Heilstätten GmbH (SHG; association providing a range of health care Services in Saarland). SIAP is a state-approved training centre, amongst others for Counselling with a Focus on psychological approaches as well as Children and Youth Counselling.
For both programmes, SIAP commissioned Consulting sservices, including an Evaluation, in order to a) identify potentials for improvement in teaching structures and b) develop a questionnaire for ongoing Quality assurcance.
Analyses were based on existing files and records, programme documents and a review of relevant literature, but mainly on guidline-based and group interviews with all relevant actors: students (at different stages of their Trainings), teachers for practical sessions, lectures and Supervisors.
Special Focus has been laid with the question if and how optimisation of teaching processes can take pressure of programme participants while simultaneously providing a targeted preparation for their future work in counselling. Encompassing systematic data analysis resulted in a Brief evaluation report including recommendations for programme development.
Subsequently, a questionnaire for ongoing Quality assurance of both programmes was developed which took results from previously conducted literature Reviews and qualitative interviews into consideration. Additionally, CEval developed a guideline which enabled SIAP to collect and Aanalyse data independently after the end of the evaluation project.
Evaluating the implementation of the Internet strategic concept “Country Web Portals as a Cross-cultural Internet Platform”
Object of the evaluation was the implementation of the
Internet strategy concept developed by the Goethe Institute (GI), known as Country Web Portals as a Cross-cultural Internet Platform. GI adopted this concept in 2011 and based its activities on it following the guiding principle that
– the personal experience of meeting and communicating through multimedia and online interaction is of importance,
– the platform offered online is characterised through (some) diversity and durability and
– users can participate and contribute content. Thus, the intercultural platforms target both the transfer of information as well as cross-cultural awareness and Exchange. The evaluation study examined whether the objectives of the country web portals have been achieved and the intended target groups have been reached. In addition, the analysis included the question on how effectively these web portals had enabled information transfer and promoted intercultural exchange. Also, the evaluation provided specific conclusions and recommendations derived from the collected information in order to further develop the concept or the country Portals. CEval followed a “mixed methods” approach with regard to data collection. As a result, existing documents and data derived from an online survey of the GI had been analyzed in detail. Additional surveys had also been conducted (qualitative guideline based interviews).
Process Evaluation of the Career Support Program ’Talente sichern & Zukunft gestalten’ (Securing Talents Shaping the Future)
Talente sichern Zukunft gestalten (Securing Talents – Shaping the Future) is a leadership development programme for young women run by Cusanuswerk – scholarship body of the Catholic Church in Germany. Mentoring and coaching by experienced academics and professionals as well as building a peer network constitute the main elements of this program.
Since its launch in 2007, the quality of the program had been assessed annually by means of external process evaluations conducted by the Center for Evaluation (CEval), and a similar evaluation had been carried out in the programs fourth year 2013-2014. This evaluation was designed to provide responsible members of staff at Cusanuswerk with information relevant to secure a successful performance adapted to the participants specific needs. Therefore, questions about overall satisfaction with offered courses and the quality as well as the professional and personal benefits of mentoring activities for both mentees and mentors were of particular interest.
The content of the existing and proven evaluation concept was sharpened as part of the evaluation by applying the CIPP (Context, Input, Process, Product) model, while ensuring comparability of evaluation results since the start of the career development program. Methodologically, the evaluation process was based on standardized online questionnaires, which mentees and mentors completed at three different stages of the program, the aim being to identify potential difficulties and appropriate solutions sufficiently early in the process.
Member Survey of IMMOEBS e.V.
IMMOEBS e.V. is the alumni and sponsor association of post graduate and master programs of real estate economics at the European Business School and the University of Regensburg. IMMOEBS was founded in 1991 and has about 2.600 members at the moment. It is the major alumni network of real estate economics in Germany.
CEval was commissioned by IMMOEBS with the conduction of a member survey. The general purpose of this online survey was the assessment of IMMOEBS and its activities by its members as well as the identification of potentials for development and adaptation requirements.
East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.Talking about migrants lives in Russia
Subject of Evaluation was the EU-funded project Talking about migrants lives in Russia of the DW Akademie and the Foundation for Independent Radio Broadcasting. The project aimed to increase awareness about the situation of migrants within Russian society by strengthening cooperation between the Russian media and civil society organizations (CSOs) who are working on migrant issues. In order to achieve this goal, training activities for radio journalists and CSO activists were provided. The workshops aimed to introduce the CSO activists to agenda-setting mechanism of the media and to equip the journalists with storytelling skills and in-the-field reporting skills in order to do genuine social reporting on migration issues. Following the training activities the workshop participants took part in panel discussions where they presented their media productions and exchanged experiences on the reporting on migrant issues. A major medium-term outcome of those activities was the establishment of a sustainable network between CSO activists and radio journalists.
Goal of the summative ex-post evaluation was to analyze and assess the short- and medium-term outcomes of the training workshops. For this purpose the evaluation additionally analyzed to what extent the projects web platform, the panel discussions and organizational aspects in general influenced the expected outcomes. For data acquisition, CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews, document as well as secondary analysis of module evaluation surveys. Additionally a standardized online survey in Russian of all workshop participants was carried out.