Evaluation of the Baden-Württemberg-STIPENDIUM for University Students BWS plus
The Baden-Württemberg Stiftung has awarded the Baden-Württemberg-STIPENDIUM for more than ten years. Each year, up to 1,400 students of all nationalities receive a scholarship for Placements abroad. With the Baden-Württemberg-STIPENDIUM for University Students – BWS plus, the Baden-Württemberg Stiftung supports innovative cooperation of Baden-Württemberg universities. The aim is to establish and expand new relationships with foreign universities and the Lasting intensification of existing contacts.
CEval is commissioned to assess the degree to which the programme fulfils stated objectives. It distinguishes between objectives in term of the programme (e.g. expanding the international network, promoting innovative ideas of students) and in Terms of structure (e.g. improving the awareness of the scholarship programs and of the Baden-Württemberg Stiftung).
CEval employs a mixed-methods approach with regard to data collection. Therefore, relevant documents and accessible data will be analysed. Data will be collected using a standardized online Survey. In addition, a qualitative approach will be carried out by conducting semi-structured interviews with project participants.
East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.Talking about migrants lives in Russia
Subject of Evaluation was the EU-funded project Talking about migrants lives in Russia of the DW Akademie and the Foundation for Independent Radio Broadcasting. The project aimed to increase awareness about the situation of migrants within Russian society by strengthening cooperation between the Russian media and civil society organizations (CSOs) who are working on migrant issues. In order to achieve this goal, training activities for radio journalists and CSO activists were provided. The workshops aimed to introduce the CSO activists to agenda-setting mechanism of the media and to equip the journalists with storytelling skills and in-the-field reporting skills in order to do genuine social reporting on migration issues. Following the training activities the workshop participants took part in panel discussions where they presented their media productions and exchanged experiences on the reporting on migrant issues. A major medium-term outcome of those activities was the establishment of a sustainable network between CSO activists and radio journalists.
Goal of the summative ex-post evaluation was to analyze and assess the short- and medium-term outcomes of the training workshops. For this purpose the evaluation additionally analyzed to what extent the projects web platform, the panel discussions and organizational aspects in general influenced the expected outcomes. For data acquisition, CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews, document as well as secondary analysis of module evaluation surveys. Additionally a standardized online survey in Russian of all workshop participants was carried out.Strengthening evaluation capacities within government and civil society in Uganda (ECD Uganda): Development of a blended-learning Master of Evaluation (2012-2015)
Evaluation as a scientific means to describe and assess interventions is gaining relevance worldwide. The needs for qualified evaluation professionals are increasing in Uganda, too. However, at the moment these needs cannot be satisfied due to lack of training- and evaluation programs in this region. It is for this reason that the BMZ together with CEval and GIZ initiated a program for developing evaluation capacities in Uganda.
The most important component of this program is the development and implementation of a master course in evaluation in the blended-learning format at Uganda Technology and Management University (UTAMU) in Kampala, Uganda. Blended-learning is an innovative concept which is gaining popularity worldwide and is now the default means of teaching at UTAMU. Blended-learning unites traditional face-to-face classes with modern forms of e-learning in a didactical coherent way. An advantage of this concept is, that it is possible not only for Ugandan students to attend classes but also for all students of English-speaking African countries.
The Master of Evaluation of Saarland University in cooperation with HTW Saarland serves as a template for the Ugandan Master, which will be introduced in fall 2015. The contents of the master program have been developed in cooperation with CEval GmbH, the teachers of the German master as well as Ugandan contact persons of Makarere University and have been adapted to the Ugandan context. The Ugandan Master of Monitoring and Evaluation will comprise 15 modules in total, seven of which are in compliance with the basic teaching program of evaluation at Saarland universities. This is to ensure that all topics relevant to acquire profound knowledge in the field of evaluation are covered. UTAMU has complemented this program with further modules regarding project management, monitoring as well as business start-up consulting. The modules are roughly comprised of web-based trainings, e-tutoring and face-to-face seminars. The teaching concepts have also been developed in cooperation with all stakeholders. The GIZ is responsible for the technical implementation of the e-learning concepts.
Two modules of the master of evaluation have been tested in an experimental phase together with UTAMU. During this experimental phase e-tutoring as well as the face-to-face seminars have been supervised in the tandem-principle by teachers of UTAMU and Saarland University/HTW. The hosting of the online study course has been undertaken by the GIZ platform GC 21. In a next step, the seven modules have been integrated into the already existing UTAMU master and the new course of study has been submitted to the responsible committees. The course of study is now aggregated and will be introduced experimentally. A decision about whether the course of study will be continued and which modifications would be necessary for this procedure, will be made in December 2016.
Constraints Faced by Graduates in the Labour Market of the Greater Region
The Unit for co-operation between Science and the Working World (KoWa) in cooperation with Arbeitskammer (Chamber of Labour) Saarland commissioned the research project Labour Market for Graduates in the Greater Region (Saarland and Rhineland-Palatinate in Germany, the Grand Duchy of Luxembourg, the French region of Lorraine and Belgian Wallonia). The intended use of this study was to provide findings about graduates labour market opportunities in Saarland, as well as about favorable and constraining conditions for graduates who wish to enter the border-crossing or the Saarland labour market. Thanks to previous research, it was known that graduates face considerable difficulties when trying to gain access to the border-crossing labour market, despite a number of specialized placement services or study programs which both involve several universities in the Greater Region.
Yet, an encompassing, representative survey among graduates of higher education institutions in Saarland, focusing on motives and backgrounds of the decision if or if not to search for jobs in neighbouring foreign countries during their transition into employment, has not yet been conducted. Also, there was hardly any information about those graduates from Lorraine, Luxembourg and Wallonia who search for jobs in Saarland. This study aimed at providing the lacking data by conducting a graduate survey among all students who graduated from Saarland University during the last eight years. Via email and via postcards sent to their home addresses, graduates had been invited to participate in an online survey. Additional graduate surveys at different universities of the Greater Region were being planned.
Evaluation of Staff/Management Talks at Saarstahl AG
In 2013, Saarstahl AG Management introduced Performance appraisals between Supervisors and their Managers (‘Mitarbeitergespräche’, short: MAGs) to the company’s human resource management. These annual talks take place in private, Independent of any particular incident or Event and on a non-hierarchical Basis. The aim of the MAGs at Saarstahl is to improve orientation towards the company’s Targets and its corporate culture, to promote employee Training and development and to improve cooperation within the company.
The evaluation of the 2013 MAGs was commissioned in order to Analyse if the implementation process had been successful. A special Focus wa on the following aspects: preparation of employees (seminars, information material); implementation and communication of target Agreements as well as employee support and development strategies; General manageability of this tool regarding Content and structure.
In close cooperation with Saarstahl AG, CEval developed and conducted a standardised online Survey among employees at all levels of the company. In preparation, guideline-based interviews were conducted with those members of staff responsible for the implementation process as well as with selected employees at different levels of the company.
Quality Management Support at Saarland Institute for Vocational Training and Further Education in Psychotherapy (SIAP)
The Saarland Institute for Vocational Training and Further Education in Psychotherapy (SIAP) was founded in 1995 and is part of Saarland Heilstätten GmbH (SHG; association providing a range of health care Services in Saarland). SIAP is a state-approved training centre, amongst others for Counselling with a Focus on psychological approaches as well as Children and Youth Counselling.
For both programmes, SIAP commissioned Consulting sservices, including an Evaluation, in order to a) identify potentials for improvement in teaching structures and b) develop a questionnaire for ongoing Quality assurcance.
Analyses were based on existing files and records, programme documents and a review of relevant literature, but mainly on guidline-based and group interviews with all relevant actors: students (at different stages of their Trainings), teachers for practical sessions, lectures and Supervisors.
Special Focus has been laid with the question if and how optimisation of teaching processes can take pressure of programme participants while simultaneously providing a targeted preparation for their future work in counselling. Encompassing systematic data analysis resulted in a Brief evaluation report including recommendations for programme development.
Subsequently, a questionnaire for ongoing Quality assurance of both programmes was developed which took results from previously conducted literature Reviews and qualitative interviews into consideration. Additionally, CEval developed a guideline which enabled SIAP to collect and Aanalyse data independently after the end of the evaluation project.
Evaluating the implementation of the Internet strategic concept “Country Web Portals as a Cross-cultural Internet Platform”
Object of the evaluation was the implementation of the
Internet strategy concept developed by the Goethe Institute (GI), known as Country Web Portals as a Cross-cultural Internet Platform. GI adopted this concept in 2011 and based its activities on it following the guiding principle that
– the personal experience of meeting and communicating through multimedia and online interaction is of importance,
– the platform offered online is characterised through (some) diversity and durability and
– users can participate and contribute content. Thus, the intercultural platforms target both the transfer of information as well as cross-cultural awareness and Exchange. The evaluation study examined whether the objectives of the country web portals have been achieved and the intended target groups have been reached. In addition, the analysis included the question on how effectively these web portals had enabled information transfer and promoted intercultural exchange. Also, the evaluation provided specific conclusions and recommendations derived from the collected information in order to further develop the concept or the country Portals. CEval followed a “mixed methods” approach with regard to data collection. As a result, existing documents and data derived from an online survey of the GI had been analyzed in detail. Additional surveys had also been conducted (qualitative guideline based interviews).
Process Evaluation of the Career Support Program ’Talente sichern & Zukunft gestalten’ (Securing Talents Shaping the Future)
Talente sichern Zukunft gestalten (Securing Talents – Shaping the Future) is a leadership development programme for young women run by Cusanuswerk – scholarship body of the Catholic Church in Germany. Mentoring and coaching by experienced academics and professionals as well as building a peer network constitute the main elements of this program.
Since its launch in 2007, the quality of the program had been assessed annually by means of external process evaluations conducted by the Center for Evaluation (CEval), and a similar evaluation had been carried out in the programs fourth year 2013-2014. This evaluation was designed to provide responsible members of staff at Cusanuswerk with information relevant to secure a successful performance adapted to the participants specific needs. Therefore, questions about overall satisfaction with offered courses and the quality as well as the professional and personal benefits of mentoring activities for both mentees and mentors were of particular interest.
The content of the existing and proven evaluation concept was sharpened as part of the evaluation by applying the CIPP (Context, Input, Process, Product) model, while ensuring comparability of evaluation results since the start of the career development program. Methodologically, the evaluation process was based on standardized online questionnaires, which mentees and mentors completed at three different stages of the program, the aim being to identify potential difficulties and appropriate solutions sufficiently early in the process.
Member Survey of IMMOEBS e.V.
IMMOEBS e.V. is the alumni and sponsor association of post graduate and master programs of real estate economics at the European Business School and the University of Regensburg. IMMOEBS was founded in 1991 and has about 2.600 members at the moment. It is the major alumni network of real estate economics in Germany.
CEval was commissioned by IMMOEBS with the conduction of a member survey. The general purpose of this online survey was the assessment of IMMOEBS and its activities by its members as well as the identification of potentials for development and adaptation requirements.