How to manage sustainability and effectiveness?
The actors of German cultural relations and education policy (AKBP) face the challenge that the context of their actions is always in interaction with complex, dynamic political and social contexts. This complicates the creation of knowledge in order to manage projects and thus to secure sustainability and effectiveness of AKBP-projects.
Hence, the aim of this research project was to analyze which methods and tools are suitable in volatile contexts to generate knowledge about effects in changing conditions. In addition, the possibilities to use this knowledge for project management was examined. Therefore, on the one hand a stock taking of existing instruments was made. On the other hand more recent approaches for monitoring and decision-making were analyzed with regard to their applicability in the AKBP.
Evaluation of §§20a and b Police Act of North Rhine-Westphalia (NRW)
Purpose of this project was the scientific support of the evaluation of §§ 20a and b in the North Rhine-Westphalian Police Act which became operative as of 1 of July 2013. It concerned the entitlement of receiving information on telecommunication and media data and included further a norm on data collection by the police with technical means. The new legislation has included the evaluation of impacts and a review of the application in practice with the support of an independent scientific expert after three years of experience. The balance between entitling the police with certain rights to inquire and collect data to protect citizens and at the same time limit the intrusion into private data has been a highly discussed topic in politics and public.
CEval was involved in the concretization and operationalization of leading research questions and in the identification of appropriate indicators. It supported the development of the evaluation design, the data collection instruments as well as data collection itself and data analysis. Data was collected in the form of guideline based interviews, focus group discussions and standardized surveys.
Framework contract for developing and implementing effect-oriented monitoring systems for DAAD programs (2016)
The aim of this project was to empower the DAAD to be able to cope with the increasing challenges in terms of results-based monitoring and control systems. Orienting itself more strongly to actual developmental results and ensuring a stronger focus on its partners’ needs (managing for results), the DAAD, as mediating party, was depending on evidence-based planning, controlling, and reporting of project performing parties (i.e., universities). The monitoring systems that had been implemented in this project was considered to increase the quality of information on program activities – in the context of project performing parties’ accounting as well within interactions between project parties in general (i.e., funding organization, DAAD, and university). Thereby, future monitoring systems promote collaborative learning and strategic development of DAAD’s funding programs.
In this project, CEval was mainly working in the following areas: developing of specific results models and indicator catalogs for existing as well as for future funding programs; adapting existing monitoring processes, procedures, and instruments for project planning, project controlling, and project reporting within DAAD’s funding programs; development of program-specific monitoring concepts and their realization. Moreover, CEval provided training on results-based planning and monitoring (capacity building) for employees of DAAD and of involved universities.
Evaluation of the 2014 Staff/Management Talks at Saarstahl AG
In 2013, Saarstahl AG Management introduced Performance appraisals between Supervisors and their Managers (Mitarbeitergespraeche, short: MAGs) to the company’s human resource Management. These annual talks take place in private, Independent of any particular incident or Event and on a non-hierarchical Basis. The aim of the MAGs at Saarstahl AG is to improve orientation towards the company’s targets and its corporate culture, to promote employee Training and development and to improve cooperation within the Company.
The evaluation of the 2014 MAGs assessed if they were well-received among employees as well as Management in their second year and if Trends concerning acceptance and implementation could be identified.
Special focus has been laid with the following aspects: preparation of employees(seminars, Information material); implementation and communication of target Agreements as well as employee Support and development strategies, General manageability of this tool regarding Content and structure.
Additionally, the study evalated if and how MAGs can be considered as an appropriate tool in human resource Management and if they triggered improvements in Saarstahl AG managerial culture. The aim was to evaluate MAGs in their present form and to identify potentials for optimisation.
In close cooperation with Saarstahl AG, CEval developed and contucted a standardised online Survey among employees at all levels of the Company. Quantitative data of this year’s Evaluation was complemented by data collected in the 2013 MAGs study in order to allow a longitudinal Analysis.
Accompanying evaluation of the project ‘Media in Libya – Stability through Structure"
The „Media in Libya- Stability through Structure“ project had the main goal of supporting the introduction of modern media legislation in Libya and to strengthen the countrys media institutions. In addition, the project should contribute to strengthen independent and professional journalism in the provinces of Libya. The project had a total duration of 30 months and was funded by the European Commission within the SPRING program. The project was controlled and implemented by the Deutsche Welle Academy (Department of Media Development, Team Africa) in cooperation with the Institute for War and Peace Reporting (IWPR). The activities of the project were divided into four activity clusters, of which DW Academy three realised, while IWPR edited one. CEval evaluated in this project clusters 1 to 3, which were in responsibility of the DW Academy.
The evaluation of the project was divided into a baseline, an intermediate and a final evaluation at the end of the implementation phase. CEval followed a mixed-methods approach for data mining. An in-depth analysis of already existing documents was carried out as well as own surveys (standardized online surveys of the participants of the trainings and qualitative guideline interviews with external experts and officials DW Academy).
Tracer Study of the Young Talent Scholarship Program of the Heinrich-Böll-Foundation
The Heinrich-Boell-Foundation granted since 1990 approximately 3000 scholarships to students and doctoral students of all subjects and nationalities per year. These stipend holders pursued their degree at universities, universities of applied sciences (‘Fachhochschulen’), or universities of the arts (‘Kunsthochschulen’). It is expected by the Heinrich-Boell-Foundation that scholarship recipients have excellent academic records, are socially and politically engaged, and have an active interest in the basic values of the foundation, basically ecology and sustainability, democracy and human rights, self-determination and justice.
The CEval conducted a tracer study of the promotion of young talent scholarship programme in order to evaluate its long-term impact. The impact of the scholarship programme for the target population was analyzed with a special focus on gender, diversity and the different sponsorship and conveying efforts of the foundation. The CEval followed a “mixed methods” approach with regard to data collection. As a result, existing data and documents were analyzed in detail and data derived from a quantitative online-survey with all previous stipend holders of the hbs was assessed. Qualitative semi-structured interviews with alumni and project managers of the hbs were conducted and form the qualitative part of the analysis.
Accompanying evaluation of the career support program ‘Securing talents shaping the future’
The program Securing talents shaping the future by the Cusanuswerk aims to support future female leaders. The program focuses on mentoring elements that is tutorial support for scholarship holders by professionally experienced academics. It also focuses on occupationally specific training and support measures as well as on building a peer-network.
The Center for Evaluation (CEval) has been conducting accompanying external evaluations to ensure quality management since the start of the career support program in 2007.
An accompanying evaluation will also take place during the fifth program run in 2015/16. The main task of this evaluation is to provide the responsible employees with management-related information. In this context, questions regarding satisfaction with the offered events, quality of tutorial relationships as well as questions regarding professional and personal use of the relationships for both sides are especially relevant.
The evaluation concept is based on Stufflebeams CIPP model and comparability of evaluation results regarding different program runs since the start of the career support program has been ensured. In regards to methods, the evaluation covers standardized questionnaires of mentees and mentors compared at three points in time. We will also conduct an alumni-survey of mentees that have been supported thus far.
Evaluation of the Baden-Württemberg-STIPENDIUM for University Students BWS plus
The Baden-Württemberg Stiftung has awarded the Baden-Württemberg-STIPENDIUM for more than ten years. Each year, up to 1,400 students of all nationalities receive a scholarship for Placements abroad. With the Baden-Württemberg-STIPENDIUM for University Students – BWS plus, the Baden-Württemberg Stiftung supports innovative cooperation of Baden-Württemberg universities. The aim is to establish and expand new relationships with foreign universities and the Lasting intensification of existing contacts.
CEval is commissioned to assess the degree to which the programme fulfils stated objectives. It distinguishes between objectives in term of the programme (e.g. expanding the international network, promoting innovative ideas of students) and in Terms of structure (e.g. improving the awareness of the scholarship programs and of the Baden-Württemberg Stiftung).
CEval employs a mixed-methods approach with regard to data collection. Therefore, relevant documents and accessible data will be analysed. Data will be collected using a standardized online Survey. In addition, a qualitative approach will be carried out by conducting semi-structured interviews with project participants.
Member Survey of IMMOEBS e.V.
IMMOEBS e.V. is the alumni and sponsor association of post graduate and master programs of real estate economics at the European Business School and the University of Regensburg. IMMOEBS was founded in 1991 and has about 2.600 members at the moment. It is the major alumni network of real estate economics in Germany.
CEval was commissioned by IMMOEBS with the conduction of a member survey. The general purpose of this online survey was the assessment of IMMOEBS and its activities by its members as well as the identification of potentials for development and adaptation requirements.
East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.