Evaluation of §§20a and b Police Act of North Rhine-Westphalia (NRW)
Purpose of this project was the scientific support of the evaluation of §§ 20a and b in the North Rhine-Westphalian Police Act which became operative as of 1 of July 2013. It concerned the entitlement of receiving information on telecommunication and media data and included further a norm on data collection by the police with technical means. The new legislation has included the evaluation of impacts and a review of the application in practice with the support of an independent scientific expert after three years of experience. The balance between entitling the police with certain rights to inquire and collect data to protect citizens and at the same time limit the intrusion into private data has been a highly discussed topic in politics and public.
CEval was involved in the concretization and operationalization of leading research questions and in the identification of appropriate indicators. It supported the development of the evaluation design, the data collection instruments as well as data collection itself and data analysis. Data was collected in the form of guideline based interviews, focus group discussions and standardized surveys.
Framework contract for developing and implementing effect-oriented monitoring systems for DAAD programs (2016)
The aim of this project was to empower the DAAD to be able to cope with the increasing challenges in terms of results-based monitoring and control systems. Orienting itself more strongly to actual developmental results and ensuring a stronger focus on its partners’ needs (managing for results), the DAAD, as mediating party, was depending on evidence-based planning, controlling, and reporting of project performing parties (i.e., universities). The monitoring systems that had been implemented in this project was considered to increase the quality of information on program activities – in the context of project performing parties’ accounting as well within interactions between project parties in general (i.e., funding organization, DAAD, and university). Thereby, future monitoring systems promote collaborative learning and strategic development of DAAD’s funding programs.
In this project, CEval was mainly working in the following areas: developing of specific results models and indicator catalogs for existing as well as for future funding programs; adapting existing monitoring processes, procedures, and instruments for project planning, project controlling, and project reporting within DAAD’s funding programs; development of program-specific monitoring concepts and their realization. Moreover, CEval provided training on results-based planning and monitoring (capacity building) for employees of DAAD and of involved universities.
Evaluation of the Initiative “Schulen – Partner der Zukunft” (PASCH)
The initiative Schools Partners of the future (PASCH), founded in 2008, is part of the central initiatives of the German foreign cultural and educational policy within the current legislature period. PASCH is a worldwide network currently including around 1800 partner schools, which are linked to Germany and especially teach German. The initiative concentrates diversified measures, which are supposed to contribute to establish lively and long-term relations to Germany, and encourage schools, teachers and students to exchange their knowledge and ideas as well as to work together. PASCH is coordinated by the German Federal Foreign Office and is implemented by the Central Office for Schools Abroad, the Goethe Institute, the Pedagogic Exchange Service, the Conference of Education Ministers and the German Academic Exchange Service.
Within the scope of the planned evaluation, PASCH will be analyzed regarding its relevance, efficiency, effectivity, impact and sustainability. It will also be analyzed to what extent the existing offers, processes and structures are suitable instruments for achieving the objectives. Recommendations for possible optimization of the measures, processes and structures and the creation of synergies will be derived on the basis of these results.
The evaluation is divided into two phases: During the first phase, document analyses as well as guided personal and telephone interviews with stakeholders in Germany and selected partner countries (Poland, Turkey and China) will be conducted. The focus of the second phase is then on quantitative online surveys with target groups in China, Turkey and Poland as well as on qualitative and quantitative data collection in another case study country.
Evaluation of the project ‘Media in Libya: Stability through reconciliation’
The object of this evaluation was the project ‘Media in Libya: Stability through reconciliation’ (MLSR) of the DW Akademie. MLSR was a follow-up of the precursor project Media in Libya: Stability through Structure. The overall goal of MLSR was to enable a process of public discourse and eventually reconciliation across a fragmented Libyan society prerequisites for development of democratic reform and constitutional process in the country. By promoting [the] right to access to information to the people of Libya and [the] freedom of expression in Libya (DW Akademie, 2017, p. 4). The approach of MLSR is fourfold (i.e., four activity clusters): (1) supporting Libyan institutions/authorities efforts in the field of media governance, (2) developing a Libyan Media Network, (3) training local freelance journalists from all regions of Libya with a special focus on ethnic minorities and gender, and (4) providing grants for Libyan media projects.
The evaluation of MLSR had been considered as a planned impact evaluation, which needs valid qualitative/quantitative data on the situation at the beginning of MLSR (i.e., baseline). The second stage of evaluation (i.e., interim evaluation) started in January 2019. Comparing the results of baseline and interim evaluation, the evaluators had been able to identify and assess the preliminary effects of MLSR. Moreover, the evaluators may give first recommendations on how to improve MLSR (e.g., identification of [un]successful project strategies, alternative actions). The third stage of evaluation started shortly after the end of MLSR (i.e., January 2021). The main goal of the final evaluation was to assess the effectiveness of MLSR (i.e., achievements in relevant targeted areas).
Methodologically, a robust mixed-method approach was pursued against the background of the very volatile and difficult framework conditions in Libya: Secondary analysis of existing documents/data, content analysis of short written and in-depth oral interviews and statistical analysis of quantitative data from a standardized questionnaire survey.
Evaluation of the 2014 Staff/Management Talks at Saarstahl AG
In 2013, Saarstahl AG Management introduced Performance appraisals between Supervisors and their Managers (Mitarbeitergespraeche, short: MAGs) to the company’s human resource Management. These annual talks take place in private, Independent of any particular incident or Event and on a non-hierarchical Basis. The aim of the MAGs at Saarstahl AG is to improve orientation towards the company’s targets and its corporate culture, to promote employee Training and development and to improve cooperation within the Company.
The evaluation of the 2014 MAGs assessed if they were well-received among employees as well as Management in their second year and if Trends concerning acceptance and implementation could be identified.
Special focus has been laid with the following aspects: preparation of employees(seminars, Information material); implementation and communication of target Agreements as well as employee Support and development strategies, General manageability of this tool regarding Content and structure.
Additionally, the study evalated if and how MAGs can be considered as an appropriate tool in human resource Management and if they triggered improvements in Saarstahl AG managerial culture. The aim was to evaluate MAGs in their present form and to identify potentials for optimisation.
In close cooperation with Saarstahl AG, CEval developed and contucted a standardised online Survey among employees at all levels of the Company. Quantitative data of this year’s Evaluation was complemented by data collected in the 2013 MAGs study in order to allow a longitudinal Analysis.
Accompanying evaluation of the project ‘Media in Libya – Stability through Structure"
The „Media in Libya- Stability through Structure“ project had the main goal of supporting the introduction of modern media legislation in Libya and to strengthen the countrys media institutions. In addition, the project should contribute to strengthen independent and professional journalism in the provinces of Libya. The project had a total duration of 30 months and was funded by the European Commission within the SPRING program. The project was controlled and implemented by the Deutsche Welle Academy (Department of Media Development, Team Africa) in cooperation with the Institute for War and Peace Reporting (IWPR). The activities of the project were divided into four activity clusters, of which DW Academy three realised, while IWPR edited one. CEval evaluated in this project clusters 1 to 3, which were in responsibility of the DW Academy.
The evaluation of the project was divided into a baseline, an intermediate and a final evaluation at the end of the implementation phase. CEval followed a mixed-methods approach for data mining. An in-depth analysis of already existing documents was carried out as well as own surveys (standardized online surveys of the participants of the trainings and qualitative guideline interviews with external experts and officials DW Academy).
Tracer Study of the Young Talent Scholarship Program of the Heinrich-Böll-Foundation
The Heinrich-Boell-Foundation granted since 1990 approximately 3000 scholarships to students and doctoral students of all subjects and nationalities per year. These stipend holders pursued their degree at universities, universities of applied sciences (‘Fachhochschulen’), or universities of the arts (‘Kunsthochschulen’). It is expected by the Heinrich-Boell-Foundation that scholarship recipients have excellent academic records, are socially and politically engaged, and have an active interest in the basic values of the foundation, basically ecology and sustainability, democracy and human rights, self-determination and justice.
The CEval conducted a tracer study of the promotion of young talent scholarship programme in order to evaluate its long-term impact. The impact of the scholarship programme for the target population was analyzed with a special focus on gender, diversity and the different sponsorship and conveying efforts of the foundation. The CEval followed a “mixed methods” approach with regard to data collection. As a result, existing data and documents were analyzed in detail and data derived from a quantitative online-survey with all previous stipend holders of the hbs was assessed. Qualitative semi-structured interviews with alumni and project managers of the hbs were conducted and form the qualitative part of the analysis.
Accompanying evaluation of the career support program ‘Securing talents shaping the future’ (2015-2016)
The program Securing talents shaping the future by the Cusanuswerk aims to support future female leaders. The program focuses on mentoring elements that is tutorial support for scholarship holders by professionally experienced academics. It also focuses on occupationally specific training and support measures as well as on building a peer-network.
The Center for Evaluation (CEval) has been conducting accompanying external evaluations to ensure quality management since the start of the career support program in 2007.
An accompanying evaluation will also take place during the fifth program run in 2015/16. The main task of this evaluation is to provide the responsible employees with management-related information. In this context, questions regarding satisfaction with the offered events, quality of tutorial relationships as well as questions regarding professional and personal use of the relationships for both sides are especially relevant.
The evaluation concept is based on Stufflebeams CIPP model and comparability of evaluation results regarding different program runs since the start of the career support program has been ensured. In regards to methods, the evaluation covers standardized questionnaires of mentees and mentors compared at three points in time. We will also conduct an alumni-survey of mentees that have been supported thus far.
Evaluation of the Baden-Württemberg-STIPENDIUM for University Students BWS plus
The Baden-Württemberg Stiftung has awarded the Baden-Württemberg-STIPENDIUM for more than ten years. Each year, up to 1,400 students of all nationalities receive a scholarship for Placements abroad. With the Baden-Württemberg-STIPENDIUM for University Students – BWS plus, the Baden-Württemberg Stiftung supports innovative cooperation of Baden-Württemberg universities. The aim is to establish and expand new relationships with foreign universities and the Lasting intensification of existing contacts.
CEval is commissioned to assess the degree to which the programme fulfils stated objectives. It distinguishes between objectives in term of the programme (e.g. expanding the international network, promoting innovative ideas of students) and in Terms of structure (e.g. improving the awareness of the scholarship programs and of the Baden-Württemberg Stiftung).
CEval employs a mixed-methods approach with regard to data collection. Therefore, relevant documents and accessible data will be analysed. Data will be collected using a standardized online Survey. In addition, a qualitative approach will be carried out by conducting semi-structured interviews with project participants.
East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.