Process Evaluation of the Career Support Program ’Talente sichern & Zukunft gestalten’ (Securing Talents Shaping the Future)
Talente sichern Zukunft gestalten (Securing Talents – Shaping the Future) is a leadership development programme for young women run by Cusanuswerk – scholarship body of the Catholic Church in Germany. Mentoring and coaching by experienced academics and professionals as well as building a peer network constitute the main elements of this program.
Since its launch in 2007, the quality of the program had been assessed annually by means of external process evaluations conducted by the Center for Evaluation (CEval), and a similar evaluation had been carried out in the programs fourth year 2013-2014. This evaluation was designed to provide responsible members of staff at Cusanuswerk with information relevant to secure a successful performance adapted to the participants specific needs. Therefore, questions about overall satisfaction with offered courses and the quality as well as the professional and personal benefits of mentoring activities for both mentees and mentors were of particular interest.
The content of the existing and proven evaluation concept was sharpened as part of the evaluation by applying the CIPP (Context, Input, Process, Product) model, while ensuring comparability of evaluation results since the start of the career development program. Methodologically, the evaluation process was based on standardized online questionnaires, which mentees and mentors completed at three different stages of the program, the aim being to identify potential difficulties and appropriate solutions sufficiently early in the process.
Member Survey of IMMOEBS e.V.
IMMOEBS e.V. is the alumni and sponsor association of post graduate and master programs of real estate economics at the European Business School and the University of Regensburg. IMMOEBS was founded in 1991 and has about 2.600 members at the moment. It is the major alumni network of real estate economics in Germany.
CEval was commissioned by IMMOEBS with the conduction of a member survey. The general purpose of this online survey was the assessment of IMMOEBS and its activities by its members as well as the identification of potentials for development and adaptation requirements.
University Cooperation with Universidad de Costa Rica (UCR), with Funding Provided by the DAAD Program Subject-Specific Partnerships with Universities in Developing Countries
Having completed a successful 3-year cooperation focusing on the further development of the postgraduate study programme Project and Programme Evaluation for Social Development established at Universidad de Costa Rica (UCR) in 1995, the Center for Evaluation (CEval) and the Universidad de Costa Rica (UCR) identified and realised new areas of cooperation. Since 2008, the German Academic Exchange Service (DAAD) has supported this cooperation with its funding project for sending German academics to universities in developing countries. Since 2012, the DAAD has also provided funding through the programme Subject-Specific Partnerships with Universities in Developing Countries (2012-2015), financially ensuring the plans to enhance the partnership between the two universities in Germany and Costa Rica.
The goal of the DAAD programme was to improve the teaching situation and to set up sustainable teaching structures in developing countries by facilitating academic exchange. Therefore, the international exchange between students, PhD candidates and lecturers was one of the most important aspects in the context of this enhanced cooperation. The focus also laid on opening up the master programme to other Central-American areas (regionalisation), embedding the partnership into an existing, international network (internationalisation), and adapting teaching of the master programme at UCR to regional demands and ensuing educational needs (creating a distinct programme profile). The four year partnership helped establishing or expanding regional and international professional networks and developing additional capacities regarding teaching and research.
It is planned to continue with the partnership in 2017 and integrate another Latinamerican university into the cooperation.
Evaluation of PROFIN – Programme to Support the Integration of International Students
The evaluated project “PROFIN – Programme to Support the Integration of International Students” funded by the German Federal Ministry of Education and Research was realised by the DAAD – German Academic Exchange Service from 2009 to 2012. The DAAD implemented PROFIN in order to improve General, academic and social integration of international students in their host universities and in Society through different Projects at universities. PROFIN also aimed at reaching a paradigm shift in Relations between international students and German universities: moving from supervision to integration.
This Evaluation analysed if the defined programme Goals were achieved and whiche Projects and measures were particularly successful and effective.
CEval applied a mixed method approach to collect relevant data: an Analysis of programme documents was combined with data based on qualitative guideline-based interviews and standarised online surveys.
DAAD scholarship program ’Deutsche Auslandsschulen’
The DAAD scholarship program Deutsche Auslandsschulen (German schools abroad) exists since 2001; since 2008, it is integrated into the initiative Schulen: Partner der Zukunft (Schools: Partners for the future PASCH) of the Federal Foreign Office. The programme aims at offering the most talented and high-performing graduates from German schools abroad (including, since 2004, so-called Sprachdiplomschulen and since 2010, so-called Fit schools) scholarship-funded academic education prospects in Germany immediately after graduation, thereby tying them to the German academic landscape and economic market. In so doing, it is hoped that the scholarship holders and program alumni will act as multipliers for German and bilateral concerns in science, politics, culture and the economy.
Besides assessing the degree to which the program has achieved its objectives, the evaluation commissioned by DAAD addressed the appraisal of key programme parameters and program implementation. In this context, another key issue that had been addressed relates to the question to what extent the scholarship program was integrated into the PASCH initiative. Based on the evaluation results, recommendations for further program development and its integration with other DAAD and PASCH measures had been elaborated.
For data acquisition, CEval GmbH followed a mixed-methods approach, combining the analysis of program documents and statistics with a series of in-depth stakeholder interviews, several standardized online surveys and participant observation.
External auditing as part of the awarding of the quality seal “Fan project in accordance with the National Concept of Sport and Safety”.
In Germany, there is a dense nationwide network of social work facilities for young soccer fans in the form of fan projects, which work in accordance with the framework and specifications of the National Concept for Sport and Safety (NKSS). For the purposes of quality assurance and continuous improvement of the work of the fan projects, the fan project quality seal was introduced in 2010 in accordance with the NKSS. This is intended to contribute to an improved perception of the fan projects and to provide the fan projects with certainty of action. The quality seal was included in the revised NKSS in 2012 and is thus recognized at a high political level. The quality seal is awarded by the Quality Assurance Working Group, in which stakeholders from soccer, the highest state and family authorities, academia and fan projects are represented.
The award procedure for the quality seal should be revised from both a content and procedural perspective in order to create a seal that is a valid sign of the goodness and quality of a project. The award procedure should be as objective as possible and ensure transparency about the reasons for awarding or not awarding the seal. In the context of the revision of the seal, it was therefore the task of the Center for Evaluation (CEval) to develop a criteria catalog as well as a standardized survey questionnaire. In addition, the CEval, in consultation with the partners, developed a concept for the implementation of a pilot phase in which the revised quality seal and its awarding process were tested for their practical suitability. The pilot phase was implemented with five selected fan projects. Based on the findings and experiences of the pilot phase, the concept and the instruments were finalized.
Evaluation of the project ‘AnschlussDirekt’
The subject of the evaluation was the project AnschlussDirekt, initiated in 2010 by the Ministry of Economics and Science of Saarland and implemented over a period of three years. Within the scope of the project, pupils with a positive prognosis of a secondary school leaving certificate were intensively supported in the phase of vocational orientation and during the search for a training place. The support of the young people was primarily provided by the staff of the project coordination office, but also – and this is the special feature of the model project – by volunteer sponsors (economic experts and human resources managers of Saarland companies). The aim was to increase the proportion of pupils who start in-company vocational training immediately after completing secondary school.
The task of the accompanying evaluation was initially to support the development of a monitoring system in which data relevant to project management can be recorded and retrieved as required. In addition, the sponsors were asked about their impressions and evaluations of the project and their consultations within the framework of a telephone survey.
Furthermore, the task of the evaluation was to record project effects at the level of the target group. To this end, standardized surveys were conducted with the young people involved in the project at various points in time. The results of these surveys were systematically compared with specially collected baseline data from pupils not participating in the project.
Regular recommendations for project optimization were derived from the evaluation findings and made available to project coordination.
Ex-post evaluation of the regional development program ”La Malacateca” in Guatemala
From October 1998 till September 2012, World Vision implemented a regional development program in the municipality of Malacatán in Guatemala. The program adressed four key areas: health, education, economic development and sponsorships.
Three years after the conclusion of the program, World Vision wants to conduct an ex-post evaluation to appraise the sustainability of ist effects. To this end, the Center for Evaluation works with a local Guatemalan consultancy firm (CONSULTA) on the implementation of a quantitative household survey (including a control group) and qualitative interviews with the target group and relevant key informants.
Evaluation of the African Good Governance Networks (AGGN)
Young Africans educated at German universities are regarded by the DAAD as important resource persons who are able to play a key role in processes of development towards rule of law, democracy and economic prosperity after their return to their home countries. The African Good Governance Network (AGGN) was established at the beginning of 2007 under the patronage of Germany’s Federal President. This network is intended to contribute to the expansion and deepening of academic cooperation between Sub-Sahara Africa and Germany and at the same time to promote future African leaders in the field of good governance. The members of the AGGN support the principles of good governance, recognise them and actively disseminate them.
The AGGN can be understood as a long-term training programme in which up to ten new members are admitted annually. The network currently has 32 members from fourteen African countries who meet up to three times a year for workshops and seminars. These seminars deal with various theoretical and practical aspects of good governance as well as various personal skills. The aim is to raise awareness and understanding of good governance and the associated social attitude. The task of the evaluation was to evaluate the results and effects achieved so far and, on this basis, to derive recommendations for action for the design of further programme activities. The evaluation also takes the further development of the network and its potential into account.
The evaluation was methodically based on secondary and document analyses, a standardised online survey of all AGGN members and participation in one of the AGGN seminars in South Africa, in the context of which the results of the standardised survey were discussed with the AGGN members themselves.
Development of a concept for the evaluation of e-learning offers within the framework of VISU (Virtual Saar University)
On behalf of Prof. Dr. Reinhard Daugs, head of the VISU (Virtual Saarland University) project, a comprehensive concept was developed for the evaluation of Internet-based university courses. The aim was to provide all disciplines and chairs at Saarland University with a suitable set of tools for evaluating online courses across all phases of planning, development and implementation (provider side) and application (user side).
