East4South – Development Awareness from the Roots
Subject of the evaluation was the EU-funded project of the Deutsche Welle Academy (DWA) East4South – Development Awareness from the Roots”. The projects overall goal was to raise awareness about development issues among EU-12 citizens and positively changing their attitude towards EU development policy. The project aimed specifically at enabling the EU12 journalists to prepare high quality media products covering development issues and to publish them in their home countries media by providing training in intercultural journalism and on-site research and a media production stay in Africa for young journalists from EU-12 countries. In the course of the project a total of 60 young journalists from Eastern Europe and Africa were fostered and supported when creating their own network.
Goal of the summative ex-post evaluation was to verify and analyze the outcome of the project activities and the quality of the project management. The CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews and secondary and document analysis of module evaluation surveys and participants short reports with a standardized online survey of all participants.Talking about migrants lives in Russia
Subject of Evaluation was the EU-funded project Talking about migrants lives in Russia of the DW Akademie and the Foundation for Independent Radio Broadcasting. The project aimed to increase awareness about the situation of migrants within Russian society by strengthening cooperation between the Russian media and civil society organizations (CSOs) who are working on migrant issues. In order to achieve this goal, training activities for radio journalists and CSO activists were provided. The workshops aimed to introduce the CSO activists to agenda-setting mechanism of the media and to equip the journalists with storytelling skills and in-the-field reporting skills in order to do genuine social reporting on migration issues. Following the training activities the workshop participants took part in panel discussions where they presented their media productions and exchanged experiences on the reporting on migrant issues. A major medium-term outcome of those activities was the establishment of a sustainable network between CSO activists and radio journalists.
Goal of the summative ex-post evaluation was to analyze and assess the short- and medium-term outcomes of the training workshops. For this purpose the evaluation additionally analyzed to what extent the projects web platform, the panel discussions and organizational aspects in general influenced the expected outcomes. For data acquisition, CEval GmbH followed a mixed-method approach combining in-depth stakeholder interviews, document as well as secondary analysis of module evaluation surveys. Additionally a standardized online survey in Russian of all workshop participants was carried out.Strengthening evaluation capacities within government and civil society in Uganda (ECD Uganda): Development of a blended-learning Master of Evaluation (2012-2015)
Evaluation as a scientific means to describe and assess interventions is gaining relevance worldwide. The needs for qualified evaluation professionals are increasing in Uganda, too. However, at the moment these needs cannot be satisfied due to lack of training- and evaluation programs in this region. It is for this reason that the BMZ together with CEval and GIZ initiated a program for developing evaluation capacities in Uganda.
The most important component of this program is the development and implementation of a master course in evaluation in the blended-learning format at Uganda Technology and Management University (UTAMU) in Kampala, Uganda. Blended-learning is an innovative concept which is gaining popularity worldwide and is now the default means of teaching at UTAMU. Blended-learning unites traditional face-to-face classes with modern forms of e-learning in a didactical coherent way. An advantage of this concept is, that it is possible not only for Ugandan students to attend classes but also for all students of English-speaking African countries.
The Master of Evaluation of Saarland University in cooperation with HTW Saarland serves as a template for the Ugandan Master, which will be introduced in fall 2015. The contents of the master program have been developed in cooperation with CEval GmbH, the teachers of the German master as well as Ugandan contact persons of Makarere University and have been adapted to the Ugandan context. The Ugandan Master of Monitoring and Evaluation will comprise 15 modules in total, seven of which are in compliance with the basic teaching program of evaluation at Saarland universities. This is to ensure that all topics relevant to acquire profound knowledge in the field of evaluation are covered. UTAMU has complemented this program with further modules regarding project management, monitoring as well as business start-up consulting. The modules are roughly comprised of web-based trainings, e-tutoring and face-to-face seminars. The teaching concepts have also been developed in cooperation with all stakeholders. The GIZ is responsible for the technical implementation of the e-learning concepts.
Two modules of the master of evaluation have been tested in an experimental phase together with UTAMU. During this experimental phase e-tutoring as well as the face-to-face seminars have been supervised in the tandem-principle by teachers of UTAMU and Saarland University/HTW. The hosting of the online study course has been undertaken by the GIZ platform GC 21. In a next step, the seven modules have been integrated into the already existing UTAMU master and the new course of study has been submitted to the responsible committees. The course of study is now aggregated and will be introduced experimentally. A decision about whether the course of study will be continued and which modifications would be necessary for this procedure, will be made in December 2016.
University Cooperation with Universidad de Costa Rica (UCR), with Funding Provided by the DAAD Program Subject-Specific Partnerships with Universities in Developing Countries
Having completed a successful 3-year cooperation focusing on the further development of the postgraduate study programme Project and Programme Evaluation for Social Development established at Universidad de Costa Rica (UCR) in 1995, the Center for Evaluation (CEval) and the Universidad de Costa Rica (UCR) identified and realised new areas of cooperation. Since 2008, the German Academic Exchange Service (DAAD) has supported this cooperation with its funding project for sending German academics to universities in developing countries. Since 2012, the DAAD has also provided funding through the programme Subject-Specific Partnerships with Universities in Developing Countries (2012-2015), financially ensuring the plans to enhance the partnership between the two universities in Germany and Costa Rica.
The goal of the DAAD programme was to improve the teaching situation and to set up sustainable teaching structures in developing countries by facilitating academic exchange. Therefore, the international exchange between students, PhD candidates and lecturers was one of the most important aspects in the context of this enhanced cooperation. The focus also laid on opening up the master programme to other Central-American areas (regionalisation), embedding the partnership into an existing, international network (internationalisation), and adapting teaching of the master programme at UCR to regional demands and ensuing educational needs (creating a distinct programme profile). The four year partnership helped establishing or expanding regional and international professional networks and developing additional capacities regarding teaching and research.
It is planned to continue with the partnership in 2017 and integrate another Latinamerican university into the cooperation.
Evaluation of PROFIN – Programme to Support the Integration of International Students
The evaluated project “PROFIN – Programme to Support the Integration of International Students” funded by the German Federal Ministry of Education and Research was realised by the DAAD – German Academic Exchange Service from 2009 to 2012. The DAAD implemented PROFIN in order to improve General, academic and social integration of international students in their host universities and in Society through different Projects at universities. PROFIN also aimed at reaching a paradigm shift in Relations between international students and German universities: moving from supervision to integration.
This Evaluation analysed if the defined programme Goals were achieved and whiche Projects and measures were particularly successful and effective.
CEval applied a mixed method approach to collect relevant data: an Analysis of programme documents was combined with data based on qualitative guideline-based interviews and standarised online surveys.
DAAD scholarship program ’Deutsche Auslandsschulen’
The DAAD scholarship program Deutsche Auslandsschulen (German schools abroad) exists since 2001; since 2008, it is integrated into the initiative Schulen: Partner der Zukunft (Schools: Partners for the future PASCH) of the Federal Foreign Office. The programme aims at offering the most talented and high-performing graduates from German schools abroad (including, since 2004, so-called Sprachdiplomschulen and since 2010, so-called Fit schools) scholarship-funded academic education prospects in Germany immediately after graduation, thereby tying them to the German academic landscape and economic market. In so doing, it is hoped that the scholarship holders and program alumni will act as multipliers for German and bilateral concerns in science, politics, culture and the economy.
Besides assessing the degree to which the program has achieved its objectives, the evaluation commissioned by DAAD addressed the appraisal of key programme parameters and program implementation. In this context, another key issue that had been addressed relates to the question to what extent the scholarship program was integrated into the PASCH initiative. Based on the evaluation results, recommendations for further program development and its integration with other DAAD and PASCH measures had been elaborated.
For data acquisition, CEval GmbH followed a mixed-methods approach, combining the analysis of program documents and statistics with a series of in-depth stakeholder interviews, several standardized online surveys and participant observation.
External auditing as part of the awarding of the quality seal “Fan project in accordance with the National Concept of Sport and Safety”.
In Germany, there is a dense nationwide network of social work facilities for young soccer fans in the form of fan projects, which work in accordance with the framework and specifications of the National Concept for Sport and Safety (NKSS). For the purposes of quality assurance and continuous improvement of the work of the fan projects, the fan project quality seal was introduced in 2010 in accordance with the NKSS. This is intended to contribute to an improved perception of the fan projects and to provide the fan projects with certainty of action. The quality seal was included in the revised NKSS in 2012 and is thus recognized at a high political level. The quality seal is awarded by the Quality Assurance Working Group, in which stakeholders from soccer, the highest state and family authorities, academia and fan projects are represented.
The award procedure for the quality seal should be revised from both a content and procedural perspective in order to create a seal that is a valid sign of the goodness and quality of a project. The award procedure should be as objective as possible and ensure transparency about the reasons for awarding or not awarding the seal. In the context of the revision of the seal, it was therefore the task of the Center for Evaluation (CEval) to develop a criteria catalog as well as a standardized survey questionnaire. In addition, the CEval, in consultation with the partners, developed a concept for the implementation of a pilot phase in which the revised quality seal and its awarding process were tested for their practical suitability. The pilot phase was implemented with five selected fan projects. Based on the findings and experiences of the pilot phase, the concept and the instruments were finalized.
Evaluation of the project ‘AnschlussDirekt’
The subject of the evaluation was the project AnschlussDirekt, initiated in 2010 by the Ministry of Economics and Science of Saarland and implemented over a period of three years. Within the scope of the project, pupils with a positive prognosis of a secondary school leaving certificate were intensively supported in the phase of vocational orientation and during the search for a training place. The support of the young people was primarily provided by the staff of the project coordination office, but also – and this is the special feature of the model project – by volunteer sponsors (economic experts and human resources managers of Saarland companies). The aim was to increase the proportion of pupils who start in-company vocational training immediately after completing secondary school.
The task of the accompanying evaluation was initially to support the development of a monitoring system in which data relevant to project management can be recorded and retrieved as required. In addition, the sponsors were asked about their impressions and evaluations of the project and their consultations within the framework of a telephone survey.
Furthermore, the task of the evaluation was to record project effects at the level of the target group. To this end, standardized surveys were conducted with the young people involved in the project at various points in time. The results of these surveys were systematically compared with specially collected baseline data from pupils not participating in the project.
Regular recommendations for project optimization were derived from the evaluation findings and made available to project coordination.
Ex-post evaluation of the regional development program ”La Malacateca” in Guatemala
From October 1998 till September 2012, World Vision implemented a regional development program in the municipality of Malacatán in Guatemala. The program adressed four key areas: health, education, economic development and sponsorships.
Three years after the conclusion of the program, World Vision wants to conduct an ex-post evaluation to appraise the sustainability of ist effects. To this end, the Center for Evaluation works with a local Guatemalan consultancy firm (CONSULTA) on the implementation of a quantitative household survey (including a control group) and qualitative interviews with the target group and relevant key informants.
Evaluation of the African Good Governance Networks (AGGN)
Young Africans educated at German universities are regarded by the DAAD as important resource persons who are able to play a key role in processes of development towards rule of law, democracy and economic prosperity after their return to their home countries. The African Good Governance Network (AGGN) was established at the beginning of 2007 under the patronage of Germany’s Federal President. This network is intended to contribute to the expansion and deepening of academic cooperation between Sub-Sahara Africa and Germany and at the same time to promote future African leaders in the field of good governance. The members of the AGGN support the principles of good governance, recognise them and actively disseminate them.
The AGGN can be understood as a long-term training programme in which up to ten new members are admitted annually. The network currently has 32 members from fourteen African countries who meet up to three times a year for workshops and seminars. These seminars deal with various theoretical and practical aspects of good governance as well as various personal skills. The aim is to raise awareness and understanding of good governance and the associated social attitude. The task of the evaluation was to evaluate the results and effects achieved so far and, on this basis, to derive recommendations for action for the design of further programme activities. The evaluation also takes the further development of the network and its potential into account.
The evaluation was methodically based on secondary and document analyses, a standardised online survey of all AGGN members and participation in one of the AGGN seminars in South Africa, in the context of which the results of the standardised survey were discussed with the AGGN members themselves.
