Support of the BfN in the Evaluation by the German Science and Humanities Council (WR) 2023/2024
As a federal departmental research institution, the Federal Agency for Nature Conservation (BfN) is evaluated at regular intervals by external experts. After the first external evaluation by the German Council of Science and Humanities (WR) in 2007 and the second evaluation in 2015, the third evaluation is now due in 2024. The aim of the project is to support the BfN in the conceptual preparation and follow-up of the upcoming evaluation process through external, scientific expertise.
The following core tasks will be fulfilled:
– Support in the collection, evaluation and preparation of data and information needed to answer the questions to the BfN and the preparation of the report;
– Support in the preparation and implementation of the inspection by the WR;
– Support in the analysis of the evaluation results and in the follow-up of the evaluation;
– Analysis of the effectiveness and quality of BfN’s science-based policy advice and knowledge communication, including the development of recommendations for its systematic further development. Development of proposals for the integration of these recommendations into the quality management of BfN’s scientific work.
Program evaluation of the federal program for biological diversity
The German National Strategy on biodiversity strives to improve the protection and sustainable use of biodiversity in Germany. The strategy was adopted by the federal government in 2007 and has been supplemented by the Nature Conservation Offensive 2020 since 2015.
Since 2011, the Federal Program for Biological Diversity has taken on a crucial role within the framework of the biodiversity strategy as Germany’s most financially powerful funding instrument for nature conservation projects. It promotes measures aimed at halting the decline in biodiversity in Germany and reversing it into a positive trend in the medium to long term.
To assess its success, the federal programme has established evaluations of individual projects. However, a systematic and aggregated analysis of the pool of projects funded within the framework of the programme was missing in its 10-year duration.
Therefore, CEval was working with adelphi and PAN to develop a structured, precisely fitting analysis methodology to systematically assess the quality of the programme.
For this purpose, documents from more than 60 completed or far advanced projects were systematically evaluated and complemented by expert interviews. In addition, further interviews were conducted at the level of programme management in order to identify potentials for the improvement of the operational and strategic programme steering. Overall, key aspects, successes, weaknesses and potentials at programme and project level were identified and systematically reflected to formulate recommendations for the improvement of future funding practices.
External Evaluation of Climate-friendly Communal Catering within the Project “MehrWertKonsum” of the North Rhine-Westphalia Consumer Advice Centre
The project MehrWertKonsum More appreciation of food and finite resources conducted by the consumer advice centre of North Rhine-Westphalia aims to reduce carbon dioxide emissions encouraging a responsible, sustainable and climate-friendly consumption. The primary objective of the project is to contribute to saving greenhouse gas emissions. Regarding communal catering, among other things that means to diminish leftovers and to offer more vegetarian dishes with seasonal and regional foods that are organically grown. Information and educational material will be passed to day care centres, schools, youth hostels and their food suppliers, waste will be measured, leftovers on plates and at the service counters will be analysed and suggestions for improvement will be developed. Furthermore, menus will be checked and enhanced regarding to their desirability as well as climate and environment protection. Schools, in addition, will have the possibility to get participative educational services and activities for their students. In youth hostels, there will be a special focus on the communication with the guests to encourage the acceptance of the measures and thus to contribute to waste prevention. The project is funded through the EFRE OP NRW, an operational programme of North Rhine-Westphalia concerning the promotion of investments in growth and employment supported by the European Regional Development Fund, and North Rhine-Westphalias Ministry of Environment. Within EFRE OP NRW projects greenhouse gas savings (t CO2 equivalents/year) have to be substantiated. Moreover, the following objectives of the field of action climate-friendly communal catering should be evaluated:
– To reduce food and catering waste
– To establish climate-friendly and healthy catering offers
– To raise consciousness of and appreciation for food, climate-friendly nutrition and food waste
In the course of the external evaluation of MehrWertKonsum the impact of the project measures will be analysed both on the outcome level (in particular altered behaviour of the target group) as well as on the impact level (greenhouse gas savings). To do so, the evaluation team of CEval GmbH, schmidt evaluation and EBP applies a complex multi-method approach including participant observation, qualitative telephone interviews and quantitative online surveys. The results will be combined with the internal monitoring of the project MehrWertKonsum in a way that greenhouse gas savings can be calculated.
Evaluation of Individual Projects of the International Climate Initiative (IKI)
Since 2008, the International Climate Initiative (IKI) is funding initiatives regarding (i) climate protection, (ii) adaptation to the impacts of climate change, (iii) conserving natural carbon sinks and (iv) the protection of biological diversity in developing and newly industrialising countries on a project basis. Overall, the IKI now includes almost 600 individual projects and 2.8 billion euros in funding. The IKI focuses on climate protection which comprises almost half of the projects. There was a first evaluation conducted between 2010 and 2013 that included 115 projects. Within this evaluation, a full survey of the projects completed or almost completed by 2011 was realised. The current initiative aims at the evaluation of the further projects completed by the end of 2019. The individual project evaluations consist of detailed analyses on the basis of the OECDs DAC criteria as well as other project specific criteria. In terms of the impact chain, the focus lies on the analysis of the inputs, outputs and outcomes. Higher-level impacts are analysed on the basis of the reliable information available at the point of evaluation.
Results oriented monitoring of the technical cooperation measure COPLAN II and evaluation of the measure’s contribution towards a better alignment of development cooperation and foreign trade facilitation on a systemic level
Climate Change and the uninterrupted decline of fossil fuels affect different regions of the world in different ways. Many nations have thus formulated ambitious goals: Mexico for instance is determined to cover one third of its energy production by means of renewable energies by 2024. Despite such ambitions however environmental conservation has not been marked by major success to this date. This is partly due to a lack of dispersion of modern environmental technology. Against this background the GIZ Project Cooperation Platform Latin America North II (COPLAN II) scrutinizes central deficits that hinder international investment and cooperation such as a lack of professional expertise regarding modern environmental technology in small and medium enterprises (SMEs).
Within the scope of the present assignment CEval together with iSPO Institute support COPLAN II in its monitoring and evaluation efforts. Starting point for any results oriented monitoring is typically an impact model with respective indicators and assumptions. Changes within the mechanisms of cooperation are however rarely or marginally observed under the monitoring component. The developments of these cooperation systems as well as the feasibility of the projects instruments are hence at the center of the evaluation component.
All in all, next to monitoring tasks and the compilation of results of individual measures the assignment at hand intends to also generate insights into COPLAN IIs contribution towards a better alignment of development cooperation and foreign trade facilitation as well as public-private sector cooperation on a systemic level.
Scientific support of an impact analysis of the project “egal war gestern – towards sustainability at vocational schools”
The Landesbund für Vogelschutz in Bayern (LBV) e.V. implemented the project Towards sustainability at vocational schools in the school term 2018/19 for the second time. It aimed at mobilizing the youth for matters of sustainability. More precisely, the project was geared towards students at Bavarian vocational schools who find themselves in the important period of professional orientation. The goal was to direct their attention to sustainability questions and to strengthen their awareness of those issues. To achieve this goal, the students were expected to independently implement projects in their respective schools in order to contribute to a more sustainable school life.
The project was subject to an impact analysis, which was jointly implemented by LBV and CEval GmbH within the context of an accompanying evaluation. The purpose was to determine whether the intended effects of the project were actually achieved, that is, whether the attitudes and behavior of the participating students actually changed in relation to sustainability matters and whether those matters are more readily integrated into the everyday life at Bavarian vocational schools. In terms of methodology the analysis was based on a standardized survey before and after the project with both the participating students and a comparison group of students. Furthermore, guided interviews with the teachers who support the students in the implementation of their projects were conducted before and after the project.
Evaluation of hazard prevention and damage after the heavy rain events in Saarland in May/June 2018
In May and June 2018, Saarland was hit by heavy rainfalls which caused high damages in a number of municipalities. As a consequence, the Saarland Ministry for the Environment and Consumer Protection (MUV) commissioned CEval GmbH to review the measures of hazard prevention and disaster management in the wake of these events. For this purpose, an analysis of strengths and weaknesses of hazard prevention and disaster management with regard to prevention, deployment and after-care was carried out in order to lay the foundation for identifying potential improvements and evidence-based recommendations. In addition, the damages caused by the heavy rainfalls should be assessed.
To analyse the strengths and weaknesses of hazard prevention and disaster management, 62 guided interviews were conducted with representatives of the Ministry of Interior, Construction and Sport, fire-fighting coordinators, local aid organisations and mayors of the most affected municipalities. The results of the content analysis of these interviews formed the basis for identifying potential for improvement and deriving concrete recommendations for action. The evaluation of damages in the area of municipal infrastructure, agriculture and private households was carried out quantitatively on the basis of the data provided by the client.
Development of an operational guidebook for integrating and using impact evaluation-based M&E approaches in the field of climate change adaption
Climate change adaption projects require methodologically elaborated Monitoring and Evaluation (M&E) Systems that are not limited to measuring Project Outputs but also Need to include outcomes and further Impacts as well as their attribution to the measures undertaken.
GIZ engaged the Center for Evaluation (CEval) for technical support to develop an operationally viable impact evaluation and provided examples for their applicability in adaptation projects. Results were presented in a guidebook for integrating and using impact evaluation-based M&E approaches in the field of climate change Adaptation. Target Groups of the Framework are ministries, donors, and other related institutions that are looking for the underlying Drivers of Adaptation and evidence-based findings to improve the design of future climate change adaptation projects.
Evaluation of the Projects “Sustainable Municipal Development through Partnership Projects” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects”
As part of Engagement Global – Service für Entwicklungsinitiativen (Service for Development Initiatives) the Service Agency Communities in One World in Germany ist the competence Center for local development cooperation. Founded in 2001 the Service Agency offers a platform for dialogue for municipalities in Germany and the South and supports development acitivities of local actors by providing them with Training, Information and advisory Services.
The two programs “Sustainable Municipal Development through Partnership Projects – Nachhaltige Kommunalentwicklung durch Partnerschaftsprojekte (Nakopa)” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects – Förderprogramm für kommunale Klimaschutz- und Klimaanpassungsprojekte (FKKP)” are core activities of the Service Agency. While Nakopa supports existing and new municipal partnership Projects in selected relevant fields of development cooperation, FKKP explicitly addresses municipal activities in the field of climate protection and climate change adaptation.
After the first Project cycle of three years has been completed, the current Evaluation aims at evaluating These two programs with respect to processual aspects of project Administration as well as Impacts of both the funded partnership Projects and the Support programs Nakopa and FKKP. Leading questions of this evaluation are guided by the OECD-DAC evaluation criteria and will be answered using the approach of contribution Analysis and documentary and empirical data. Hence, the evaluation is based on Monitoring data and Project documents on the one Hand and empirical data gathered through extensive partly standardized interviews and partly standardized online surweys with participating communities on the other hand.
Evaluation of the German Strategy for Adaptation to Climate Change
The German Strategy for Adaptation to Climate Change (DAS) and the corresponding Adaptation Action Plan are the main pillars of Germanys climate policy. The Federal Government aims to reduce the vulnerability of the economy, the environment and society to the effects of climate change in the long term. To this end, a number of concrete measures have recently been implemented at various levels. In order to successfully adapt to climate change and to continue to develop the national climate change strategy, it is important to critically reflect on the current process as well as to acknowledge achievements and possible shortcomings.
The overarching goal of the evaluation commissioned by the German Federal Environmental Agency (UBA) was to determine whether the national adaptation strategy represents a successful approach for increasing the adaptive capacity and reducing the vulnerability of natural, social and economic systems to the effects of climate change. The evaluation included three central parts: (1) the process of developing and implementing the strategy on the nationl level (2) the implementation status of the adaptation measures at the federal level and (3), an impact assessment applying contribution analysis of the strategy’s effects on the vulnerability and adaptative capacities of natural, social and economic systems.
The Center for Evaluation (CEval) implemented the evaluation together with adelphi. To do so, a methodology developed in a previous project was used which has already been adopted politically. With the use of central evaluation questions as well as a policy impact model, first, data were collected using a variety of methods (semi structured interviews, document analysis etc.). Next, using defined evaluation criteria, a comprehensive data analysis and evaluation was carried out. Finally, the results and hypotheses were reflected in an online-based Delphi method and then further condensed.
