Evaluation of the Projects “Sustainable Municipal Development through Partnership Projects” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects”
As part of Engagement Global – Service für Entwicklungsinitiativen (Service for Development Initiatives) the Service Agency Communities in One World in Germany ist the competence Center for local development cooperation. Founded in 2001 the Service Agency offers a platform for dialogue for municipalities in Germany and the South and supports development activities of local actors by providing them with Training, Information and advisory Services.
The two programs “Sustainable Municipal Development through Partnership Projects – Nachhaltige Kommunalentwicklung durch Partnerschaftsprojekte (Nakopa)” and “Support Program for Municipal Climate Protection and Climate Change Adaption Projects – Förderprogramm für kommunale Klimaschutz- und Klimaanpassungsprojekte (FKKP)” are core activities of the Service Agency. While Nakopa supports existing and new municipal partnership Projects in selected relevant fields of development cooperation, FKKP explicitly addresses municipal activities in the field of climate protection and climate change adaptation.
After the first Project cycle of three years has been completed, the current Evaluation aims at evaluating These two programs with respect to processual aspects of project Administration as well as Impacts of both the funded partnership Projects and the Support programs Nakopa and FKKP. Leading questions of this evaluation are guided by the OECD-DAC evaluation criteria and had been answered using the approach of contribution Analysis and documentary and empirical data. Hence, the evaluation was based on Monitoring data and Project documents on the one Hand and empirical data gathered through extensive partly standardized interviews and partly standardized online surweys with participating communities on the other hand.
Evaluation of the website www.bne-portal.de
The German Commission for UNESCO is a Quango in the field of multilateral cultural, educational, Research, and media policy. The website www.bne-portal.de is one of their efforts in contributing to the Global Action Programme on Education for Sustainable Development. The website is considered to open the field of education for sustainable development (German: Weltaktionsprogramm Bildung für nachhaltige Entwicklung) to actual as well as future protagonists in Germany. After a re-launch in 2016 and an uptime of one year, CEval GmbH was entrusted to evaluate the website www.bne-portal.de.
The evaluation of the website was focused on (a) clarity, relevance, and appeal of the provided information, (b) attractiveness of design and structure, (c) information accessibility, (d) practical relevance of information, (e) usefulness of provided contact channels, (f) degree of fulfilling user expectations, (g) intensity of use, (h) popularity/attractiveness of its education materials, and (h) popularity/attractiveness of its protagonist database.
The evaluation was based on an online user survey followed by quantitative data analysis.
Evaluation of the German Strategy for Adaptation to Climate Change
The German Strategy for Adaptation to Climate Change (DAS) and the corresponding Adaptation Action Plan are the main pillars of Germanys climate policy. The Federal Government aims to reduce the vulnerability of the economy, the environment and society to the effects of climate change in the long term. To this end, a number of concrete measures have recently been implemented at various levels. In order to successfully adapt to climate change and to continue to develop the national climate change strategy, it is important to critically reflect on the current process as well as to acknowledge achievements and possible shortcomings.
The overarching goal of the evaluation commissioned by the German Federal Environmental Agency (UBA) was to determine whether the national adaptation strategy represents a successful approach for increasing the adaptive capacity and reducing the vulnerability of natural, social and economic systems to the effects of climate change. The evaluation included three central parts: (1) the process of developing and implementing the strategy on the national level (2) the implementation status of the adaptation measures at the federal level and (3), an impact assessment applying contribution analysis of the strategy’s effects on the vulnerability and adapative capacities of natural, social and economic systems.
Together with adelphi, the Center for Evaluation (CEval) carried out the evaluation on behalf of the UBA. The consortium drew on the methodology developed in a previous project and already politically agreed. Data was initially collected using various methods (guided interviews, document analyses, etc.) based on key evaluation questions and taking into account an impact model. A comprehensive analysis and evaluation of the data was then carried out on the basis of defined evaluation criteria. The resulting findings and recommendations were reflected upon and further consolidated in an online Delphi process.
Evaluation of the International Climate Protection Fellowship for Young climate experts from developing countries
In 2009, the Alexander von Humboldt-Foundation (AvH) initiated the International Climate Protection Fellowship (ICPF) for young climate experts from non-European developing/emerging countries. From 2010 to 2015, they supported 100 future leaders from academic and non-academic fields.
The target Groups of ICPF are young academics and future leaders form industry/politics from developing/emerging countries with higher education. Those Groups participate in developing/applying measures of (1) climate protection, (2) climate-related protection of resources, and (3) adapting to climate Change in developing/emerging countries. The ICPF’s Goal is to accompany their fellows over a long way: 12-months Fellowship for conducting their own research/research-related projects in cooperation with partners/Hosts in Germany, (2) participation in an alumni program (e.g., fundings for further stays in Germany, invitations to their home institutions abroad for their German partnsers/Hosts, invitations to AvH’s Network).
The Evaluation of ICPF examined the extent to which its effects contributed to (1) climate protection and climate-related protection of resources, (2) adapting to climate change in developing/emerging countries. Vital aspects of the evaluation were program-level examination of relevant goal achievements and the analysis of already testable program effects.
The evaluation was based on the DAC criteria. Combining document/secondary analyses with qualitative (extensive partly standarized interviews, case studies)as well as quantitative methods (partly standarized surveys), the evaluation can be characterized by its multi-method approach.
Evaluation of the Funding Activities of the Foundation Environment and Development North Rhine-Westphalia
The Foundation Environment and Development North Rhine-Westphalia was founded 2001 and promotes mainly volunteer environmental and development work of civil society groups. The objective of the foundation is to support the environmental and development work in North Rhine-Westphalia (NRW) in order to establish this work in the consciousness and in the practical involvement of the population permanently and at the state of the art.
For this purpose about 1,100 projects of non-profit organisations in NRW were supported with a volume of EUR 47.5 million so far. In addition, the Foundation carries out its own projects, to also promote the implementation of Agenda 21 of the United Nations Conference on Environment and Development in Rio de Janeiro in in 1994.
Within the evaluation, the results and effects of the funding activities of the Foundation were recorded and assessed. Furthermore, the evaluation was intended to find out information on potential inhibitory and / or promoting factors in the funding activities of the Foundation with respect to the foundation goals. Based on these findings recommendations on priorities of the Foundation with regard to content, actors, target groups and the methods and formats will be derived.
CEval applied a mixed method approach to collect relevant data. Therefore, relevant documents and accessible data of the foundation and of funded projects will be analysed. This will be combined with data based on qualitative guideline-based interviews with several stakeholders and standardised online surveys of current and completed projects.
Evaluation and Further Development of German Strategy for Adaptation to Climate Change
The German Strategy for Adaptation to Climate Change (DAS) and the associated Adaptation Action Plan are central pillars of German adaptation policy. Over the long term, the Federal Government is pursuing the goal of reducing the vulnerability of the economy, environment and society to the impacts of climate change. In order to achieve this, a number of tangible measures have been implemented at various levels in recent years. For 2019, it is planned that DAS and the strategic elements associated with it are to be evaluated in a progress report.
For the technical preparations for the evaluation of the DAS procedures, CEval GmbH supported adelphi in developing and testing an appropriate methodology. However, such an evaluation sometimes takes shape with difficulties: firstly, the identification of cause-effect relationships between adaptation activities and reduced vulnerability is not unproblematic. Secondly, the development of quantitative, measurable goals represents a major task. The methodology which was developed took account of these and other challenges in the evaluation of adaptation strategies and also developed appropriate solutions.
To this end, adelphi analysed and assessed exiting methods for the evaluation of national adaptation and sustainability strategies. During this process, it was also examined to what extent individual factors can be sensibly carried over into the evaluation of DAS procedures. The outcomes of the evaluation and the suggested methodologies were finally be discussed during several workshops with the relevant political stakeholders. To close, the methodology was tested in a pilot procedure and the insights gained were included in the eventual finalisation of the evaluation method.
Practice-Impact II: Development and Testing of a Concept for Documentation and Evaluation of Productive Interactions and Impacts on Practice and Society in Agricultural Research
Cooperation Partner: University of Kassel, Faculty of Organic Agricultural Sciences, Organic Farming and Cropping Systems
Currently, established research evaluation procedures are largely based on peer review publications, citation based indicators and third-party funds. However, the focus on the “scientific impact” only partially captures to what extent research contributes to solving societal or ecological problems. An increasing number of scientists and initiatives point out the responsibility of (public funded) science to contribute appropriate solutions for problems like global warming, food safety and loss of ecosystem services and research evaluations should set incentives to do so. This project was aimed at contributing to the implementation of complementary assessment of activities and impact of research regarding the needs of practice and society.
Concepts for an evaluation beyond scientific impact do already exist for inter- and transdisciplinary research, applied agricultural research and broader/social/societal impact assessments and are partly used by funding agencies. They include criteria for productive interactions between research and society, which make an impact probable, as well as the impact on practice and society itself. Nevertheless, broader use of the existing concepts is hampered by the lack of reliable, easy to use data.
The main task of the project was the further development and testing of a documentation and evaluation concept for agricultural research. The concept was defined to be broad enough to include all productive interactions and impacts of different types of agricultural research and can be used for the evaluation of scientists, institutions, projects and programmes. This requires a structured, standardized documentation in a database-system that enables to filter, aggregate and analyse data in different ways.
In order not to increase the effort necessary for scientists to document their projects, the developed documentation system is connected to general documentation procedures in proposals and reports for research funding but also includes project independent research.
The project developed and tested the structured documentation system and an assessment framework. This included three perspectives: Scientists responsible for documentation, funding agencies and evaluation experts. Furthermore, the project engaged in the development of applicable and accepted pathways to implement such evaluation in alignment with agricultural research and other relevant stakeholders and experts.
Accompanying Research and Acceptance Research on Current Issues of Grid Extension in Germany Scientific Monitoring of Planning Practice (Module III)
In order to reach a broad acceptance for grid extension within German society, information and communication actions on a large scale are necessary. First, Module III of the project provided an overview of existing and planned actions. Subsequently, the implementation of selected information and communication measures were analyzed to identify effects on attitude change towards grid extension, contributing factors and opportunities for optimization.
Assessing the Impacts of Multinational Corporations on Global Development and Value Creation (Global Value)
The importance of multinational corporations for furthering global economic and social development is well established. Corporations invest in foreign countries, provide jobs, and thus generate formalized labor relations and income. Furthermore, they pay taxes that contribute to the provision of public services. At the same time, multinational corporations particularly have received criticism by being accused of supporting exploitative employment conditions and human rights abuses, causing environmental deterioration, and failing to mainstream responsible conduct in developing countries.
The GLOBAL VALUE project developed an innovative framework for assessing impacts of multinational corporations on issues related to sustainable development, working conditions, human rights, transparency, and anti-corruption. It shed light on institutional arrangement; analysed systems of governance for responsible business practices; explored responsible competitiveness; assessed the complementarity of public and private sector activities; and derived recommendations for decision makers in business, policy and non-governmental organisations.
The project was carried out by universities and civil society organisations from Europe and ICPC countries. A toolkit for impact evaluation was developed and tested in close collaboration with leading multinational corporations (BATA (garment, Bangladesh), OLAM (food, Tanzania) and NOKIA (ICT, India)).
The main responsibility of CEval was the development of an indicator-set for the framework which is based on Millennium Development Goals, human rights, gender & diversity, and anti-corruption & transparency and takes corporate management approaches (supply chain management, life cycle analysis and base-of-pyramid) as well as systems of governance for responsible business practices, competitiveness, and complementarity with official development aid (ODA) into account. In addition, it carried out two research case studies focusing on the complementarity of ODA with multinational corporations´ activities.
Project Evaluation "Diakonie and Caritas as Places of Sustainable Lifestyles and Procurement"
The evaluated project “Diakonie and Caritas (Protestant and Catholic social welfare organisations) as Places of Sustainable Lifestyles and Procurement” was launched on November 1, 2010 and ended on April 30, 2013. The project aim was to improve the sustainability of procurement in institutions providing inpatient care for the elderly, children and young people and people with disabilities. At the same time, staff, residents and visitors were informed and encouraged to adopt sustainable Lifestyles. In addition, approaches to reduce the use of resources and to introduce sustainable procurement Standards tested in pilot Projects were intended to be nationally disseminated and implemented in Diakonie and Caritas institutions as well as in non-church-based social welfare organisations.
The Evaluation was formative, accompanied the process and began one and a half years into the project. It should provide relevant findings for the remaining Duration of the project and its planned continuation. Broadly speaking, a detailed Content description of the four project phases was compared to its current Status at the beginning of the Evaluation. More specifically, the aim was to document activities which had been realised when the evalzuation began, to obtain Information on the achievements of sub-Targets (“Improving Procurement Regarding Sustainability Standards”, “National Dissemination of the Project”, “Creating Network Structures”) and to identify potentials for optimisation. Special Focus has been laid on Institution Management and staff and how they can be motivated to pursue sustainable procurement.
CEval applied a mixed method Approach focusing on qualitative Guideline-based interviews and standardised online surveys to collect relevant data. Data and method analyses were enhanced by document analyses.
