Enabling adaptation strategies: knowing what works and what doesn't. by SEA Change CoP
Adaptation strategies are well known, but do they work?! Let's collectively learn from our successes and failures to enable adaptation.
When reading the contest pitch "How can businesses, governments, and individuals coordinate to ensure the implementation of effective domestic and international strategies to prepare for the effects of climate change?", we first and foremost need to know what makes an effective domestic and international strategy or adaptation action.
In the monitoring and evaluation of climate change adaptation interventions it is the word 'effective' that actually hampers evaluation evidence-based learning.
Donors, governments and businesses are often mainly interested to find out what happened with their money and whether the proposed output has been reached. The effectiveness is proven by evaluating whether goals were reached. The efficiency of an adaptation action is proven by evaluating whether goals were reached within given budget and timeframe.
We need to look beyond the output of adaptation interventions and need to learn from the adaptation process itself, which can be either a programme or policy implementation or strategy development process.
We need to see whether there actually is any longer-term impact to be expected from an intervention, ie. reaching short-term outputs does not necessarily guarantee any adaptation success towards long-term climate change effects.
How do we do that?
- First and foremost; we should not be afraid to report failure. The figure of 70% of change initiatives failing is well known, but strangely enough most reporting on adaptation interventions seems to be on successes, not on failure.
- We need to mend the disconnect between academic research on adaptation and adaptation practice on the ground to enable learning.
- We need to invest time and money into developing not only donor reporting, but also lessons learned from the evaluations of climate change adaptation interventions.
There is a strong role here for Communities of Practice bringing together academics, practitioners and donors, and advocate on behalf of and assist in coordinating their actions.
Category of the action
What actions do you propose?
Communities of practice for monitoring and evaluation of climate change interventions play a bridging role between academics, government officials, policy makers and climate change practitioners.
The SEA Change Community of Practice (www.seachangecop.org) supports practitioners, policy makers and funders in leveraging the power of networking to capture learning, monitor progress, evaluate achievements of climate change adaptation interventions and learn from one another about climate change adaptation-specific monitoring and evaluation practices.
SEA Change CoP identifies, operationalizes, disseminates and also actively develops information and knowledge in the emerging field of the monitoring and evaluation of climate change interventions. The scope of interventions includes programs and projects to climate policy and finance, bridging multiple sectors, approaches and implementation levels.
Organizational Network Analysis (ONA) over time shows that SEA Change CoP is very able to connect both individual practitioners and organizations to enable the sharing of evaluation findings beyond donor reporting. The range of intellectual resources being accessed, the development of focal resources accessed by members and a further general diversification in connections between members is visible when comparing baseline data of the 2010, 2011 and 2012 ONA.
(For the full ONA report: http://www.seachangecop.org/node/821)
The Community has grown in the meanwhile to over 700 members, with 14,000 monthly pageviews on resources shared.
I propose the following actions:
- Improve evidence-based learning by sharing evaluation results of adaptation actions and developing related knowledge materials by / through SEA Change CoP, its member base and collaborating partner institutions and organizations.
- SEA Change CoP is a steering committee member of the Climate Knowledge Brokers (CKB) group (http://en.openei.org/wiki/Climate_Knowledge_Brokers_Group), an informal network and an emerging ‘community of practice’ for organisations carrying out knowledge and information work on climate and development. The focus is on online initiatives, and those that play an explicit knowledge brokerage role, rather than being simply institutional websites.
- All resources shared and developed by SEA Change CoP are also shared through the CKB group members, extending the Community's reach well beyond its member base.
- SEA Change CoP also partners and collaborates with a range of organizations - mentioned below - on both the dissemination of information and further evidence-based knowledge generation.
Organizations are often afraid to report on failure, given it might impact their future funding.
- Anonymize evaluation results by developing anomymous evaluation reviews and lessons learned of projects, programmes, policies and strategies that failed. There is a wealth of information out there that currently is ending up in desk drawers due to a fear of reporting on failure
- SEA Change CoP is currently developing a collaborative partnership with UKCIP (http://www.ukcip.org.uk/) to develop anonimous 'evaluation reviews' and 'guidance notes' to capture learning lessons on process and output of climate change adaptation interventions, looking at both their successes and failures. This collaboration could certainly do with financial support to further enhance the range of knowledge materials planned to be developed.
- Promote reporting of failure and advocate towards donors and governments the need to learn from failure. The development of anonymous 'evaluation reviews' is a start in this respect.
13% of the SEA Change CoP member base is working for donors and foundations, while 10% is working for governmental institutions, guaranteeing the appropriate constituency to advocate the need to learn from failure.
The Rockfeller Foundation, SEA Change CoP's main donor at current, is also actively promoting evidence-based and failure-based learning. The Rockefeller Foundation is open to continue support to SEA Change CoP if there is proof of others being interested in co-funding the initiative. Winning ClimateCoLab would certainly provide a great deal of attention to the SEA Change CoP initiative, and would also further cement its credibility.
2. Adaptation actions are "sexy", evaluation is not.
- Though there is a slowly changing mindset, donors and governments are still hesitant to spend money on the evaluation of adaptation actions, or the development of knowledge materials based on evaluation evidence. Moreover, evaluation is often focused on output/outcome indicators towards donor reporting and it only focuses to a lesser extend on process evaluation to inform and improve learning.
- Advocate towards donors, governments and implementing organizations - through the SEA Change CoP member base, the members of the CKB group and other partner networks like the GEF Evaluation Office - the need to evaluate processes as much as evaluating outputs/outcomes.
- Advocate towards donors and government - through the SEA Change CoP member base, the members of the CKB group and other partner networks like the GEF Evaluation Office - the need to spend more money on climate change adaptation related evaluation, the learning from adaptation interventions and development of related knowledge materials, opposed to mainly providing financial support towards the adaptation intervention itself.
13% of the SEA Change CoP member base is working for donors and foundations, 10% is working for governmental institutions and 38% of the member base is working on climate change adaptation implementations for NGO's.
This mix of members guarantees the cross-pollenation between the donors, policy makers and people implementing on the ground.
- Advocate towards evaluation professionals - through the SEA Change CoP member base, the members of the CKB group and other partner networks like the GEF Evaluation Office - the need to also take on a knowledge brokerage role and develop lessons learned, opposed to mainly evaluate for donor reporting.
- Support implementing organizations in developing lessons learned documents based on their evaluation evidence and spread that message through online communities of practice.
3. Mend the disconnect between academic research and climate change practitioners.
Climate change and monitoring and evaluation communities of practice have a knowledge brokerage role to play in order to:
- disseminate academic research in order to inform adaptation action;
- use evaluation-evidence to inform the academic world about knoweldge gaps in the field of climate change adaptation, in order to develop research tailored towards adaptation action practitioners.
16% of the SEA Change CoP member base is working for academic institutions, while 38% of the member base is working on climate change adaptation implementations for NGO's and 24% is working on adaptation and related monitoring and evaluation as consultant.
This mix of members guarantees the cross-pollenation between the academic world and people implementing on the ground.
Who will take these actions?
Communities of practice are groups of people who share a passion for something they do and who interact regularly to learn how to do it better (Etienne Wenger, 2000).
Communities of practice define themselves along three dimensions: what they are about, how they function, and what capabilities they produce (Olivier Serrat, 2008; Olivier Serrat, 2010).
It would be the SEA Change Community of Practice, supported by its members and extended network of partners and collaborators, who would take these actions!
SEA Change CoP partners with networks and organizations to improve knowledge sharing and networked knowledge generation, a.o.:
- Climate Knowledge Brokers (CKB) group
- Global Environment Facility – Evaluation Office (GEF-EO)
- Climate-Eval Community of Practice
- GIZ Competence Centre for Climate Change
- Asia Pacific Adaptation Network (APAN)
- Asian Cities Climate Resilience Network (ACCCRN)
- International Development Evaluation Association (IDEAS)
- The South-Asian Evaluation Conclave and Community of Evaluators
- International Organization for Cooperation in Evaluation (IOCE)
Potential pitfalls and limitations:
- Evaluation isn't sexy. Donors might see competing networks not specifically focusing on adaptation evaluation as 'better' / 'more visible' candidates to fund.
- Constraints in voluntary participation, given members are volunteering their time next to their work, and partnering and collaborating institutes / organizations do so voluntarily as well.
Strategies to counter-act these pitfalls and limitations:
- Donors sponsoring evaluation CoPs should be seen as innovators, pioneers, trend-setters. Don't focus on evaluation not being 'sexy', focus on funding adaptation evaluation being 'innovative' and 'pioneering'.
- Focus on the power of collaboration. Collaborating organizations often have overlapping goals and a coordinated response towards these goals is the most effective / efficient.
- Focus on non-tacid elements of volunteerism; networking, friendships, etc.
Where will these actions be taken?
These actions will be taken globally and through online information exchange and collaboration, supported by the SEA Change CoP members and the existing networks partnering and collaborating with SEA Change CoP.
No new networks or online portals will be developed!!
This needs to be pointed out very clearly, because we see new initiatives, networks and portals being developed daily, while it makes much more sense to sponsor existing initiatives.
There actually is a name for it in climate change, being the 'Portal Proliferation Syndrome' (PPS) (For more information: http://cdkn.org/2011/06/portal-proliferation-syndrome/), which is actually the reason why the Climate Knowledge Brokers (CKB) group came into existence.
What are other key benefits?
Other key benefits are as follows:
- Evidence-informed adaptation action, knowing what works and what doesn't, will increase the rate of success and decrease the cost of climate change adaptation interventions.
- Changing people's mindset towards evaluation learning and reporting on failure will have a positive effect well beyond the field of climate change adaptation.
- At the same time we can learn about what works and what doesn't towards knowledge brokerage, information dissemination and knowledge generation through communities of practice. This information can be used in other fields where learning is taking place through networks and communities of practice.
What are the proposal’s costs?
With a focus on knowledge generation through collaboration, partnerships and harvesting the knowledge and experience of our member base, the development of knowledge materials like the 'evaluation reviews' and 'guidance notes', and the continuation of existing activities like our current series of newsletters / online resource library and monthly webinars, SEA Change CoP budget is 285,000 USD / year.
The 10,000 USD would be used to develop knowledge materials as mentioned above, that were not part of the current budget and related logical model.
Moreover, the grand prize would also help to cement SEA Change CoP as online adaptation evaluation knowledge base and would also help to attract co-funding for the initiative, needed to guarantee continued long-term support by our main funder, the Rockefeller Foundation.
The timeline for the development of knowledge materials as explained above would be starting in August 2013 and run up to June 2014.
The specific topics for the 'evaluation reviews' and 'guidance notes' will be based on the results of our continuous gap analysis on adaptation evaluation knowledge needs, which is already part of the SEA Change CoP initiative's design.
Related to the need to improve evaluation-evidence based learning towards climate change adaptation interventions, there will also be a need to evaluate the impact of communities of practice towards its members and the wider community of climate change professionals.
SEA Change CoP's Organizational Network Analysis (ONA) approach is quite innovative, but we would like to also look at other promising approaches, like the mapping of 'boundary partners' as it is part of the 'outcome mapping' Monitoring and Evaluation (M&E) technique.
Serrat, O. (2010). Building communities of practice. Manila: Asian Development Bank.
Serrat, O. (2010). identifying and sharing good practices. Manila: Asian Development Bank.
Wenger, E. and Snyder, W. (2000) Communities of practice: the organizational frontier. Harvard Business Review. January-February, pp. 139-145.
Wenger, E. (2000) Communities of practice: stewarding knowledge. In Despres, C. and Chauvel, D. (eds.) Knowledge Horizons: the Present and the Promise of Knowledge. Butterworth-Heinemann, Boston.
Wenger, E., McDermott, R., and Snyder, W. (2002). Cultivating communities of practice: a guide to managing knowledge. Cambridge, MA: Harvard Business School Press.