Since there are no currently active contests, we have switched Climate CoLab to read-only mode.
Learn more at
Skip navigation
Share via:


Organizing the scientific community to assess the credibility of climate change media coverage to ensure an informed citizenry



We are at a critical moment in history, a time when our democracy must make important decisions about climate change. Having access to scientifically accurate and credible information is essential if we’re to choose the right courses of action.

But when so much contradictory information exists in the digital world, how can citizens with no expertise in climate science possibly know which information is accurate, and which is biased by political interests? Furthermore, how can citizens agree on collective actions when conflicting coverage of climate change is preventing a collective understanding of the problem?

Climate Feedback, simply put, aims to solve this problem.

Its objective is to organize the scientific community to assess the credibility of online climate change news coverage, with the aim of giving every citizen the chance to ‘see’ which articles and sources are scientifically accurate, and which ones are not.

The way it works is simple:

Climate Feedback will organize a community of climate scientists from around the world, and enlist them to comment on the scientific accuracy of news articles using the web-browser annotation platform Hypothesis.

These comments, or ‘annotations’, will run directly alongside the article text, allowing every citizen to “see” where and why the coverage is consistent (or inconsistent) with state-of-the-art thinking and knowledge in climate science.


Climate Feedback will then assign a “credibility” rating to each article based on the annotations and evaluations made by the participating scientists, giving readers a simple guide as to the overall accuracy of the selected text.


By calling out inaccurate coverage through our ‘peer-review’ process, and by bringing the scientific community closer to journalists and the public at large (and thereby provide new avenues to sources of scientific information), we believe this project can ultimately make a lasting impact in improving the accuracy and credibility of climate change reporting.

What actions do you propose?

Climate Feedback will:

1) Organize scientists into a community of annotators

Many scientists are already assessing the accuracy of online climate change coverage, but most of their efforts are done on an individual level and in relative isolation from the rest of the scientific community (e.g. analyses posted in “comments” sections, blogs or in press articles).

That’s why our first action will be organizing scientists into a collective community of annotators. We believe that in consolidating efforts under one project, we can amplify the voice of science in climate change coverage through the use of a ‘crowdsourcing’ strategy (i.e. each scientist will only have to contribute a small amount of work, but each scientist’s small contribution will build towards a much larger outcome), and through a ‘strength in numbers’ strategy (i.e. scientists’ annotations will carry the weight of the community at large).

2) Enlist the community to annotate articles, calling exact attention to ‘where’ and ‘why’ an article is consistent with scientific thinking or not

Line-by-line annotation allows for an easy-to-follow format, ensures that every piece of information is ‘peer-reviewed’, and calls to attention precisely what information is accurate and what is inaccurate or incomplete. This concise and ample ‘body of evidence’ for our community’s assessments will only further support Climate Feedback as an objective, evidence-based, and unbiased project.

3) Provide a top-level “credibility rating” for every annotated article

By providing a “credibility rating” for each article, we will provide readers with a useful tool for them to quickly decide whether the article they are about to read is of high or low scientific quality.

4) Aggregate these annotated articles in a central hub (Climate Feedback website)

By aggregating the articles’ evaluations in one central location, we allow for easy comparing and contrasting of news sources and articles. For example, if someone was interested in which sources and journalists have the best history of scientific accuracy, a simple scroll through our website will point the reader towards “higher” rated sources.

5) Engaging with journalists and readers on social media

We’ll turn to journalists to help us define the standards of science journalism (in their own terms). Climate Feedback will collaborate with journalists who wish to improve the scientific level of their work by providing analyses of their work, suggestions of relevant references, and also help connect them with a number of experts in the area they are covering.

We’ll invite anyone interested in this project to share the scientists’ work through their social networks, and help us expand our digital reach. The Climate Feedback team will connect with both digital and traditional media outlets and work to widely publicize scientists’ contributions in the media.

6) And Lead a ‘Citizen-Science’ Joint Effort to Re-Map Online Content

While the Climate Feedback project mainly depends of the efforts of scientists with advanced degrees, we also wish to incorporate the contributions of non-scientists in this effort.

We will design and lead a ‘citizen-science’ experiment, in which Internet users will annotate links between web pages. Participants will semantically tag the links between pages according to the ‘positivity’ or ‘negativity’ of the author’s intention. Highly credible pages usually link positively to other credible pages, and negatively (or not at all) to low credibility pages. In this experiment, the public will have a direct role in building a large network of web pages of a new kind: mapping online content that is consistent or inconsistent with scientific knowledge.  

The data created from contributions by both scientists and non-scientists will be useful in evaluating the credibility of a large number of web pages, and will open up new perspectives on using ‘credibility’ as a possible filter in page-ranking methods.

Who will take these actions?

1) Scientists
A community of scientists from around the world will be recruited to launch Climate Feedback and its annotation process. As of June 2015, Climate Feedback has enlisted over two dozen scientists from some of the best research institutions to its cause, and will look to grow this community to hundreds of participants in the future. These experts are motivated to contribute for a variety of reasons, but all remain linked by the core belief that it’s the civic duty of scientific professionals to better inform their fellow citizens in their area of expertise.

2) Coordinators
At first, a few (1-5) coordinators will be required to organize our article-analysis effort: identify articles to focus on, call for participation of relevant experts depending on the article’s topic, ensure the article annotations and evaluations are completed, and write a short blog-like post on the scientists’ conclusion.

These coordinators will range in backgrounds, and include editors, science journalists or graduate students willing to help our project. As our project grows in size, we will recruit more coordinators to help with the increase in workload.

3) Concerned citizens
Every citizen is invited to help spread the word about the scientific accuracy of different news outlets, and put pressure on these outlets to commit to the highest degree of scientific accuracy. Members of the public will also be invited to participate in our ‘citizen-science’ experiment to annotate the nature of the links between webpages.

Where will these actions be taken?

Climate Feedback aims to eventually target news sources from around the world, but for the initial launch phase, we will focus on countries where science denialism and political polarization of scientific issues is high, most notably the United States, Canada, and Australia.

Our community of experts will not be restricted to these countries, however. Any scientist who meets minimum academic qualifications criteria will be allowed to annotate.

How will these actions have a high impact in addressing climate change?

It's difficult to quantify success in terms of emissions reductions or other physical actions. We, instead, propose to quantify success in terms of citizens reached by our efforts.

Online news sources, especially the most influential ones, have the ability to reach hundreds of thousands to millions of readers with every article published. By focusing our efforts on articles published by influential news sources (and thereby connecting each source's large 'built-in' audience to our annotation efforts), we believe this project can restore the importance of scientific credibility in online coverage of climate change.

We also believe that by organizing the scientific community to annotate under one project, we can effectively communicate the overwhelming consensus that exists in the scientific community about climate change, in the hopes of forming an equally strong one among the general public.

What are other key benefits?

We believe this project will help develop the critical thinking skills needed in today’s digital age for readers to remain properly informed about issues. Through this project, readers will have a chance to see the patterns and characteristics of accurate and inaccurate reporting, improving their ability to do so in the future. Our hope is that whenever they encounter a news article, readers will automatically think, “What would an expert in this field think about the information provided here?”

This project will also make the scientific community more visible to the general public. Scientists are still the most trusted source of information on climate change*, and we believe bringing them directly to the public in this way will not only increase the trust in the scientific community, but will also open up more avenues of collaboration between journalists and experts in climate science.

What are the proposal’s costs?

  • Approximately $50,000 for the initial development of the web platform

  • Approximately $200,000/yr to support initiative in time by paying a few essential staff members (e.g. editor, coordinators)

  • The project has successfully raised $25,000 of initial funding and secured a commitment of ~$60,000/yr for 2 years

Time line

2015: Initial website and project development; experimentation with annotation and back-end processes; continued recruitment of scientists/contributors; focused aim of scaling up to capitalize on increase in interest and press coverage surrounding the Climate Conference COP21 in Paris in December 2015.

2016: Consolidation of all methods; pursue avenues to grow in both credibility and project awareness; explore opportunities to expand beyond climate science; explore ways to utilize credibility ratings archive to recognize most accurate journalists.

2018: Major search engines (e.g. Google) adopt ‘scientific credibility measure’ derived from crowdsourced evaluation of web pages by scientists. This new ‘search’ feature can help people avoid finding scientifically inaccurate content when browsing, or can be implemented into existing search algorithm to help filter out misinformation.

2020: Scientific credibility supplements popularity as a major feature for finding websites.

Related proposals


Ceccarelli L. (2011) Manufactured scientific controversy: Science, rhetoric, and public debate. Rhetoric & Public Affairs, 14(2), 195-228.

Cooper C. B. (2011) Media literacy as a key strategy toward improving public acceptance of climate change science. BioScience, 61(3), 231-237.

* Leiserowitz A. et al (2010) Climate change in the American Mind. Yale Project on Climate Change Communication.

Lewandowsky S. et al (2012). Misinformation and its correction continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131.

Oreskes N., & Conway E. M. (2010). Merchants of doubt: how a handful of scientists obscured the truth on issues from tobacco smoke to global warming. Bloomsbury Publishing USA.