Developing a tool to measure belief in climate change denial arguments

Communities worldwide are experiencing the devastating impacts of climate change. These impacts—which include rising sea levels, increased intensity and frequency of storms, droughts, flooding, and changing weather patterns—are projected to worsen over time. Even with overwhelming scientific consensus, however, misinformation about human-caused, or anthropogenic climate change (ACC), abounds and only about half of Americans believe in ACC. Among educators, roughly a third do not believe in ACC. Other educators, many of whom do not realize the extent of scientific consensus around ACC, believe that ACC is a “debate” with two valid sides. To ensure that accurate ACC information is being taught in classrooms and that teachers are equipped to address any climate denial theories presented by students, the researchers in this study assert that teachers must be aware of the anti-ACC misinformation that exists. This two-part study investigated existing types of dissenter messages and developed a tool—the Anthropogenic Climate Change Dissenter Inventory (ACCDI)—to measure the extent to which people believe these climate change dissenter messages.

The researchers conducted their study in two phases. During the first phase, the researchers investigated dissenter messages and used this information to create the ACCDI survey. During the second phase, the researchers refined and finalized the ACCDI survey and grouped dissenter statements into broad categories.

For the first phase of the study, the researchers investigated the types of ACC dissenter messages that exist on the internet. Specifically, they focused their investigation on the ACC dissenter videos posted on YouTube. They selected YouTube to search for anti-ACC content because most Americans seek out scientific information on the internet and because YouTube does not have an accuracy check on its video content. Using search terms like “climate change is fake” and eliminating videos with less than 15,000 views, the researchers identified three hours of video footage (with a combined total of over two million views) to analyze for common themes. The researchers uncovered 41 distinct dissenter messages, which fell into 2 broad categories: messages that used scientific arguments (30% of video runtime) to refute ACC and messages that used nonscientific arguments (22% of video runtime). Based on the 41 messages they identified, the researchers developed a preliminary 73-question ACCDI survey.

The second phase of the study involved refining and finalizing the ACCDI survey and categorizing dissenter statements. The researchers asked three education researchers to review the survey, make editing suggestions, and flag any issues. Based on feedback from the expert reviewers, the researchers amended the survey and then posted a pilot 73-question survey on Amazon Mechanical Turk (MTurk). MTurk is an online marketplace where “workers” can complete surveys for a small monetary incentive. A total of 133 people completed the survey. All survey respondents were from the U.S. and were over 18 years old. The researchers used statistics to analyze survey responses and pare down the survey to 44 questions. All survey questions asked respondents to read a statement and indicate the degree to which they agreed or disagreed with the statement (from strongly agree to strongly disagree). Examples of statements included: (1) Climate change science is not a science; it is a political propaganda; (2) Increasing the level of carbon dioxide in the atmosphere would allow farmers to grow more food; (3) A change in the energy output of the sun is responsible for the recent rapid warming of Earth's oceans. Once the 44-question instrument was finalized and its accuracy tested and confirmed, the authors used MTurk to recruit an additional 151 participants to complete the finalized ACCDI survey.

Results from the second phase of the study indicated that the 41 dissenter statements identified during the first phase could be grouped into five types of arguments:

Naïve statements refuting ACC: This was the largest category and included statements that would be recognized as incorrect by anyone with a basic understanding of earth sciences (e.g., there is no connection between atmospheric CO2 and climate change).
Sophisticated scientific arguments refuting ACC: The second largest category, these arguments referred to other factors that impact climate but are not the drivers of current climate change (e.g. Earth's tilt, the sun).
“Natural” (beyond human control) explanations for climate change: These arguments focused on ways that “Mother Earth” controls climate. \
Arguments that promoted the benefits of ACC: These arguments were more antiquated than the others but still present in the anti-ACC discourse (e.g., climate change extends growing seasons for farmers).
Arguments that climate change is part of a larger warming-cooling cycle.

This study had limitations. Both groups of participants were recruited through Amazon MTurk and, therefore, were required to have both internet access and free time to take surveys. Participants were not representative of the larger population. Conducting this study with another group of participants might yield different results. The ACCDI tool, itself, is limited because dissenter arguments continually change and evolve. To maintain relevance, the tool requires regular review.

The researchers recommend that educators be aware of current ACC dissenter arguments so that they do not inadvertently teach these arguments in the classroom. They also recommend that educators maintain awareness of anti-ACC discourse so that they can address the flaws of dissenter arguments and share accurate information with their students. The researchers suggest that teachers use the ACCDI survey to measure the extent to which their students believe in different types of dissenter arguments and then use that information as a launching place for discussion.