New Diagnostic Test Measures Misconceptions

Surveys and other close-ended questionnaires have been used extensively to identify misconceptions people have about environmental issues. We know, for example, that many people incorrectly believe that ozone-layer depletion causes global warming. Identifying misconceptions is important, because misconceptions often are difficult to change and, as a result, require specialized teaching approaches that are different from simply educating people about something that they don't know. It's also important to identify teachers' misconceptions because of the risk that they will pass these strongly held, but incorrect, notions on to students.

But, the authors of this paper argue, the tools we most often use to diagnose misconceptions are not refined enough to paint a clear picture of what people actually believe. Multiple-choice questions can result in guessing; through this process, a respondent might reveal a lack of knowledge by selecting the incorrect answer, yet he or she might not necessarily hold a misconception. That person, for example, might choose a response that indicates that ozone-layer depletion causes global warming. While that response could be interpreted as a misconception, it may be the case that the person might simply have guessed wrong or selected the wrong response for a reason other than holding a misconception.

To address this issue, researchers have developed two-tier tests that add a second question to clarify whether a person can identify a reason for his or her choice. In this study, the authors added another aspect: They included a third tier to measures an individual's certainty in his or her response. To be classified as a true misconception, a person must answer incorrectly on both the first and second question, and also be confident in their response.

After initial testing, the authors administered this new three-tier test—the Atmosphere-Related Environmental Problems Diagnostic Test (AREPDiT)—to 256 preservice teachers in their third or fourth year of a teacher certification training at a university in the American Southwest. The test included questions related to global warming, greenhouse effect, ozone layer depletion, and acid rain.

In general, the results revealed low levels of understanding of all four issues. Interestingly, although the preservice teachers' knowledge scores were low, their certainty scores were moderate, which the authors indicate aligns with previous research suggesting that people tend to be overconfident in assessing their own knowledge.

The results also indicated that the teachers held five prevalent misconceptions:
• Global warming is caused by ozone layer depletion
• Global warming will cause skin cancer
• Acid rain is a result of global warming
• The greenhouse effect is a totally harmful phenomenon for humankind
• Using public transportation reduces ozone layer depletion

But the three tiers proved valuable in analyzing the results. For example, 56% of the preservice teachers linked global warming and ozone layer depletion in their first-tier question, which is in line with previous research that has indicated similarly high levels of support for this incorrect idea. But when the authors looked at the first two tiers together, the misconception decreased to 40%. And when they accounted for all three tiers, just 18% of preservice teachers held the misconception. The authors then question whether “a wrong answer on a one-tier test truly [identifies] a misconception? The results in this study indicate that identifying misconceptions by one-tier or even two-tier tests overestimates the percentages of misconceptions.”

The authors believe that this test can be used to assess misconceptions among preservice teachers, but also in-service teachers or high school students. They also urge educators to develop similarly designed three-tier tests to assess their students' misconceptions about other science topics.