Computer-Based Concept Maps Effectively Assess System Thinking

Brandstädter, K. ., Harms, U. ., & Grossschedl, J. . (2012). Assessing system thinking through different concept-mapping practices. International Journal of Science Education, 34, 2147-2170.

System thinking plays a central role in much of environmental education. Helping students understand how elements within a system are connected is a common—and often critical—teaching goal. Therefore, assessing whether students understand the structure and function of systems can be important.

The authors of this study focused on one way to assess system thinking: the concept map. Concept maps allow students to illustrate their mental models of how elements are connected by creating a visualization of concepts that depicts the interconnected “nodes,” which are separate elements joined by lines that are then labeled to indicate the nature of the connection. This study's authors investigated whether a paper-and-pencil or computer-based map better assessed students' knowledge. The authors also evaluated the effects of the level of direction given to the students in developing their concept maps.

The authors randomly assigned 154 German fourth graders and 93 German eighth graders to one of three groups: highly directed paper-and-pencil mapping, nondirected paper-and-pencil mapping, and highly directed computer mapping. (The computer program did not allow for a nondirected mapping option.) In the directed group, students received suggested concepts and linking words to use in their maps; in the nondirected group, students received no such direction. The students received a series of lessons in their classrooms regarding ecosystem interactions in a local marine environment. After the lessons, they were asked to construct concept maps (either with pencil and paper or on the computer). The students also completed a questionnaire with multiple choice and open-ended questions that measured their system thinking.

The researchers found that both the fourth and eighth graders performed better with the computer-based concept mapping than the paper-based maps. The researchers believe that students probably found the computer easier to use than the paper and pencil. They explain, “We can assume that if a student has a tool at his disposal, and this tool facilitates the creation of maps that are more complex and easier to organize, he will perform better than with paper-pencil maps.” The authors believe that the computer maps might simply be easier to organize and revise. The authors did not, however, find that one method was more valid than the other.

The authors conclude, “To summarize, we consider highly directed computer-based practices to be appropriate for system thinking measurement, particularly for fourth graders, who obviously benefit from support by providing concepts and linking words that draw students' attentions more intensively towards the underlying system and facilitate the demand for conceptual knowledge.”

The Bottom Line

<p>This study points to the effectiveness of concept maps in assessing students' system thinking, which is a key concept in many environmental education settings. Highly directed assessments, in which students are given a set of concepts and linking words, can accurately reflect students' knowledge of how elements fit together. Although this study demonstrates that computer-based concepts maps are particularly effective for fourth-grade and eighth-grade students, paper-and-pencil maps, which may be more feasible in many environmental education settings, can still be valid assessments of system thinking.</p>