New Scales Measure Responsibility, Character, and Attitudes

A team of researchers working with staff at Maryland's NorthBay Adventure Center (see related summary “Residential EE Program Yields Positive Results, Especially for Urban Participants” in the Teaching Methods section) set out to measure the impact of NorthBay programs on middle school students. The center offers five-day residential outdoor education programs with an environmental education curriculum based on the Investigating and Evaluating Environmental Issues and Actions (IEEIA) model.

NorthBay's mission goes beyond affecting students' environmental responsibility, and includes a concern for fostering personal responsibility. As a result, the NorthBay staff who participated in the year-long planning process for the program evaluation identified three key outcomes: environmental responsibility, character development and leadership, and attitudes toward school.

The team created three scales with multiple question types to address each of the key outcomes. All of the questions were measured on five-point, Likert-type scales (ranging from “strongly agree” to “strongly disagree,” or from “always” to “never,” depending on the question). After pilot testing an initial survey, the researchers administered the survey immediately before and after the NorthBay residential program to 868 students over three weeks. Three months later, 349 students completed follow-up surveys. The researchers then used structural equation modeling and confirmatory factor analysis to test and refine the scales. Through these analyses, they concluded that the scales are valid and reliable—in other words, that the scales measure the concepts faithfully and consistently.

The researchers encourage environmental education practitioners and researchers who work with middle school students to use the scales if their programs also aim to address the same outcomes. Although “character development and leadership” and “attitudes toward school” outcomes have traditionally been associated with after-school and positive-youth-development programs, increasing numbers of environmental education programs are becoming interested in these outcomes. The researchers note, “The convergence of these different concepts to measure the outcomes of the NorthBay program reflects not only the innovation of the program itself, but also signifies a first step toward acknowledging, quantifying, and evaluating the impact of environmental education programs on additional outcomes of common interest.”