The goal of EE is to achieve environmental literacy (EL), which is having the knowledge and skills to make informed and strategic decisions to address environmental challenges. Evaluation of environmental education programming is critical for continual improvement and achievement of EL. However, numerous barriers exist to evaluating programs, including a lack of survey instruments that can be easily implemented to measure the impact of EE on participants. The authors felt that some instruments are too expensive, or too lengthy for informal and outdoor education settings (where attention spans are often short). In addition, many instruments only measure specific aspects of EE programs, and the authors believed that a need existed to measure EL more broadly. Validating an instrument is the process of ensuring that, when implemented correctly, a survey dependably measures the outcomes it intends to measure. Using validated instruments in evaluation can help practitioners better understand whether their programs are achieving intended goals, including EL. This study described how the authors developed and validated the Environmental Literacy Instrument for Adolescents (ELI-A).
EL is a complex topic and challenging to measure. Previous research has established four domains of environmental literacy: knowledge, affect, cognitive skills, and self-reported behavior. The authors of this study chose to develop and validate a survey instrument that measured specific concepts within these four domains, namely ecological knowledge, hope, cognitive skills, and behavior. The ecological knowledge questions on ELI-A ask about knowledge of physical and ecological systems and are multiple choice. Hope is how the authors chose to measure the emotional aspect of EL and is defined as the extent to which a participant believes they can and will achieve a goal. The authors adapted a previous instrument to measure hope for the ELI-A, which includes statements a participant agrees with (five choices from strongly disagree to strongly agree). Cognitive skills are defined as the ability for a participant to understand and analyze environmental issues. To measure cognitive skills, participants read a prompt about fishery collapse, then rank five questions based on their importance to solving the problem. The authors chose to measure behavior by asking about the frequency a participant engages in pro-environmental behaviors, such as recycling or conserving water (five choices ranging from never to always). Finally, the authors worked to minimize the survey length, around 5-15 minutes so that it would be usable in many settings.
This study took place in North Carolina, and all participants were high school students enrolled in the Agriculture Applications course. The authors took three steps to validate the survey. First, they piloted the draft survey with 20 high school students, who were asked to take the survey and provide feedback. Second, they recruited 14 high school agriculture teachers who gave the survey to over 700 students enrolled in their course; a total of 665 surveys were included in the analysis. Among these participants, 41% were female and 50% were male; the average age was 15 and ranged from below 14 to above 18; 55% described themselves as white and 23% as African American. The third step tested the final survey with 67 students to determine how long it took to complete. To analyze the validity of the survey, the authors used a variety of statistical methods.
The authors found that the survey is valid for use by EE practitioners interested in measuring ecological knowledge, hope, cognitive skills, and behavior in adolescents in formal and informal settings. In addition, the survey took an average of 10 minutes to complete.
Two aspects of the ELI-A are different from existing instruments. This survey is the first to use hope as a proxy for the affect aspect of EL. In addition, the use of ranking questions for the cognitive skills aspect helps to minimize reading requirements and the possibility that participants with lower reading comprehension score more poorly on this section. In addition, the authors link hope and ecological knowledge, confirming previous research that knowledge contributes to behavior particularly when tied to emotions and/or past experiences.
The authors acknowledge some limitations. Although its short length infers that the survey would work well in field settings and the authors encourage that testing, the validation described here is only in classroom settings. Also, statistical testing showed weak relationships between cognitive skills and behavior, suggesting that additional testing is warranted. The authors hope that practitioners will use the instrument with both younger and older adolescents, with students with differing academic abilities, and with diverse audiences.
The authors recommend that practitioners use the ELI-A to evaluate EE programs that work with teenagers. This survey can help programs understand whether a program is helping to develop EL among participants, or if adjustments are needed to achieve this outcome. The authors intend to publish this instrument in open access format to make sure that it freely available to use.
The Bottom Line
This study details the development of the Environmental Literacy Instrument of Adolescents (ELI-A), a short (5-15 minute) survey that evaluates four aspects of environmental literacy—ecological knowledge, hope, cognitive skills, and behavior. After testing the survey with 665 high school students enrolled in an agriculture course, the authors confirmed that the survey is valid and accurately measures these concepts. The authors recommend that the ELI-A be used to evaluate EE programs, particularly in settings where time may be limited.