Which approaches are associated with better outcomes? Evidence from a national study of environmental education field trip programs for adolescent youth in the United States

Most evaluations in environmental education look at one program at a time. That can show whether a program worked, but not which parts made the difference. The researchers set out to answer that second question by comparing many programs side by side.

In the first half of 2018, pairs of trained researchers observed 299 single-day field trips for students in grades 5 to 8. The trips were run by 90 nature centers, parks, gardens, museums and similar organizations in 24 states and Washington, DC. Observers rated 66 features of each program, educator and setting, from "not present" to "a major part of the program". At the end of every trip, students completed the same short paper survey about what they had gained in ten areas. These included learning, interest in science, connection to the place, attitudes toward the environment, motivation to care for it, and motivation to work hard at school. The researchers analyzed about 4,400 surveys and compared programs.

What they found:

1. Comparing programs identified 11 features linked to better outcomes. Together these explained close to a fifth of the differences between programs. The researchers point to other likely influences, such as students' backgrounds and how they were prepared before the trip.
2. Smaller groups did better. Group size was the feature most strongly linked to outcomes. Results improved steadily as groups got smaller; the average group had 16 students.
3. Natural, unfamiliar and place-focused programs did better. Programs in more natural places, in places that felt new or special to students, and built around what was special about the site had better results than programs whose activities could have happened anywhere.
4. How educators talked with students mattered. More back-and-forth conversation, open thought-provoking questions, clear transitions between activities and a conclusion that brought the lesson together all went with better results.
5. Clear, warm, responsive educators got better results. Educators who explained clearly, seemed at ease, responded to students' questions and cues, and offered encouragement had better results. Slow pacing, lecturing and educators who mostly recited facts went with weaker results.
6. Groups that arrived organized did better than groups that arrived late or in chaos.
7. The best and weakest programs differed in clear ways. Compared with the lowest-scoring quarter, the highest-scoring quarter had smaller groups, more natural and novel settings, more focus on the place and more back-and-forth talk. Their educators were also more responsive, clear and supportive.
8. What educators aimed for and did was narrower than the field recommends. Before their programs, about two-thirds of educators named knowledge as a goal, but only a quarter named behavior. Reflection, group discussion, multiple viewpoints and storytelling almost never appeared. Most programs focused mainly on facts.

What this means for evaluating your programs:

• Compare, don't just check. Comparing many programs showed which features mattered, which no single evaluation could. If you run several programs, sites or educators, give every group the same short survey so you can compare results across them.
• Record what actually happens. The study found specific, observable features linked to outcomes. When you evaluate, note group size, how natural and unfamiliar the setting is, how much the session uses the place, the quality of questions, transitions, whether there is a conclusion, and how clear and responsive the educator is. The researchers rated each feature from absent to a major part of the program.
• Study your best and weakest sessions. Top and bottom programs differed clearly in this study. Rank your sessions by results, then look at what the strongest ones do that the weakest don't.
• Check practice against intentions. Many recommended practices were rarely seen, and most educators aimed only at knowledge. Observing sessions, not just reading lesson plans, shows whether your program delivers what you intend.
• Look beyond one snapshot. Program features explained only part of the differences between programs, and outcomes were measured straight after each trip. The researchers suggest following students up later, and pairing surveys with detailed observation of individual programs. This would show longer-term effects and the influences a short survey misses.

RETURN TO EVALUATING ENVIRONMENTAL EDUCATION EERESEARCH COLLECTION