Many environmental educators lack the time, money or training to evaluate their programs, and evaluation can feel like a box to tick. The COVID-19 pandemic made this harder: from 2020, many organizations had to build online programs quickly, with little evidence about what worked.
From September 2020 to September 2021, a university research team ran a community of practice: a group of educators who meet regularly to learn together and share ideas. In all, 63 educators from 37 organizations offering online programs for students in grades 5 to 12 took part.
The network worked like this:
- A shared student survey. After each program, every organization gave students the same short online survey. It asked about 12 outcomes, from knowledge to skills, attitudes and intentions to act. Over two rounds, in fall 2020 and spring 2021, 28 organizations collected surveys from about 4,100 students.
- Private reports. After each round, the researchers sent each organization a report comparing its students' scores with the average for the whole network.
- Meetings. Members met online every month to build evaluation skills and share ideas. After each round they also joined a "learning exchange": two two-hour online sessions to discuss results, review research on effective teaching and plan changes. Each organization then wrote an improvement plan and adjusted its program before the next round.
At the end, 28 of the 63 members completed a survey about what they had gained.
What they found:
- Members valued the support of colleagues. They felt less alone in a hard year. They said the shared resources were relevant to their work and that suggestions from others were useful.
- Members' evaluation skills and attitudes improved. They reported learning to identify program outcomes, interpret results and use them to adjust programs. They felt more positive about evaluation, more likely to do it in future and better prepared to work with outside evaluators. They felt least sure about collecting reliable data.
- Members changed how they teach. They said they had tried techniques learned in the network and that their teaching had improved. Some began aiming at outcomes beyond knowledge, such as students caring for the environment.
- Ideas spread beyond the network. Twenty-five of the 28 shared resources within their organization, and seven shared them more widely, for example with education partners.
- More involvement brought more benefit. Members who attended more meetings reported more benefits. The 11 who completed the full cycle reported more useful suggestions, more improvement in their teaching and more new techniques than other members. The full cycle meant collecting data, changing the program, then collecting data again.
- Arranging surveys with schools was the biggest barrier. Members found it hardest to coordinate surveys with the teachers and schools in their programs. School firewalls sometimes blocked the survey, and some students lacked devices.
Because many organizations collected only a few surveys, the study could not show whether the program changes improved students' results.
What this means for your practice:
- Evaluate with peers. Members gained skills, more positive attitudes toward evaluation and new teaching ideas by working through results with other organizations. A regional group of educators could agree on a shared survey and meet to compare results.
- Aim beyond knowledge. Members said that seeing a broad list of outcomes helped them set clearer goals. Consider whether your programs could also aim for skills, attitudes and intentions to act, and measure those.
- Plan small, specific changes. In this network, facilitators encouraged small, manageable changes, such as adding an introductory lesson or reworking one activity. Each organization wrote down what would change, which outcome it should improve, what steps it needed, and who would do it by when. This is how the network worked; the study did not test whether this approach worked better than others.
- See each other's programs. Members asked to see colleagues' full programs, and the researchers added sessions for this. The researchers suggest that seeing real examples helps educators turn research into practice.
- Sort out survey logistics early. This was the biggest barrier, so agree with teachers before the program on when students will take the survey. Check that school firewalls won't block it and that students will have devices.
- Commit to the full cycle. The most involved members gained the most. The researchers suggest a smaller group of committed educators may work better than a large network.
RETURN TO EVALUATING ENVIRONMENTAL EDUCATION EERESEARCH COLLECTION
The Bottom Line
Researchers ran a year-long online learning network for 63 educators from parks, nature centers, zoos, aquariums and similar organizations, then surveyed members about what they gained. Members reported better evaluation skills, more positive attitudes toward evaluation, new teaching ideas and a supportive group of colleagues. Many tried new techniques in their programs and shared what they had learned with others. Members who took part more fully reported the greatest benefits. The hardest practical step was arranging student surveys with schools. For educators, the study suggests that evaluating programs alongside peers, with support from experienced evaluators, can make evaluation feel useful and lead to concrete plans for improving programs.