Many environmental educators evaluate programs because a funder requires it. That can turn evaluation into a chore that produces reports but few useful lessons. This study asked whether the purpose and style of an evaluation affect how useful educators find it.
In mid-2020, the researchers sent an online survey through NAAEE, the Association of Nature Center Administrators and social media. In all, 410 practitioners whose organizations run programs in the United States described how they evaluate, what they want evaluation to do, and how satisfied they are.
What they found:
- Satisfaction is low. On average, educators were between "slightly" and "somewhat" satisfied with their evaluations.
- Formal evaluation pays off. Nearly four in ten organizations used only informal evaluation: staff observations, discussions and reading feedback, with no planned data collection. Educators using formal evaluation were more satisfied and met most evaluation goals better. This held whether their own staff ran it or an outside evaluator helped.
- Learning goals drive satisfaction. Satisfaction was highest where evaluations showed whether programs met their goals, why, and how to improve them, and whether participants were happy. These factors, together with evaluation type, explained close to half the differences in satisfaction.
- What matters most is achieved least. Educators rated improvement goals as more important than reporting goals but achieved them less often. Four in ten said their evaluations mostly or fully explained why programs succeed or fall short. Fewer than three in ten understood how programs affect different groups of participants.
- Outside evaluators help with reporting. Organizations working with an outside evaluator were best at meeting grant requirements and showing success to funders and partners.
What this means for your practice:
- Ask what your evaluation is for. The authors call for a shift from evaluating to prove success to evaluating to find what can be improved. Treat a less-than-perfect result as a starting point, not a threat.
- Add something systematic. Collecting the same planned data in the same way each time takes you beyond impressions. Formal evaluations, even when run in-house, went with greater satisfaction.
- Plan to learn why. Alongside outcomes, note what actually happened in each session. Record the educator's delivery style, the order of activities, the setting, how students were prepared, and whether visiting teachers took part. Comparing sessions shows which elements go with better results. You can also vary one element on purpose and compare.
- Close the loop. The authors describe this as adaptive management: evaluate, adjust the program, then evaluate again to see if the change helped.
- Look beyond averages. Collect basic information about who takes part, such as age group, school or first language, and compare results between groups. An average can hide who benefits most and least.
- Consider outside help. An external evaluator can help run a formal evaluation, build your team's skills and strengthen reports to funders.
- Talk to funders. The authors suggest funders hold programs accountable for continuous learning rather than for hitting preset numbers.
The Bottom Line
Researchers surveyed 410 environmental education practitioners in the United States about their organizations' evaluations. Overall satisfaction was low. Educators whose organizations ran formal evaluations, with planned data collection and analysis, were more satisfied than those relying only on informal reflection and feedback. Satisfaction was most closely tied to evaluations that told staff whether programs met their goals, why, and how to improve. Yet these improvement goals, though rated most important, were achieved less often than reporting goals such as satisfying funders. Understanding why programs work, and how they work for different groups of participants, lagged furthest behind. For educators, the study points to treating evaluation as a tool for learning and improving, not just for proving success.