Does student enthusiasm equal learning? The mismatch between observed and self-reported student engagement and environmental literacy outcomes in a residential setting

Educators often judge a lesson by how engaged students look. But a student who nods and smiles may not be able to answer a simple question afterwards, while a student who fidgets may have learned a lot. Engagement involves thinking as well as behavior and feelings, and thinking is hard to see.

The researchers worked at NorthBay, a residential center in Maryland where middle school students spend five days taking two-hour outdoor lessons across a 90-acre campus. They observed 80 lessons with 15 groups of sixth graders from 17 schools. During each lesson, observers rated how engaged students looked on six signs: disruption, frustration, detachment, attention, effort and enjoyment. After each lesson, students completed a short survey. It asked about their own engagement, with statements such as "I enjoyed the lesson", and about what they gained, with statements such as "This lesson made me appreciate nature more than I did before." The researchers analyzed 1,279 surveys.

What they found:

1. Students' own reports of engagement were linked to what they gained. Students who rated themselves as more engaged in a lesson also reported gaining more from it, such as a greater appreciation of nature.
2. How engaged students looked was not linked to what they gained. Observers' ratings of attention, effort, enjoyment and the other signs did not match what students reported gaining.
3. The researchers give three reasons for this mismatch. The thinking side of engagement cannot be seen. Observers may misread behavior and emotions, which look different from one student to the next. And some students hide their feelings or play down their interest because of their classmates or their cultural norms.
4. Watching students still matters for teaching. The researchers point out that noticing how students respond helps educators adapt a lesson to students' interests, experience and abilities as it happens.

What this means for your practice:

• Don't use smiles as your evidence. How engaged students looked did not match what they gained. A lively, happy group shows the session felt good, not that students learned what you intended.
• Keep watching, but to adjust rather than judge. The researchers recommend continuing to read students' responses as you teach, so you can slow down, switch activities or follow their interests. Use what you see to steer the lesson, not to decide whether the program worked.
• Check outcomes in a planned way. The researchers recommend evaluating the outcomes you want systematically rather than relying on impressions. In this study, a short survey after each lesson asked students what they had gained; a similar survey could fit at the end of your own programs.

RETURN TO EVALUATING ENVIRONMENTAL EDUCATION EERESEARCH COLLECTION