Most educators judge how a lesson went by watching their students. A group that is smiling, nodding and busy feels like a success. But a student who looks engaged may not be able to answer a simple question afterwards, and a student who fidgets and looks around may have learned a lot.
The researchers worked at NorthBay, a residential environmental education center in Maryland, where sixth graders spend five days in two-hour outdoor lessons. After a four-week trial to refine their tools, they observed 80 lessons with 15 groups from 17 schools. Observers rated how engaged students looked during each lesson. Students then completed a short survey about their own engagement and what they gained, responding to statements such as "This lesson made me appreciate nature more than I did before." The team analyzed 1,279 surveys.
What they found:
- Only self-reported engagement mattered. Students' own ratings of their engagement were linked to better outcomes. Observers' ratings of engagement were not.
- The thinking part of engagement is invisible. Observers can see behavior and emotion, but not whether a student is really thinking about the material.
- Observers can misread what they see. Engagement looks different from student to student, and an observer's skill or assumptions shape what they record.
- Students may hide engagement. Some students mask their feelings or play down their interest because of their peers or their cultural norms.
- Watching still has a job. Noticing how students respond helps educators adjust in the moment to students' interests, experience and abilities.
What this means for your practice:
- Don't judge a program's success by smiles. A lively, happy group is a good sign for the session, not proof that students learned what you intended.
- Keep watching, for a different reason. Use what you notice to adapt as you teach: slow down, change an activity or follow students' interests.
- Ask students directly. A short survey straight after a lesson, with statements about what they enjoyed and gained, tells you more about outcomes than your impressions do.
- Measure the outcomes you actually want. Decide what students should gain, such as knowledge, appreciation of nature or motivation to act, and check for those specifically.
- Test your tools first. Like these researchers, trial a survey or observation guide with a few groups and refine it before relying on it.
- Mind how surveys are given. Students filling in surveys together may rate themselves more positively. Give them privacy to answer honestly.
The Bottom Line
Researchers observed 80 lessons for sixth graders at a residential environmental education center and collected about 1,300 student surveys straight afterwards. Observers rated how engaged students looked, from disruptive or detached to attentive and enjoying themselves. Students rated their own engagement and what they gained from the lesson. Only students' own reports of engagement were linked to better outcomes. How engaged students looked was not. As the authors put it, smiles do not equal learning. For educators, the message is clear: watching students is valuable for adjusting your teaching in the moment, but it is no substitute for checking what students actually learned.