Citizen science lets members of the public help collect or make sense of real scientific data. Educators increasingly use it, and many projects report strong results. But the strength of that evidence depends on how outcomes were measured.
The researchers searched published research up to September 2018 and found 100 studies, from many countries, that measured what participants gained. They recorded which outcomes each study measured and which methods it used.
What they found:
- Projects deliver a wide range of outcomes worth measuring. Studies found gains in science knowledge (56 studies), research skills (32), community connection and cooperation (30), and caring for the environment (29). Others found gains in attitudes, confidence and connection to place.
- Surveys and interviews dominated. Fifty-four studies used surveys and 45 interviews. Fewer used observation (25), focus groups (21), group meetings (21) or participants' work, such as journals or reports (10). Only six checked the quality of participants' data. More than half combined two or more methods.
- Knowledge was often self-reported. Every study measuring knowledge found gains. Many of these simply asked people whether they had learned, rather than testing them before and after. The authors note that people's self-ratings of knowledge are often inaccurate.
- Behavior was hard to pin down. Studies mixed intentions, people's reports of past actions and directly observed actions. The authors note that good intentions don't reliably predict lasting change.
- Few used tested measures, although tested survey scales for common citizen science outcomes now exist.
- Easy outcomes get measured more. The authors caution that some outcomes appear often partly because they're easier to measure. A missing outcome doesn't mean it didn't happen.
What this means for your practice:
- Match the method to the outcome. For knowledge and skills, check directly. For confidence, identity, interest and attitudes, people's own words are the right evidence; use interviews or open questions.
- Build assessment into the activity. Short tasks within the project, such as correctly identifying a species, show skills without a test-like survey.
- Use data-quality checks as evidence. Reviewing the accuracy of participants' data also shows how well they've learned the skills.
- Observe. A simple observation guide adds evidence that doesn't depend on self-report.
- Separate intention from action. Record what people actually do, not only what they plan to do.
- Borrow tested survey scales rather than writing new questions.
- Combine methods. Pairing numbers with interviews gives a fuller and more convincing picture.
- Measure what matters, not just what's easy. Decide your key outcomes first, then find a way to capture them.
- Try newer methods, such as arts-based activities or walking interviews.
The Bottom Line
The researchers reviewed 100 studies, published up to 2018, that measured what participants gained from environmental citizen science. Nearly all found benefits, most often in knowledge, research skills, community connection and caring for the environment. But how those benefits were measured varied widely. Surveys and interviews dominated, and many relied on participants saying they had learned, even for knowledge, which can be tested directly. Few used tested measures, and behavior was often judged from intentions rather than action. The authors recommend matching methods to outcomes: direct checks such as built-in tasks and observation for knowledge and skills, and interviews for confidence and identity. For educators, the review is a guide to evaluating hands-on programs more convincingly.