The vision for ee360+ is a more just and sustainable future for everyone, everywhere. But what does that look like and how do we measure it? When U.S. EPA asked NAAEE and the ee360+ partners to document their activities' progress toward Justice40, we at COSI Center for Research and Evaluation (CRE) recognized this as an opportunity to measure and illustrate performance toward equity, diversity, and justice goals more broadly. (Justice40 is an executive order by the White House with a goal that “40 percent of the benefits of Federal investments flow to disadvantaged communities that are marginalized, underserved, and overburdened by pollution.”) As the evaluators for ee360+, CRE introduced our first report on Justice40 during the second year of the ee360+ (2022–2023), and we continue to refine our approach to data collection and reporting.

Starting with Questions

As we considered how to measure progress toward Justice40, several questions immediately came to mind: 

• How do we measure impact across a range of training programs, some that may be relatively short and others that are in-depth and longer-term? And how does a training program demonstrate impact when it takes place over time and change isn’t immediately evident? 
• How do we demonstrate the great depth that some partners build through partnerships with communities, organizations, tribes, and others over time?
• How do we demonstrate the breadth of reach for training activities across the country?
• And finally, how do we collect consistent data from all ee360+ partners without causing an undue burden or overwhelming them with data requests?

Turning to Maps

Another question was how best to report on the information collected effectively, and we turned to mapping as a potential visual tool for illustrating progress. Zip codes are a useful tool for evaluators to estimate depth, reach, and impact of programs. They proved especially worthwhile for measuring ee360+ progress toward Justice40 for two key reasons: First, it is relatively easy for ee360+ partners to collect and share zip code data for their programs. Second, census tracts identifying Justice40 communities were already designated in the White House’s Climate and Economic Justice Screening Tool.

Using Quantum Geographic Information System (QGIS) software, CRE can approximate ee360+’s impact by identifying the overlap between Justice40 census tracts and zip codes provided by ee360+ partners. In this case, the spatial overlap is a proxy in the absence of direct, identifying information about individual participants due to the inherent sensitivity of such information, as well as the difficulty in collecting zip codes for thousands of participants by partners. We used census tracts from the Climate and Economic Justice Screening Tool (CESJT) that exceeded at least one Justice40 criterion. Justice40 tracts that overlap with ee360+ zip codes are more likely to benefit directly from the ee360+ project than those that do not.

If you are interested in learning more about our methodology, refer to Mapping ee360+ and Justice40 to see the progression of how we developed these maps.

Using Stories, Too

As we think about the depth of ee360+ partners' work, CRE turned to stories and qualitative information. We interviewed some ee360+ partners working on longer-term community or school partnerships. These interviews led to stories about progress toward key outcomes of the partnership, which allowed us to provide NAAEE and EPA with insights into what some of the numbers in the reach could mean. These stories were added to the Justice40 report.

This year we are refining our approach to collecting, analyzing, and reporting on ee360+ activities and impact. We’re asking partners to keep tracking zip codes for their activities and share more stories from all of their wonderful programs! Partners, please reach out to us if you have any questions or want more information.

About the Authors

Joe Heimlich and Justin Meyer, both of COSI Center for Research and Evaluation (CRE), contributed to this article. While Joe and Justin are both social science researchers (Joe through education psychology and Justin through city and regional planning), both have roots in the arts. Justin is also a professional singer and Joe started his career in theatre and dance, and now sits on boards for arts organization!

This eePRO blog series, Ripple Effect, highlights stories of collaboration and impact among partners in the ee360+ Leadership and Training Collaborative. ee360+ is an ambitious multi-year initiative that connects, trains, and promotes innovative leaders dedicated to using the power of education to create a more just and sustainable future for everyone, everywhere. Led by NAAEE, ee360+ is made possible through funding and support from U.S. EPA and twenty-seven partner organizations representing universities and nonprofits across the country, as well as five federal agencies. Through this partnership, ee360+ brings together more than five decades of expertise to grow, strengthen, and diversify the environmental education field.