Considerations for reviewing research on outdoor adventure education

Personal and social development (PSD) is the growth of knowledge, skills, and attitudes in learners and is largely discussed as a key benefit of environmental education. PSD can include traits such as increased confidence, trust, teamwork, motivation, social awareness, and pro-environmental behaviors in students. Many studies have used qualitative data to support the long-held claim that outdoor adventure education (OAE) helps participants develop PSD. However, there is growing demand to prove OAE's success in developing PSD quantitatively. The researchers reviewed previous attempts to quantify PSD outcomes in OAE, identified shortcoming and strengths, and provided feedback on ways educators and researchers alike can conduct this research.

The researchers collected 22 studies and 6 meta-analyses for review. The studies spanned a 12-year period prior to 2015 and came from both the Journal of Adventure Education and Outdoor Learning as well as the Journal of Experiential Education. Together, these 28 publications used quantitative data to show the PSD of students participating in various OAE programs as opposed to the traditional qualitative (anecdotal) studies. Most of the studies' participants were teens and college students in OAE. Only two studies were focused on children. One study reviewed adults in OAE programming, and there was one study that looked at family groups in OAE. The 6 meta-analyses yielded 6 criteria categories under which the researchers reviewed and scored the 22 studies. These six categories were: 1) sample size; 2) use of control group; 3) assignment of participants; 4) questionnaire quality; 5) presence of pre- and/or post-tests; and, 6) project description and methodology clarity.

The researchers found the most common shortcomings of quantitative OAE research were size and participant pool of the samples, self-reported questionnaires, and variable management. They explained that the sample size is important for statistical measure. However, when a large sample or a control group cannot be obtained, the researchers acknowledged that small samples or studies without a control group can also elicit valuable results. For example, a small sample size can represent a small and/or specific population, and using pre-and post-tests for experiment groups can help mitigate the issues of studies without control groups.. The researchers found the majority of questionnaires did not align with OAE and were not uniform because they varied in the questions asked and ways to measure those answers. For example, some used a Likert scale (scaled responses, such as 1, strongly disagree, to 5, strongly agree) while others used short responses (where participants wrote their answers in their own words). Both measurement approaches have their pitfalls. There was also variation in administering pre-and post-tests. In some studies, pre-tests were delivered a week before the activity or only a few days prior, and some post-tests were given a week afterward or months after the activity. Further, there was little acknowledgment of the data distribution in these studies which inhibits the statistical analysis. Finally, OAE presents a unique and vast set of variables that contribute to the results of PSD research. Variables can include the study location, the educator, participants' previous experience, and more.

There are limitations to this literature review. First, the articles reviewed only came from two journals which may provide some bias in the type of study or results. Second, the meta-analyses may not capture the full extent of criteria that these studies should be evaluated against.

The researchers had several recommendations for future quantitative research in OAE. First, they recommended a sample size of 80 to 100 participants. Further, experiment and control groups should be randomly assigned, especially in larger sample sizes, to better represent a proportional distribution between genders, ages, races, and more. Second, the researchers recommended that questionnaires include intentionally-designed measurement scales for the sample population and that questionnaires are delivered a few weeks before and after the activity. Third, they recommended identifying and controlling variables in the study that researchers do not want to impact the dependent variables and the results.