In their 2014 paper “Active learning increases student performance in science, engineering, and mathematics,” Freeman, et al., suggest that we are seeing a new wave of “second-generation research” in the education literature that explores “which aspects of instructor behavior are most important for achieving the greatest gains with active learning, and elaborate on recent work indicating that underprepared and underrepresented students may benefit most from active [learning instructional] methods.”

Indeed, a growing body of research shows that there are specific teaching strategies that (on average) improve learning outcomes for all students and also (on average) improve learning outcomes disproportionately for students who have been historically excluded from STEM: women, African American, Hispanic/Latinx, Native American, first-generation, low-income students.

For several years, I tried to maintain an annotated bibliography of such research. My goal was to provide higher education faculty and faculty developers with evidence to support teaching strategies that produce more equitable learning outcomes for all students, but particularly those who have been historically left out of STEM fields.

It must noted, however, that these learning gains from active learning pedagogies are not automatic. Much more research is needed to tease apart the nuances and conditions under which students who have been historically underrepresented in STEM benefit from active learning pedagogies. And in fact, please read this other post about research that provides more nuance on how active learning pedagogies don’t always lead to more equitable outcomes.

(Papers are listed in chronological order.)

---

Lorenzo, M., Crouch, C.H. and Mazur, E., 2006. Reducing the gender gap in the physics classroom. American Journal of Physics, 74(2), pp.118-122.

Multiple versions of an introductory calculus-based physics course for non-majors at Harvard were compared over a period of 8 years: (1) traditional lecture and recitation sections, (2) Peer Instruction with traditional recitation sections, and (3) Peer Instruction with cooperative problem solving activities during recitation sections. (It seems that there was continuous evolution of instructional practices throughout those 8 years, but the researchers groups students in these three buckets.) In Peer Instruction, the instructor alternates between giving short mini-lectures and facilitating small group student discussions on conceptual questions. Student learning gains were measured using the Force Concept Inventory. Learning gains increased with the uptake of more active learning pedagogies. In addition, performance differences between men and women decreased with the uptake of more active learning pedagogies. In version 3 of the course, there was no statistically significant difference in normalized learning gains between men and women, despite there being a difference in pre-test knowledge between men and women.

---

Beichner, R.J., Saul, J.M., Abbott, D.S., Morse, J.J., Deardorff, D., Allain, R.J., Bonham, S.W., Dancy, M.H. and Risley, J.S., 2007. The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project. Research-Based Reform of University Physics, 1(1), pp.2-39.

This report details the efforts of several groups of faculty to show that active learning pedagogies can be used in introductory physics classes with up to 100 students. The SCALE-UP approach involves using the bulk of class time for students in groups of 3-4 to work cooperatively on rich, computer-based activities and class discussions. Lecture is limited to 10-15 minute segments, mostly to introduce course material. Learning gains were measured using the Force Concept Inventory and Force and Motion Conceptual Evaluation. Student groups were regularly rearranged so that each group had students “from the top, middle, and bottom thirds of the class ranking” and that any women and minority students were not the only ones in their group. They found that when compared to traditional versions of introductory physics, SCALE-UP versions led to greater conceptual understanding, improved attitudes, drastically reduced course failure rates “especially for women and minorities” (pg 37), performance in second semester physics improved.

---

Huber, Bettina J., 2010. “Does Participation in Multiple High Impact Practices Affect Student Success at Cal State Northridge? Some Preliminary Insights” Northridge, CA: California State University-Northridge Office of Institutional Research.

National Survey of Student Engagement (NSSE) results from 863 graduating seniors at CSUN showed a correlation between HIP participation and higher GPA at exit and increased likelihood of graduating on time. Low-income students (Pell Grant recipients) and Latinx students had even higher GPA bump. Exit GPAs of Latinx and Pell students who didn’t participate in HIPs were lower than those of other students but if they participated in three or more HIPs their GPAs slightly exceeded other students.

---

Haak, D.C., HilleRisLambers, J., Pitre, E. and Freeman, S., 2011. Increased structure and active learning reduce the achievement gap in introductory biology. Science, 332(6034), pp.1213-1216.

“Highly structured” (daily and weekly practice with problem-solving, data analysis, higher-order cognitive skills) large-enrollment intro biology course for undergraduate majors at University of Washington improved learning for all students compared to low-structure (lecture intensive) version. There were disproportionately large benefits for students in their Educational Opportunity Program (many of whom are first-gen and from minority groups historically underrepresented in STEM).

---

Eddy, S.L. and Hogan, K.A., 2014. Getting under the hood: how and for whom does increasing course structure work?. CBE-Life Sciences Education, 13(3), pp.453-468.

Essentially a replication of the 2011 study above except that the researchers studied differences between a “low structure” (lecture intensive), “moderate structure” (weekly ungraded preparatory assignments, 15-40% of each class for in-class activities on questions that were similar to previous exam problems) and “high structure” (even more prep assignments and in-class activities) for at the University of North Carolina. The same instructor taught all of the different versions of this course. Total of about 2400 students over 4 years of the study. Failure rate went down for all students in the more structured courses compared to lecture intensive version. Students also reported a greater sense of classroom community. Black students participated in the lecture intensive class far less than other students did, but in the more structured course, they spoke in class as much as other students. Exam grades improved for everyone in the moderate structure course, but it increased even more for Black students. In fact, Black students in the structured course outperformed the majority students in the lecture version of the course. And, a similar thing was observed for first-generation students.

---

Laursen, S.L., Hassi, M.L., Kogan, M. and Weston, T.J., 2014. Benefits for women and men of inquiry-based learning in college mathematics: A multi-institution study. Journal for Research in Mathematics Education, 45(4), pp.406-418.

Over 3000 students across 100 different course sections in four colleges and universities were included in this study of “inquiry-based learning” (IBL) in mathematics classrooms. The students were all in a math or science major, excluding students who were preservice elementary or secondary teachers. Even though there was a range of different implementations of IBL, researchers found that students in IBL courses on average performed as well as or better than their non-IBL peers. IBL students also took as many or more math courses than non-IBL students, which seems to indicate that their interest in mathematics increased as well. Pre- and post-surveys of cognitive skills in mathematics, attitudes toward mathematics, and attitudes about collaboration in a math class. Women in non-IBL courses reported significant decreases in their confidence to pursue higher mathematics, whereas men in non-IBL courses reported an increase in their confidence. In contrast, women in IBL courses reported an increase in their confidence similar to that of men in non-IBL courses.

---

Winkelmes, M.A., Bernacki, M., Butler, J., Zochowski, M., Golanics, J. and Weavil, K.H., 2016. A Teaching Intervention that Increases Underserved College Students’ Success. Peer Review, 18(1/2).

These researchers set out to measure the effect of teachers providing two transparently designed, problem-based take-home assignments (as compared to their original versions) on first-year college students. (“Transparently designed” here means something specific to the training that the faculty received. They were trained to revise their assignments to be clearer about the purpose, tasks, and criteria for the assignments.) About 1,180 students taught by 35 faculty, 61 courses, 7 institutions were involved in the study. Because the courses spanned many different disciplines, the researchers relied mostly on self-report data from the students. “Students who received more transparency reported gains in three areas that are important predictors of students’ success: academic confidence, sense of belonging, and mastery of the skills that employers value most when hiring.” And what’s more, for first-generation, low-income, and underrepresented students, those reported benefits were larger.

---

Ballen, C.J., Wieman, C., Salehi, S., Searle, J.B. and Zamudio, K.R., 2017. Enhancing diversity in undergraduate science: Self-efficacy drives performance gains with active learning. CBE—Life Sciences Education, 16(4), 6 pages.

One of the first papers I’ve read that attempts to uncover why students historically underrepresented in STEM seem to benefit disproportionately from active learning pedagogies. Instructors compared traditional lecture and active learning versions of introductory evolutionary biology and biodiversity course by measured learning gains on course content, and students’ self-report of science self-efficacy and sense of social belonging. Historical performance differences between groups of students was erased in the active learning version of the course. All treatment students reported higher science self-efficacy, but structural equation modeling revealed that the increase in self-efficacy mediated the effect of active learning pedagogies on learning outcomes for underrepresented students only. In other words, one mechanism by which active learning pedagogies might help to produce more equitable outcomes is by helping historically underrepresented students experience more self-efficacy for learning STEM content.

---

Lape, N.K., Clark, C., Bassman, L., Spencer, M., Lee, A., Spjut, E. R., Dato, A. M., Palucki Blake, L., and Tsai, T. J., 2018. Erasing a Gender Gap in Performance in a Multidisciplinary Introductory Engineering Course. ASEE Collaborative Network for Engineering and Computing Diversity Conference, Paper ID #24241.

The Department of Engineering at Harvey Mudd College transformed an introductory engineering course from a lecture-based format to one incorporating best practices in engineering education: integration of theory and hands-on practice around a theme of underwater robotics, small-group in-class activities, content delivery via videos (flipped instruction). Mastery of course content was measured in both the original lecture-based course and the revised course via a pre/post content test; student attitudes were also measured using pre/post instruments. The results show a significant increase in learning and affective gains for all students. Furthermore, a gender disparity in final course grades disappeared in the revised course.

---

Casper, A.M., Eddy, S.L. and Freeman, S., 2019. True Grit: Passion and persistence make an innovative course design work. PLoS Biology, 17(7), p.e3000359.

This paper is a replication of the 2011 and 2014 studies above involving a “high-structure” course model in an introductory biology course. This time, the study was conducted at an open-access institution: Eastern Michigan University, a regional, public university that “admits almost all applicants to its undergraduate program.” After several course iterations, the researchers found a course structure that significantly lowered DFW rates from 48% to 25% and had a disproportionately beneficial impact on historically marginalized students–in this case, 85% of the students in the class self-identify as African American. This paper also demonstrates how important educational context is in this kind of research and that details really do matter.

---

Elli J. Theobald, Mariah J. Hill, Elisa Tran, Sweta Agrawal, E. Nicole Arroyo, Shawn Behling, Nyasha Chambwe, Dianne Laboy Cintrón, Jacob D. Cooper, Gideon Dunster, Jared A. Grummer, Kelly Hennessey, Jennifer Hsiao, Nicole Iranon, Leonard Jones, Hannah Jordt, Marlowe Keller, Melissa E. Lacey, Caitlin E. Littlefield, Alexander Lowe, Shannon Newman, Vera Okolo, Savannah Olroyd, Brandon R. Peecook, Sarah B. Pickett, David L. Slager, Itzue W. Caviedes-Solis, Kathryn E. Stanchak, Vasudha Sundaravardan, Camila Valdebenito, Claire R. Williams, Kaitlin Zinsli, Scott Freeman, 2020. Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences, DOI: 10.1073/pnas.1916903117.

AFAIK, this is the first meta-analysis of the effect of active learning pedagogies on existing achievement gaps between over-represented and under-represented students. This paper includes 41 separate studies involving 53,844 students. To achieve greater statistical power, they combined students with underrepresented racial or ethnic identities in STEM with low-income students into one group they call “minoritized groups in STEM” (MGS). On average, compared to traditional lecture courses, active learning pedagogies reduced achievement gaps between MGS and non-MGS students in examination scores by 33% and narrowed gaps in passing rates by 45%. Class size, course level, and discipline did not appear to be important factors. However, there were some variations among the studies: in some cases, active learning pedagogies exacerbated achievement gaps–the amount of active learning in STEM classes seems to be positively correlated with narrower achievement gaps. Additionally, the authors begin to theorize for why certain teaching practices seem to disproportionately benefit students from MGS. This is likely to be a paper that many of us will refer to for years to come, in addition to the previous 2014 meta-analysis led by Freeman. (Focus on achievement gaps is still a bit problematic, but I totally understand the need to do something within the current paradigm of STEM education.)

---

Please let me know if you encounter other research articles that provide evidence for specific teaching strategies having disproportionately positive outcomes for women and/or students historically exclude from STEM. I will add it to this list.