Building Evidence Connecting Teaching Practices and More Equitable Student Outcomes (Continuously Updated)

In their paper “Active learning increases student performance in science, engineering, and mathematics,” Freeman, et al., suggest that we are seeing a new wave of “second-generation research” in the education literature that explores “which aspects of instructor behavior are most important for achieving the greatest gains with active learning, and elaborate on recent work indicating that underprepared and underrepresented students may benefit most from active [learning instructional] methods.”

Indeed, a growing body of research shows that there are specific teaching strategies that (on average) improve learning outcomes for all students and also (on average) improve learning outcomes disproportionately for students who have been historically excluded from STEM: women, African American, Hispanic/Latinx, Native American, first-generation, low-income students.

In this continuously updated blog post, I will try to maintain an annotated bibliography of such research. My goal is to provide higher education faculty and faculty developers with evidence to support teaching strategies that produce more equitable learning outcomes for all students, but particularly those who have been historically left out of STEM fields.

It must noted, however, that these learning gains from active learning pedagogies are not automatic. Much more research is needed to tease apart the nuances and conditions under which students who have been historically underrepresented in STEM benefit from active learning pedagogies. And in fact, please read this other post about research that provides more nuance on how active learning pedagogies don’t always lead to more equitable outcomes.

(Papers are listed in chronological order.)

Lorenzo, M., Crouch, C.H. and Mazur, E., 2006. Reducing the gender gap in the physics classroom. American Journal of Physics, 74(2), pp.118-122.

Multiple versions of an introductory calculus-based physics course for non-majors at Harvard were compared over a period of 8 years: (1) traditional lecture and recitation sections, (2) Peer Instruction with traditional recitation sections, and (3) Peer Instruction with cooperative problem solving activities during recitation sections. (It seems that there was continuous evolution of instructional practices throughout those 8 years, but the researchers groups students in these three buckets.) In Peer Instruction, the instructor alternates between giving short mini-lectures and facilitating small group student discussions on conceptual questions. Student learning gains were measured using the Force Concept Inventory. Learning gains increased with the uptake of more active learning pedagogies. In addition, performance differences between men and women decreased with the uptake of more active learning pedagogies. In version 3 of the course, there was no statistically significant difference in normalized learning gains between men and women, despite there being a difference in pre-test knowledge between men and women.

Beichner, R.J., Saul, J.M., Abbott, D.S., Morse, J.J., Deardorff, D., Allain, R.J., Bonham, S.W., Dancy, M.H. and Risley, J.S., 2007. The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project. Research-Based Reform of University Physics, 1(1), pp.2-39.

This report details the efforts of several groups of faculty to show that active learning pedagogies can be used in introductory physics classes with up to 100 students. The SCALE-UP approach involves using the bulk of class time for students in groups of 3-4 to work cooperatively on rich, computer-based activities and class discussions. Lecture is limited to 10-15 minute segments, mostly to introduce course material. Learning gains were measured using the Force Concept Inventory and Force and Motion Conceptual Evaluation. Student groups were regularly rearranged so that each group had students “from the top, middle, and bottom thirds of the class ranking” and that any women and minority students were not the only ones in their group. They found that when compared to traditional versions of introductory physics, SCALE-UP versions led to greater conceptual understanding, improved attitudes, drastically reduced course failure rates “especially for women and minorities” (pg 37), performance in second semester physics improved.

Huber, Bettina J., 2010. “Does Participation in Multiple High Impact Practices Affect Student Success at Cal State Northridge? Some Preliminary Insights” Northridge, CA: California State University-Northridge Office of Institutional Research.

National Survey of Student Engagement (NSSE) results from 863 graduating seniors at CSUN showed a correlation between HIP participation and higher GPA at exit and increased likelihood of graduating on time. Low-income students (Pell Grant recipients) and Latinx students had even higher GPA bump. Exit GPAs of Latinx and Pell students who didn’t participate in HIPs were lower than those of other students but if they participated in three or more HIPs their GPAs slightly exceeded other students.

Haak, D.C., HilleRisLambers, J., Pitre, E. and Freeman, S., 2011. Increased structure and active learning reduce the achievement gap in introductory biologyScience, 332(6034), pp.1213-1216.

“Highly structured” (daily and weekly practice with problem-solving, data analysis, higher-order cognitive skills) large-enrollment intro biology course for undergraduate majors at University of Washington improved learning for all students compared to low-structure (lecture intensive) version. There were disproportionately large benefits for students in their Educational Opportunity Program (many of whom are first-gen and from minority groups historically underrepresented in STEM).

Eddy, S.L. and Hogan, K.A., 2014. Getting under the hood: how and for whom does increasing course structure work?CBE-Life Sciences Education, 13(3), pp.453-468.

Essentially a replication of the 2011 study above except that the researchers studied differences between a “low structure” (lecture intensive), “moderate structure” (weekly ungraded preparatory assignments, 15-40% of each class for in-class activities on questions that were similar to previous exam problems) and “high structure” (even more prep assignments and in-class activities) for at the University of North Carolina. The same instructor taught all of the different versions of this course. Total of about 2400 students over 4 years of the study. Failure rate went down for all students in the more structured courses compared to lecture intensive version. Students also reported a greater sense of classroom community. Black students participated in the lecture intensive class far less than other students did, but in the more structured course, they spoke in class as much as other students. Exam grades improved for everyone in the moderate structure course, but it increased even more for Black students. In fact, Black students in the structured course outperformed the majority students in the lecture version of the course. And, a similar thing was observed for first-generation students.

Laursen, S.L., Hassi, M.L., Kogan, M. and Weston, T.J., 2014. Benefits for women and men of inquiry-based learning in college mathematics: A multi-institution studyJournal for Research in Mathematics Education, 45(4), pp.406-418.

Over 3000 students across 100 different course sections in four colleges and universities were included in this study of “inquiry-based learning” (IBL) in mathematics classrooms. The students were all in a math or science major, excluding students who were preservice elementary or secondary teachers. Even though there was a range of different implementations of IBL, researchers found that students in IBL courses on average performed as well as or better than their non-IBL peers. IBL students also took as many or more math courses than non-IBL students, which seems to indicate that their interest in mathematics increased as well. Pre- and post-surveys of cognitive skills in mathematics, attitudes toward mathematics, and attitudes about collaboration in a math class. Women in non-IBL courses reported significant decreases in their confidence to pursue higher mathematics, whereas men in non-IBL courses reported an increase in their confidence. In contrast, women in IBL courses reported an increase in their confidence similar to that of men in non-IBL courses.

Winkelmes, M.A., Bernacki, M., Butler, J., Zochowski, M., Golanics, J. and Weavil, K.H., 2016. A Teaching Intervention that Increases Underserved College Students’ SuccessPeer Review18(1/2).

These researchers set out to measure the effect of teachers providing two transparently designed, problem-based take-home assignments (as compared to their original versions) on first-year college students. (“Transparently designed” here means something specific to the training that the faculty received. They were trained to revise their assignments to be clearer about the purpose, tasks, and criteria for the assignments.) About 1,180 students taught by 35 faculty, 61 courses, 7 institutions were involved in the study. Because the courses spanned many different disciplines, the researchers relied mostly on self-report data from the students. “Students who received more transparency reported gains in three areas that are important predictors of students’ success: academic confidence, sense of belonging, and mastery of the skills that employers value most when hiring.” And what’s more, for first-generation, low-income, and underrepresented students, those reported benefits were larger.

Ballen, C.J., Wieman, C., Salehi, S., Searle, J.B. and Zamudio, K.R., 2017. Enhancing diversity in undergraduate science: Self-efficacy drives performance gains with active learning. CBE—Life Sciences Education, 16(4), 6 pages.

One of the first papers I’ve read that attempts to uncover why students historically underrepresented in STEM seem to benefit disproportionately from active learning pedagogies. Instructors compared traditional lecture and active learning versions of introductory evolutionary biology and biodiversity course by measured learning gains on course content, and students’ self-report of science self-efficacy and sense of social belonging. Historical performance differences between groups of students was erased in the active learning version of the course. All treatment students reported higher science self-efficacy, but structural equation modeling revealed that the increase in self-efficacy mediated the effect of active learning pedagogies on learning outcomes for underrepresented students only. In other words, one mechanism by which active learning pedagogies might help to produce more equitable outcomes is by helping historically underrepresented students experience more self-efficacy for learning STEM content.

Casper, A.M., Eddy, S.L. and Freeman, S., 2019. True Grit: Passion and persistence make an innovative course design work. PLoS Biology, 17(7), p.e3000359.

This papers is a replication of the 2011 and 2014 studies above involving a “high-structure” course model in an introductory biology course. This time, the study was conducted at an open-access institution: Eastern Michigan University, a regional, public university that “admits almost all applicants to its undergraduate program.” After several course iterations, the researchers found a course structure that significantly lowered DFW rates from 48% to 25% and had a disproportionately beneficial impact on historically marginalized students–in this case, 85% of the students in the class self-identify as African American. This paper also demonstrates how important educational context is in this kind of research and that details really do matter.

Please let me know if you encounter other research articles that provide evidence for specific teaching strategies having disproportionately positive outcomes for women and/or students historically exclude from STEM. I will add it to this list.

4 thoughts on “Building Evidence Connecting Teaching Practices and More Equitable Student Outcomes (Continuously Updated)

  1. Hi Darryl, This link – Huber, Bettina J. “Does Participation in Multiple High Impact Practices Affect Student Success at Cal State Northridge? Some Preliminary Insights” – isn’t working. Is there another way to access the article?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s