According to a 2013 joint report from the US Department of Education and the National Science Foundation, there are many ways to approach educational research and the effectiveness of education programs. In fact, there’s an entire “pipeline” from basic exploratory studies to robust efficacy studies. “However,” the report clarifies, “the reality of scientific investigation is more complicated, less orderly, and less linear than such a ‘pipeline’ suggests.”
Our research at HMH spans all parts of the pipeline and is developed with the understanding that the collective evidence of multiple studies paints a powerful picture. Math 180 is our program aimed at intervention students, or students who are performing two or more grades below their current grade level in math. As of this writing, we have 18 published reports on the program’s effectiveness, spanning different implementations and student populations. In this article, I take a closer look at one recent study for a glimpse into research at HMH and what it means to say that a program is research-based.
Overall Context
Math 180 has a sturdy foundation, with a research evidence base most recently updated in 2022 but reflecting work that started long before. It spans many aspects of developing a program aimed at improving learning outcomes for math intervention students, such as the theoretical foundations of rigor, data-driven instruction, learning with special populations, and family engagement, along with results from early efficacy research.
Over the 2020–21 school year, we partnered with Clewiston Middle School in Clewiston, Florida to further deepen our understanding of the program’s efficacy. During that time, seventh and eighth grade students scoring in the lowest quartile in math achievement were enrolled in Math 180, and the teachers who were implementing it received training and coaching to ensure the program was being used with fidelity. We examined the effect of Math 180 primarily in two ways: 1) school performance on state assessments, and 2) student performance on interim math assessments.
School Performance on State Assessments
In Florida, student math proficiency is measured using the Florida Statewide Assessment of Mathematics, or FSA. Math success is often discussed in terms of what percent of students scored satisfactory or better, or level 3+ on the FSA.
Largely due to the effects of the pandemic, between 2019 and 2021, the state saw a general trend downwards across middle school students. There was a 10% dip in seventh graders scoring level 3+ and a 9% dip for eighth graders. However, this was not the case for Clewiston Middle School, which implemented Math 180 during the 2020–21 school year. In the same time frame, the percent of seventh graders scoring level 3+ increased by 19% and the share of eighth graders increased by 28%. On the whole, school-wide math proficiency went up!
Student Performance on Interim Math Assessments
At the individual student level, all students additionally took an adaptive HMH-developed assessment for math ability and progress. Assessment results are provided to educators in Quantile measures, a well-established framework that can help to track student growth in math across grade levels and is often indicated using the letter “Q.” Given where students at Clewiston Middle School started at the beginning of the 2020–21 school year, when comparing against national norms, we would have expected seventh graders to grow by 88Q and eighth graders to grow by 72Q. However, according to the report, “Clewiston significantly outperformed those norms with average improvement of 253Q for 7th graders . . . and 176Q for 8th graders.”
Attributing Learning Outcomes to Interventions
The case study of Clewiston Middle School presents two broad pieces of evidence that Math 180 works. When looking at state assessments, math proficiency increased for students who were the most vulnerable to the effects of the pandemic. And when looking at individual student math assessments, the students’ Quantile measures improved beyond what would have been expected. When considered together, these results help to demonstrate a rationale that Math 180 is capable of improving student outcomes.
It is important to reiterate that research conclusions require a compelling story using a collection of evidence, and one case study is one piece of evidence, not conclusive proof. Attributing learning outcomes to a specific program or practice will always be hard to separate from other factors specific to the students and teachers involved. In the case of Math 180, for example, the program is multifaceted, comprising many different components, such as online instruction, consumable materials, assessment tools, educational games, and professional learning.
Growth Mindset and Learning
The Clewiston Middle School study took place during the 2020–2021 school year—a year that presented extraordinary challenges for both students and educators around the country and resulted in a greater number of students who needed math intervention. When designing Math 180, there was a particular focus on having learners build a growth mindset, or a belief that abilities and intelligence can be developed over time with effort. According to the program’s research evidence base, students with a growth mindset “show more resilience and, ultimately, effectiveness in their pursuit of mastery.”
One key figure in the development of research around mindset is Stanford University psychology professor Dr. Carol Dweck. Her work frequently focuses on academic achievement in K–12 students and was leveraged throughout the development of Math 180. However, her conclusions extend to learners in any setting. Many of the intervention teachers responsible for administering Math 180 were in their first year. Due to the “nature of the program itself,” according to the case study, novice teachers who may have begun teaching math with a more fixed mindset were trained on fostering a growth mindset, enabling them to “rise to this challenge” themselves. The benefits of a growth mindset apply to educators, too!
Further Research
The work is never done, and researchers at HMH continue to study the effects of Math 180. Studies are currently underway examining different implementation models, additional outcome measures, and deeper dives into specific disabilities such as dyscalculia. In addition, the Clewiston Middle School is ripe for a deeper investigation into how both students’ and educators’ mindsets change when using the program and how that impacts student success. In the meantime, if you were trying to raise math achievement for learners who are several years behind grade level, wouldn’t you like Math 180 on your side?
***
Looking to unlock mathematical learning in the students who need it most? Explore Math 180, our revolutionary approach to math intervention for students in Grades 5–12.
Get our free Math Intervention eBook.
