HMH Into Math ©2020 Grade 5 Florida Pilot

The focus of this six-week pilot study was the effectiveness of HMH Into Math ©2020, a mathematics program for students in Kindergarten to Grade 8, published by Houghton Mifflin Harcourt. This study included students in two Grade 5 classes and one teacher in Florida and focused on Module 5: Volume within the program.

The demographic profile of the participating school indicates that the percentage of non-Caucasian students is approximately 2% and the percentage of students enrolled in the National School Lunch Program is 63%.

The study was conducted with 38 students enrolled in grade 5. Only those students who took both a pretest and posttest were included in the data analysis. The teacher used the program for their math instruction five days per week for about 60 to 75 minutes per day. The program was being used by the teacher for the first time. The one teacher had at least five years of teaching experience.

Pretests and posttests were from HMH Into Math ©2020 and aligned to content within the instructional lessons taught within the module under study. In addition to an analysis of the gain scores for the total research sample, analyses were conducted separately for students with higher and lower pretest scores.

The average gain scores for the total group of students was statistically significant.

In addition, the average gain scores for the low- and high-scoring groups were statistically significant.

Houghton Mifflin Harcourt contracted with Educational Research Institute of America (ERIA) to conduct a six-week pilot study to evaluate the effectiveness of the HMH Into Math ©2020 program for Grade 5. The study compared performance on assessments administered to students at the beginning of March 2019 (pretest) to performance on assessments administered at the end of April 2019 (posttest).

The following research questions guided the design of the study and the data analyses:

• Does the implementation of one module in the HMH Into Math ©2020 program result in improved student mathematics achievement?

• Does the implementation of one module in HMH Into Math ©2020 lead to differential student achievement as a function of student ability level?

The design of the study called for the implementation of Module 5: Volume within the HMH Into Math ©2020 program for Grade 5 which took place over six weeks during the 2018-2019 school year. The school had not used the program prior to this time.

A total of one teacher in Florida participated in the study.

The teacher reported using the program five days a week for about 60 to 75 minutes per day. The one teacher had at least five years of teaching experience.

The study followed a pretest/posttest design to compare results associated with program use and evaluate instructional efficacy.

The HMH Into Math ©2020 program description according to Houghton Mifflin Harcourt’s website:

Into Math™ ©2020 supports students as they develop their conceptual understanding and grow into procedurally fluent mathematicians. Lessons begin with the opportunity for students to express what they already know and challenge themselves with what might be coming next. Question prompts throughout lessons check understanding and motivate students to start conversations about mathematics using mathematical language. With productive perseverance at the forefront, HMH Into Math features a unique learning arc in which every module weaves together conceptual understanding and procedural fluency, with application throughout, with a rigorous framework designed to boost students’ math thinking skills. Every lesson begins with rigor right from the start. Independent learning tasks encourage students to practice productive perseverance by jumping into a new challenge or working collaboratively to solve problems while teachers guide and differentiate math instruction.

The pretest and posttest used in the study aligned with the content of the HMH Into Math ©2020 Module 5: Volume used in the study.

The tests included both multiple-choice and open-ended written responses. The makeup of item types for both pretest and posttest were the same. Following is an outline of test content:

Table 1 provides the basic test statistics. The table shows that the reliabilities of the tests are high and provide adequate stability to assess mathematics achievement.

Table 2 provides the demographic characteristics of the school included in the study. It is important to note that the school data does not provide a description of the makeup of the classes that participated in the study. However, the data does provide a general description of the school and, thereby, an estimate of the makeup of the classes included in the study.

The percentage of students enrolled in free/reduced-price lunch programs was 63%. By comparison, the reported national average for students enrolled in free/reduced-price lunch programs in public schools is 52%.

The percentage of students classified as minority students (non-White) was 2%. By comparison, 48% of the students enrolled in U.S. public schools are classified as non-Caucasian.

Standard scores were used for all data analyses. Raw scores were converted to standard scores with a mean of 300 and a standard deviation of 50. Data analyses and descriptive statistics were computed using the students’ standard scores.

For most of the comparisons, paired comparison T-tests were used to determine if differences in pretest and posttest scores were significantly different. The comparisons were conducted for differences between the study HMH Into Math ©2020 pretest and the HMH Into Math ©2020 posttest. The ≤.05 level of significance was used as the level at which differences would be considered statistically significant.

In addition, effect size (Cohen’s d) was computed for each of the comparisons. This statistic provides an indication of the strength of the effect of the treatment regardless of the statistical significance. In addition to reporting the level considered to be substantively important, interpretations of effect sizes were calculated and can be interpreted using the following guidelines:

• .20 to .49 = small
• .50 to .79 = medium
• .80+ = large

Table 3 shows that the average scores of the 38 Grade 5 students participating in the study increased from pretesting to posttesting at a statistically significant level.

The total group of 38 Grade 5 students was divided into two approximately equal-sized groups based on their pretest scores. The 19 students scoring lowest on the pretest were considered lower-achieving mathematics students while the 19 students scoring highest on the pretest were considered higher-achieving mathematics students.

Table 4 shows that both groups made statistically significant gains.

Figure 1 provides a graphic representation of the gains achieved by the Grade 5 students. The Grade 5 students increased their average scores by 91 standard score points. The lower achieving mathematics students increased their scores 115 standard score points while the higher achieving mathematics students increased their scores by 66 standard score points.

This study sought to determine the effectiveness of the HMH Into Math ©2020 mathematics program by comparing growth on reliable and valid pretests and posttests. The study took place during a six-week period of time during the 2018-2019 academic year and was carried out in the state of Florida and included one school and two Grade 5 classes.

Sixty-three percent of the student population of the research school were eligible for free or reduced-price lunch programs. This compares to about 52% of the total U.S. population who are eligible for those programs. Two percent of the students in the research sample school were classified as non-Caucasian. The national percentage of non-Caucasian students is about 48%.

Two research questions guided the study and the conclusions:

• Does the implementation of one module in the HMH Into Math ©2020 program result in improved student mathematics achievement?

Student mathematics growth for the Grade 5 students was statistically significant. The effect sizes could be categorized as “large”.

• Does the implementation of one module in HMH Into Math ©2020 lead to differential student achievement as a function of student ability level?

Student mathematics growth for the high-achieving and low-achieving students was statistically significant. The effect sizes could be considered “large” for the lower pretest scoring group and the higher pretest scoring group.

Based on this study, both research questions can be answered positively.