Research Using MathByExample
Research Summary
from 2016-17, 2017-18 Random Assignment Studies
MathByExample was tested in a randomized trial across five districts in the 2016-17 and 2017-18 school years. Participating 5th grade teachers’ classrooms were randomized to either use or not use MathByExample assignments throughout the year as a supplement to their instruction and existing curriculum.
Evaluation of MathByExample was led by Drs. Julie Booth and Kelly McGinn. Results have been published in:
Impacts on 5th Grade Mathematics Knowledge
Thirty-five classrooms of 5th-grade students
(n = 709) were randomly assigned to either use or not use MathByExample assignments throughout the year. Four measures of grade-level mathematics knowledge were used:
- practice problem scores on assignments
- content knowledge scores
- released standardized test item scores
- district-administered standardized test scores
To account for how much each student used the MathByExample assignments during the year, a “dosage” score was calculated based on the number of MathByExample assignments attempted and self-explanation questions answered. Students who did not use the MathByExample assignments at all received a dosage score of zero.
The results are statistically significant for all measures when exploring the impact of MathByExample assignment dosage on students’ learning. The more MathByExample assignments students attempted, the more their practice problems, content knowledge, released standardized test items, and district-administered standardized test scores increased.
In addition, we found a statistically significant interaction between prior knowledge and intervention dosage on two outcome measures. For both students’
practice problem scores and
released standardized test item scores, as students'
prior knowledge scores decreased, the impact of intervention dosage on learning tended to increase. In other words, students with lower prior knowledge assessment scores at the beginning of the year were more likely to see bigger improvements in their mathematics scores as they attempted more assignments.
Source:
McGinn, K. M., Booth, J. L., Huyghe, A., Young, L. K., & Donovan, M. S. (2024). Improving 5th Grade Mathematics Knowledge with Worked Examples and Self-Explanation Prompts. The Journal of Experimental Education, 1-19.
Impacts on 5th Grade Standardized Assessments
A significant portion of study participants (344 5th grade students, 7 teachers, 6 schools) were from one urban district that used the iReady assessment three times during the school year and the PARCC assessment every spring to assess student mathematics knowledge. A sub-study explored student outcomes on these assessments. Students' i-Ready start-of-year (October) scores were used to control for prior knowledge (there was no statistically significant difference between randomly assigned groups of classes at baseline). The analyses accounted for the number of intervention items that students attempted (i.e., intervention dosage).
Controlling for prior knowledge and intervention dosage, analyses demonstrate statistically significant, positive results on i-Ready January and March scores, as well as PARCC Mathematics scores, for MathByExample students compared to their peers with the same teacher who did not use MathByExample assignments.
Impacts on i-Ready in 5th Grade
Students using MathByExample outperformed peers by about 3 points in January and 4 points in March, hp 2 =.34.
Impacts on PARCC in 5th Grade
Students using MathByExample outperformed peers by about 8 points on their Spring PARCC scores (Example group: EMM=723, SE=3.96; Control group: EMM=715, SE=4.13), hp 2 =.38.
