Healing Covid-19’s Educational Losses: What is the Evidence?

I’ve written several blogs (here, here, here, here, here, and here) on what schools can do when they finally open permanently, to remedy what will surely be serious harm to the educational progress of millions of students. Without doubt, the students who are suffering the most from lengthy school closures are disadvantaged students, who are most likely to lack access to remote technology or regular support when their schools have been closed.

 Recently, there have been several articles circulated in the education press (e.g., Sawchuk, 2020) and newsletters laying out the options schools might consider to greatly improve the achievement of students who lost the most, and are performing far behind grade level.

The basic problem is that if schools simply start off with usual teaching for each grade level, this may be fine for students at or just below grade level, but for those who are far below level, this is likely to add catastrophe to catastrophe. Students who cannot read the material they are being taught, or who lack the prerequisite skills for their grade level, will experience failure and frustration. So the challenge is to provide students who are far behind with intensive, additional services likely to quickly accelerate their progress, so that they can then profit from ordinary, at-grade-level lessons.

In the publications I’ve seen, there have been several solutions frequently put forward. I thought this might be a good time to review the most common prescriptions in terms of their evidence basis in rigorous experimental or quasi-experimental research.

Extra Time

One proposal is to extend the school day or school year to provide additional time for instruction. This sounds logical; if the problem is time out of school, let’s add time in school.

The effects of extra time depend, of course, on what schools provide during that additional time. Simply providing more clock hours in which typical instruction is provided makes little difference. For example, in a large Florida study (Figlio, Holden, & Ozek, 2018), high-poverty schools were given a whole hour every day for a year, for additional reading instruction. This had a small impact on reading achievement (ES=+0.09) at a cost of about $800 per student, or $300,000-$400,000 per school. Also, in a review of research on secondary reading programs by Baye, Lake, Inns & Slavin (2019), my colleagues and I examined whether remedial programs were more effective if they were provided during additional time (one class period a day more than what the control group received for one or more years) or if they were provided during regular class time (the same amount of time the control group also received). The difference was essentially zero. The extra time did not matter. What did matter was what the schools provided (here and here).

After-School Programs

Some sources suggest providing after-school programs for students experiencing difficulties. A review of research on this topic by Kidron & Lindsay (2014) examined effects of after-school programs on student achievement in reading and mathematics. The effects were essentially zero. One problem is that students often did not attend regularly, or were poorly motivated when they did attend.

Summer School

As noted in a recent blog, positive effects of summer school were found only when intensive phonics instruction was provided in grades K or 1, but even in these cases, positive effects did not last to the following spring. Summer school is also very expensive.

Tutoring

By far the most effective approach for students struggling in reading or mathematics is tutoring (see blogs here, here, and here). Outcomes for one-to-one or one-to-small group tutoring average +0.20 to +0.30 in both reading and mathematics, and there are several particular programs that routinely report outcomes of +0.40 or more. Using teaching assistants with college degrees as tutors can make tutoring very cost-effective, especially in small-group programs.

Whole-School Reforms

There are a few whole-school reforms that can have substantial impacts on reading and mathematics achievement. A recent review of our elementary school reform model, Success for All (Cheung et al., 2020), found an average effect size of +0.24 for all students across 17 studies, and an average of +0.54 for low achievers.

A secondary reform model called BARR has reported positive reading and mathematics outcomes for ninth graders (T. Borman et al., 2017)

Conclusion

Clearly, something needs to be done about students returning to in-person education who are behind grade level in reading and/or mathematics. But resources devoted to helping these students need to be focused on approaches proven to work. This is not the time to invest in plausible but unproven programs. Students need the best we have that has been repeatedly shown to work.

References

Baye, A., Lake, C., Inns, A., & Slavin, R. (2019). Effective reading programs for secondary students. Reading Research Quarterly, 54 (2), 133-166.

Borman, T., Bos, H., O’Brien, B. C., Park, S. J., & Liu, F. (2017). i3 BARR validation study impact findings: Cohorts 1 and 2. Washington, DC: American Institutes for Research.

Cheung, A., Xie, C., Zhang, T., Neitzel, A., & Slavin, R. E. (2020). Success for All: A quantitative synthesis of evaluations. Manuscript submitted for publication. (Contact us for a copy.)

Figlio, D. N., Holden, K. L., & Ozek, U. (2018). Do students benefit from longer school days? Regression discontinuity evidence from Florida’s additional hour of literacy instruction. Economics of Education Review, 67, 171-183. https://doi.org/10.1016/j.econedurev.2018.06.003

Kidron, Y., & Lindsay, J. (2014). The effects of increased learning time on student academic and nonacademic outcomes: Findings from a meta‑analytic review (REL 2014-015). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Appalachia.

Sawchuk, S. (2020, August 26). Overcoming Covid-19 learning loss. Education Week, 40 (2), 6.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org

The Summertime Blues

            A long-ago rock song said it first: “There ain’t no cure for the summertime blues.”

            In the 1970s, Barbara Heyns (1978) discovered that over the summer, disadvantaged students lost a lot more of what they had learned in school than did advantaged students. Ever since then, educators have been trying to figure out how they could use time during the summer to help disadvantaged students catch up academically. I got interested in this recently because I have been trying to learn what kinds of educational interventions might be most impactful for the millions of students who have missed many months of school due to Covid-19 school closures. Along with tutoring and after school programs, summer school is routinely mentioned as a likely solution.

            Along with colleagues Chen Xie, Alan Cheung, and Amanda Neitzel, I have been looking at the literature on summer programs for disadvantaged students.

            There are two basic approaches to summer programs intended to help at-risk students. One of these, summer book reading, gives students reading assignments over the summer (e.g., Kim & Guryan, 2010). These generally have very small impacts, but on the other hand, they are relatively inexpensive.

            Of greater interest to the quest for powerful interventions to overcome Covid-19 learning losses are summer school programs in reading and mathematics. Studies of most of the summer school programs found they made little difference in outcomes. For example, an evaluation of a 5-week, six hour a day remedial program for middle school students found no significant differences in reading or math (Somers et al., 2015). However, there was one category of summer school programs that had at least a glimmer of promise. All three involved intensive, phonics-focused programs for students in kindergarten or first grade. Schachter & Jo (2005) reported substantial impacts of such a program, with a mean effect size of +1.16 on fall reading measures. However, by the following spring, a follow-up test showed a non-significant difference of +0.18. Zvoch & Stevens (2013), using similar approaches, found effect sizes of +0.60 for kindergarten and +0.78 for first grade. However, no measure of maintenance was reported. Borman & Dowling (2006) provided first graders with a 7-week reading-focused summer school. There were substantial positive effects by fall, but these disappeared by spring. The same students qualified for a second summer school experience after second grade, and this once again showed positive effects that faded by the following spring. There was no cumulative effect.

Because these studies showed no lasting impact, one might consider them a failure. However, it is important to note the impressive initial impacts, which might suggest that intensive reading instruction could be a part of a comprehensive approach for struggling readers in the early grades, if these gains were followed up during the school year with effective interventions. What summertime offers is an opportunity to use time differently (i.e., intensive phonics for young students who need it). It would make more sense to build on the apparent potential of focused summer school, rather than abandoning it based on its lack of long-term impacts.

            All by themselves, summer programs, based on the evidence we have so far “Ain’t no cure for the summertime blues.” But in next week’s blog, I discuss some ideas about how short-term interventions with powerful impacts, such as tutoring, pre-kindergarten,  and intensive phonics for students in grades K-1 in summer school, might be followed up with school-year interventions to produce long-term positive impacts. Perhaps summer school could be part of a cure for the school year blues.

References

Borman, G. D., & Dowling, Ν. M. (2006). Longitudinal achievement effects of multiyear summer school: Evidence from the Teach Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28, 25-48. doi:10.3102/01623737028001025

Heyns, B. (1978). Summer learning and the effect of schooling. New York: Academic Press.

Kim, J. S., & Guryan, J. (2010). The efficacy of a voluntary summer book reading intervention for low-income Latino children from language minority families. Journal of Educational Psychology, 102, 20-31. doi:10.1037/a0017270

Somers, M. A., Welbeck, R., Grossman, J. B., & Gooden, S. (2015). An analysis of the effects of an academic summer program for middle school students. Retrieved from ERIC website: https://files.eric.ed.gov/fulltext/ED558507.pdf

Schacter, J., & Jo, B. (2005). Learning when school is not in session: A reading summer day-camp intervention to improve the achievement of exiting first-grade students who are economically disadvantaged. Journal of Research in Reading, 28, 158-169. doi:10.1111/j.1467-9817.2005.00260.x

Zvoch, K., & Stevens, J. J. (2013). Summer school effects in a randomized field trial. Early Childhood Research Quarterly, 28(1), 24-32. doi:10.1016/j.ecresq.2012.05.002

Photo credit: American Education: Images of Teachers and Students in Action (CC BY-NC 4.0)

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org