Launching ProvenTutoring

Today, my colleagues and I are launching www.ProvenTutoring.org. The purpose of this website is to help make certain that the opportunities made available by the American Rescue Plan and other funding actually have their intended impact on the school success of America’s children.

Websites are launched every day. Why is this one special? ProvenTutoring.org introduces the ProvenTutoring Coalition, a collaboration among 14 established tutoring programs, each of which has strong evidence of its effectiveness. Together, we are capable of disseminating proven tutoring on an unprecedented scale. Collectively, our goal is to serve 100,000 tutors throughout the U.S. with top-quality materials, software, and professional development. These 100,000 tutors, some working one-to-one and some one-to-small group (up to one to five), should be able to provide proven tutoring services to about four million children each year. Each of us has prepared ambitious scale-up plans to accomplish this. We can create the capacity to meet the obvious need. Our website is intended to help districts and schools locate and adopt proven, ready-to-go strategies appropriate to their children’s needs.

What we are doing has never been attempted before in the history of American education. The federal government, along with enlightened state and local governments, is making available substantial resources to combat COVID learning losses, or “unfinished learning.” That kind of thing has happened before. Governments are making recommendations for effective use of the money, more or less based on research. That has happened before. State and local educational leaders are scrambling to figure out how to make wise use of these resources. That has happened before.

But never before has government had available a set of proven, scalable solutions to the educational problems it is trying to solve. Not just general principles, but specific, proven, scalable solutions. If a state or local superintendent calls us, we will show up, and can offer multiple options, each of which is known to be effective at improving student learning.

It has long been known that tutoring is the most effective strategy for accelerating the learning of students struggling in reading and math. Every educator knows this, of course, because when their own children are struggling in school, they get them a tutor. The research just provides evidence to back up common sense. Not all tutoring is effective; it does matter how you do it, which is why it is essential that educators use proven approaches. But “tutoring works” is not big news.

Why, then, is tutoring not universal for students who need it?

1. Too expensive? Not anymore. At one time tutoring was thought to require a certified teacher and one-to-one. Yet there are now effective programs that use well-trained teaching assistants with college degrees and one-to-small group tutoring, at costs starting at $600 per student per year. And the American Rescue Plan and other funding now make it possible for schools to afford even the most expensive one-to-one programs.
2. Not available? Not anymore. By training tutors hired locally, quality tutoring programs can operate anywhere. Tutoring should be in-person, if at all possible, but online professional development can make tutoring available anywhere (and tutoring can, of course, be offered online if necessary).
3. Nothing succeeds at scale in education innovation? Not anymore. It is true that educational innovations that succeed at small scale all too often fail in scale-up. Yet the reason for failure at scale is that to go to scale, programs often have to reduce professional development, materials, and standards of implementation. Yet each member of ProvenTutoring has made a commitment to provide the amount and quality of service they did in the studies that established their effectiveness. Most programs in our coalition have worked in hundreds, some in thousands of schools coast to coast, outside of research.

For Whom Does This Matter?

If you are a parent, teacher, principal, superintendent, board member, or just a member of the public interested in quality education for all children, ProvenTutoring should matter to you. Knowing that American Rescue Plan money is flowing in large part to fund proven programs known to substantially improve learning should make you happy.

If you are concerned about equity in America, ProvenTutoring should matter to you. The students who have suffered most from COVID school closures are disadvantaged and minority students, increasing gaps that already existed. The impact of ProvenTutoring will help in narrow these gaps by moving struggling students toward grade level in reading and math, and this will have a particular benefit for disadvantaged and minority students.

If you are concerned about America’s place in the world economy, ProvenTutoring should matter to you. The mediocre performance of U.S. students on international assessment (like PISA) is mostly due to the number of students scoring at the lowest levels. The U.S. workforce needs to be capable through and through. ProvenTutoring will help ensure that all students reach their full potential, and that will boost our entire economy and society.

If you believe in evidence-based reform in education, then ProvenTutoring should matter to you. It is the first-ever attempt to use proven programs to move the needle at a national scale, to solve a problem of great national importance and interest. If we can make this work, it will do for education what penicillin did for biology: Out of the lab, into policy and practice. Whatever your area of focus in education, preschool to postgraduate, the success of ProvenTutoring should matter to you as a model of how research-to-practice can happen at scale.

Who Are Our Partners?

            Our partners are listed below. Check out ProvenTutoring.org starting next Monday to learn more about them.

Reading
Program	Grades	Organization
Enhanced Core Reading Instruction	K-2	U. of Oregon
Experience Corps	K-3	AARP
Future Forward	K-2	Education Analytics
Leveled Literacy Intervention (LLI)	K-2	Heineman, Fountas & Pinnell
Lindamood-Bell	1-3	Lindamood-Bell
Reading Corps	K-3	Reading & Math, Inc.
Reading Rescue	1	Benedict Silverman Foundation
Targeted Reading Instruction	K-2	U. of North Carolina
Tutoring With the Lightning Squad	K-3	Success for All Foundation
Reading, 3-5/6
ThemeReads	2-5	Success for All Foundation
Tutoring With the Lightning Squad	3-5	Success for All Foundation

Math
Program	Grades	Organization
Galaxy Math	1	Fuchs Math
Number Rockets	2-3	Fuchs Math
Whole Number Foundations (Roots, Fusion)	K-1	U. of Oregon
Math Corps	4-5	Reading & Math, Inc.
Fractions Face-Off	4	Fuchs Math
Pirate Math	3	Fuchs Math
Math Corps	6-8	Reading & Math, Inc.
Saga Math	9-10	Saga

            We are looking forward to working with federal, state, and local leaders, and with everyone with a stake in children’s success, to see that this extraordinary opportunity to make a substantial difference in the school success of millions of children will prevail. Please join us in this enterprise!

Photo credit: NASA

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

Building Back Better

Yesterday, President Joe Biden took his oath of office. He is taking office at one of the lowest points in all of American history. Every American, whatever their political beliefs, should be wishing him well, because his success is essential for the recovery of our nation.

In education, most schools remain closed or partially open, and students are struggling with remote learning. My oldest granddaughter is in kindergarten. Every school day, she receives instruction from a teacher she has never met. She has never seen the inside of “her school.” She is lucky, of course, because she has educators as grandparents (us), but it is easy to imagine the millions of kindergartners who do not even have access to computers, or do not have help in learning to read and learning mathematics. These children will enter first grade with very little of the background they need, in language and school skills as well as in content.

Of course, the problem is not just kindergarten. All students have missed a lot of school, and they will vary widely in their experiences during that time. Think of second graders who essentially missed first grade. Students who missed the year when they are taught biology. Students who missed the fundamentals of creative writing. Students who should be in Algebra 2, except that they missed Algebra 1.

Hopefully, providing vaccines as quickly as possible to school staffs will enable most schools to open this spring. But we have a long, long way to go to get back to normal, especially with disadvantaged students. We cannot just ask students on their first day back to open their math books to the page they were on in March, 2020, when school closed.

Students need to be assessed when they return, and if they are far behind in reading or math, given daily tutoring, one-to-one or one-to-small group. If you follow this blog, you’ve heard me carry on at length about this.

Tutoring services, using tutoring programs proven to be effective, will be of enormous help to students who are far behind grade level (here, here, here). But the recovery from Covid-19 school closures should not be limited to repairing the losses. Instead, I hope the Covid-19 crisis can be an opportunity to reconsider how to rebuild our school system to enhance the school success of all students.

If we are honest with ourselves, we know that schooling in America was ailing long before Covid-19. It wasn’t doing so badly for middle class children, but it was failing disadvantaged students. These very same students have suffered disproportionately from Covid-19. So in the process of bringing these children back into school, let’s not stop with getting back to normal. Let’s figure out how to create schools that use the knowledge we have gained over the past 20 years, and knowledge we can develop in the coming years, to transform learning for our most vulnerable children.

Building Back Better

Obviously, the first thing we have to do this spring is reopen schools and make them as healthy, happy, welcoming, and upbeat as possible. We need to make sure that schools are fully staffed and fully equipped. We do need to “build back” before we can “build back better.” But we cannot stop there. Below, I discuss several things that would greatly transform education for disadvantaged students.

1.  Tutoring

Yes, tutoring is the first thing we have to do to build better. Every child who is significantly below grade level needs daily one-to-small group or one-to-one tutoring, until they reach a pre-established level of performance, depending on grade level, in reading and math.

However, I am not talking about just any tutoring. Not all tutoring works. But there are many programs that have been proven to work, many times. These are the tutoring programs we need to start with as soon as possible, with adequate training resources to ensure student success.

Implementing proven tutoring programs on a massive scale is an excellent “build back” strategy, the most effective and cost-effective strategy we have. However, tutoring should also be the basis for a key “build better” strategy

2.  Establishing success as a birthright and ensuring it using proven programs of all kinds.

We need to establish adequate reading and mathematics achievement as the birthright of every child. We can debate about what that level might be, but we must hold ourselves accountable for the success of every child. And we need to accomplish this not just by using accountability assessments and hoping for the best, but by providing proven programs to all students who need them for as long as they need them.

As I’ve pointed out in many blogs (here, here, here), we now have many programs proven effective in rigorous experiments and known to improve student achievement (see www.evidenceforessa.org). Every child who is performing below level, and every school serving many children below grade level, should have resources and knowledge to adopt proven programs. Teachers and tutors need to be guaranteed sufficient professional development and in-class coaching to enable them to successfully implement proven programs. Years ago, we did not have sufficient proven programs, so policy makers kept coming up with evidence-free policies, which have just not worked as intended. But now, we have many programs ready for widespread dissemination. To build better, we have to use these tools, not return to near universal use of instructional strategies, materials, and technology that have never been successfully evaluated. Instead, we need to use what works, and to facilitate adoption and effective implementation of proven programs.

3.  Invest in development and evaluation of promising programs.

How is it that in a remarkably short time, scientists were able to develop vaccines for Covid-19, vaccines that promise to save millions of lives? Simple. We invested billions in research, development, and evaluations of alternative vaccines. Effective vaccines are very difficult to make, and the great majority failed.  But at this writing, two U.S. vaccines have succeeded, and this is a mighty good start. Now, government is investing massively in rigorous dissemination of these vaccines.

Total spending on all of education research dedicated to creating and evaluating educational innovations is a tiny fraction of what has been spent and will be spent on vaccines. But can you imagine that it is impossible to improve reading, math, science, and other outcomes, with clear goals and serious resources? Of course it could be done. A key element of “building better” could be to substantially scale up use of proven programs we have now, and to invest in new development and evaluation to make today’s best obsolete, replaced by better and better approaches. The research and evaluation of tutoring proves this could happen, and perhaps a successful rollout of tutoring will demonstrate what proven programs can do in education.

4.  Commit to Success

Education goes from fad to fad, mandate to mandate, without making much progress. In order to “build better,” we all need to commit to finding what works, disseminating it broadly, and then finding even better solutions, until all children are succeeding. This must be a long-term commitment, but if we are investing adequately and see that we are improving outcomes each year, then it is clear we can do it.            

With a change of administrations, we are going to hear a lot about hope. Hope is a good start, but it is not a plan. Let’s plan to build back better, and then for the first time in the history of education, make sure our solutions work, for all of our children.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

Is a National Tutoring Corps Affordable?

Tutoring is certainly in the news these days. The December 30 Washington Post asked its journalists to predict what the top policy issues will be for the coming year. In education, Laura Meckler focused her entire prediction on just one issue: Tutoring. In an NPR interview (Kelly, 2020) with John King, U. S. Secretary of Education at the end of the Obama Administration and now President of Education Trust, the topic was how to overcome the losses students are certain to have sustained due to Covid-19 school closures. Dr. King emphasized tutoring, based on its strong evidence base. McKinsey (Dorn et al., 2020) did a report on early information on how much students have lost due to the school closures and what to do about it. “What to do” primarily boiled down to tutoring. Earlier articles in Education Week (e.g., Sawchuk, 2020) have also emphasized tutoring as the leading solution. Two bills introduced in the Senate by Senator Coons (D-Delaware) proposed a major expansion of AmeriCorps, mostly to provide tutoring and school health aides to schools suffering from Covid-19 school closures.

            All of this is heartening, but many of these same sources are warning that all this tutoring is going to be horrifically expensive and may not happen because we cannot afford it. However, most of these estimates are based on a single, highly atypical example. A Chicago study (Cook et al., 2015) of a Saga (or Match Education) math tutoring program for ninth graders estimated a per-pupil cost of one-to-two tutoring all year of $3,600 per student, with an estimate that at scale, the costs could be as low as $2,500 per student. Yet these estimates are unique to this single program in this single study. The McKinsey report applied the lower figure ($2,500 per student) to cost out tutoring for half of all 55 million students in grades k-12. They estimated an annual cost of $66 billion, just for math tutoring!

            Our estimate is that the cost of a robust national tutoring plan would be more like $7.0 billion in 2021-2022. How could these estimates be so different?  First, the Saga study was designed as a one-off demonstration that disadvantaged students in high school could still succeed in math. No one expected that Saga Math could be replicated at a per-pupil cost of $3,600 (or $2,500). In fact, a much less expensive form of Saga Math is currently being disseminated. In fact, there are dozens of cost-effective tutoring programs widely used and evaluated since the 1980s in elementary reading and math. One is our own Tutoring With the Lightning Squad (Madden & Slavin, 2017), which provides tutors in reading for groups of four students and costs about $700 per student per year. There are many proven small-group tutoring programs known to make a substantial difference in reading or math performance, (see Neitzel et al., in press; Nickow et al., 2020; Pellegrini et al., in press). These programs, most of which use teaching assistants as tutors, cost more like $1,500 per student, on average, based on the average cost of five tutoring programs used in Baltimore elementary schools (Tutoring With the Lightning Squad, Reading Partners, mClass Tutoring, Literacy Lab, and Springboard).

            Further, it is preposterous to expect to serve 27.5 million students (half of all students in k-12) all in one year. At 40 students per tutor, this would require hiring 687,500 tutors!

            Our proposal (Slavin et al., 2020) for a National Tutoring Corps proposes hiring 100,000 tutors by September, 2021, to provide proven one-to-one or (mostly) one-to-small group tutoring programs to about 4 million grade 1 to 9 students in Title I schools. This number of tutors would serve about 21% of Title I students in these grades in 2021-2022, at a cost of roughly $7.0 billion (including administrative costs, development, evaluation, and so on). This is less than what the government of England is spending right now on a national tutoring program, a total of £1 billion, which translates to $7.8 billion (accounting for the differences in population).

            Our plan would gradually increase the numbers of tutors over time, so in later years costs could grow, but they would never surpass $10 billion, much less $66 billion just for math, as estimated by McKinsey.

            In fact, even with all the money in the world, it would not be possible to hire, train, and deploy 687,500 tutors any time soon, at least not tutors using programs proven to work. The task before us is not to just throw tutors into schools to serve lots of kids. Instead, it should be to provide carefully selected tutors with extensive professional development and coaching to enable them to implement tutoring programs that have been proven to be effective in rigorous, usually randomized experiments. No purpose is served by deploying tutors in such large numbers so quickly that we’d have to make serious compromises with the amount and quality of training. Poorly-implemented tutoring would have minimal outcomes, at best.

            I think anyone would agree that insisting on high quality at substantial scale, and then growing from success to success as tutoring organizations build capacity, is a better use of taxpayers’ money than starting too large and too fast, with unproven approaches.

            The apparent enthusiasm for tutoring is wonderful. But misplaced dollars will not ensure the outcomes we so desperately need for so many students harmed by Covid-19 school closures. Let’s invest in a plan based on high-quality implementation of proven programs and then grow it as we learn more about what works and what scales in sustainable forms of tutoring.

Photo credit: Deeper Learning 4 All, (CC BY-NC 4.0)

References

Cook, P. J., et al. (2016) Not too late: Improving academic outcomes for disadvantaged youth. Available at https://www.ipr.northwestern.edu/documents/working-papers/2015/IPR-WP-15-01.pdf

Dorn, E., et al. (2020). Covid-19 and learning loss: Disparities grow and students need help. New York: McKinsey & Co.

Kelly, M. L. (2020, December 28). Schools face a massive challenge to make up for learning lost during the pandemic. National Public Radio.

Madden, N. A., & Slavin, R. E. (2017). Evaluations of technology-assisted small-group tutoring for struggling readers. Reading & Writing Quarterly: Overcoming Learning Difficulties, 33(4), 327–334. https://doi.org/10.1080/10573569.2016.1255577

Neitzel, A., Lake, C., Pellegrini, M., & Slavin, R. (in press). A synthesis of quantitative research on programs for struggling readers in elementary schools. Reading Research Quarterly.

Nickow, A. J., Oreopoulos, P., & Quan, V. (2020). The transformative potential of tutoring for pre-k to 12 learning outcomes: Lessons from randomized evaluations. Boston: Abdul Latif Poverty Action Lab.

Pellegrini, M., Neitzel, A., Lake, C., & Slavin, R. (in press). Effective programs in elementary mathematics: A best-evidence synthesis. AERA Open.

Sawchuk, S. (2020, August 26). Overcoming Covid-19 learning loss. Education Week 40(2), 6.

Slavin, R. E., Madden, N. A., Neitzel, A., & Lake, C. (2020). The National Tutoring Corps: Scaling up proven tutoring for struggling students. Baltimore: Johns Hopkins University, Center for Research and Reform in Education.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

Large-Scale Tutoring Could Fail. Here’s How to Ensure It Does Not.

I’m delighted to see that the idea of large-scale tutoring to combat Covid-19 losses has gotten so important in the policy world that it is attracting scoffers and doubters. Michael Goldstein and Bowen Paulle (2020) published five brief commentaries recently in The Gadfly, warning about how tutoring could fail, both questioning the underlying research on tutoring outcomes (maybe just publication bias?) and noting the difficulties of rapid scale up. They also quote without citation a comment by Andy Rotherham, who quite correctly notes past disasters when government has tried and failed to scale up promising strategies: “Ed tech, class size reduction, teacher evaluations, some reading initiatives, and charter schools.” To these, I would add many others, but perhaps most importantly Supplementary Educational Services (SES), a massive attempt to implement all sorts of after school and summer school programs in high-poverty, low-achieving schools, which had near-zero impact, on average.

So if you were feeling complacent that the next hot thing, tutoring, was sure to work, no matter how it’s done, then you have not been paying attention for the past 30 years.

But rather than argue with these observations, I’d like to explain that the plan I’ve proposed, which you will find here, is fundamentally different from any of these past efforts, and if implemented as designed, with adequate funding, is highly likely to work at scale.

1.  Unlike all of the initiatives Rotherham dismisses, unlike SES, unlike just about everything ever used at scale in educational policy, the evidence base for certain specific, well-evaluated programs is solid.  And in our plan, only the proven programs would be scaled.

A little known but crucial fact: Not all tutoring programs work. The details matter. Our recent reviews of research on programs for struggling readers (Neitzel et al., in press) and math (Pellegrini et al., in press) identify individual tutoring programs that do and do not work, as well as types of tutoring that work well and those that do not.

Our scale-up plan would begin with programs that already have solid evidence of effectiveness, but it would also provide funding and third-party, rigorous evaluations of scaled-up programs without sufficient evidence, as well as new programs, designed to add additional options for schools. New and insufficiently evaluated programs would be piloted and implemented for evaluation, but they would not be scaled up unless they have solid evidence of effectiveness in randomized evaluations.

If possible, in fact, we would hope to re-evaluate even the most successful evaluated programs, to make sure they work.

If we stick to repeatedly-proven programs, rigorously evaluated in large randomized experiments, then who cares whether other programs have failed in the past? We will know that the programs being used at scale do work. Also, all this research would add greatly to knowledge about effective and ineffective program components and applications to particular groups of students, so over time, we’d expect the individual programs, and the field as a whole, to gain in the ability to provide proven tutoring approaches at scale.

2.  Scale-up of proven programs can work if we take it seriously. It is true that scale-up has many pitfalls, but I would argue that when scale-up does not occur it is for one of two reasons. First, the programs being scaled were not adequately proven in the first place. Second, the funding provided for scale-up was not sufficient to allow the program developers to scale up under the conditions they know full well are necessary. As examples of the latter, programs that provided well-trained and experienced trainers in their initial studies are often forced by insufficient funding to use trainer-of-trainers models for greatly diminished amounts of training in scale-up. As a result, the programs that worked at small scale failed in large-scale replication. This happens all the time, and this is what makes policy experts conclude that nothing works at scale.

However, the lesson they should have learned instead is just that programs proven to work at small scale can succeed if the key factors that made them work at small scale are implemented with fidelity at large scale. If anything less is done in scale-up, you’re taking big risks.

If well-trained trainers are essential, then it is critical to insist on well-trained trainers. If a certain amount or quality of training is essential, it is critical to insist on it, and make sure it happens in every school using a given program. And so on. There is no reason to skimp on the proven recipe.

But aren’t all these trainers and training days and other elements unsustainable?  This is the wrong question. The right one is, how can we make tutoring as effective as possible, to justify its cost?

Tutoring is expensive, but most of the cost is in the salaries of the tutors themselves. As an analogy, consider horse racing.  Horse owners pay millions for horses with great potential. Having done so, do you think they skimp on trainers or training? Of course not. In the same way, a hundred teaching assistants tutors cost roughly $4 million per year in salaries and benefits alone. Let’s say top-quality training for this group costs $500,000 per year, while crummy training costs $50,000. If these figures are in the ballpark, would it be wise to spend $4,500,000 on a terrific tutoring program, or $4,050,000 on a crummy one?

Successful scale-up takes place all the time in business. How does Starbucks make sure your experience in every single store is excellent? Simple. They have well-researched, well specified, obsessively monitored standards and quality metrics for every part of their operation. Scale-up in education can work just the same way, and in comparison to the costs of front-line personnel, the costs of great are trivially greater than the cost of crummy.

3.  Ongoing research will, in our proposal, formatively evaluate the entire tutoring effort over time, and development and evaluation will continually add new proven programs.  

Ordinarily, big federal education programs start with all kinds of rules and regulations and funding schemes, and these are announced with a lot of hoopla and local and national meetings to explain the new programs to local educators and leaders. Some sort of monitoring and compliance mechanism is put in place, but otherwise the program steams ahead. Several years later, some big research firm gets a huge contract to evaluate the program. On average, the result is almost always disappointing. Then there’s a political fight about just how disappointing the results are, and life goes on.

 The program we have proposed is completely different. First, as noted earlier, the individual programs that are operating at large scale will all be proven effective to begin with, and may be evaluated and proven effective again, using the same methods as those used to validate new programs. Second, new proven programs would be identified and scaled up all the time. Third, numerous studies combining observations, correlational studies, and mini-experiments would be evaluating program variations and impacts with different populations and circumstances, adding knowledge of what is happening at the chalkface and of how and why outcomes vary. This explanatory research would not be designed to decide which programs work and which do not (that would be done in the big randomized studies), but to learn from practice how to improve outcomes for each type of school and application. The idea is to get smarter over time about how to make tutoring as effective as it can be, so when the huge summative evaluation takes place, there will be no surprises. We would already know what is working, and how, and why.

Our National Tutoring Corps proposal is not a big research project, or a jobs program for researchers. The overwhelming focus is on providing struggling students the best tutoring we know how to provide. But using a small proportion of the total allocation would enable us to find out what works, rapidly enough to inform practice. If this were all to happen, we would know more and be able to do more every year, serving more and more struggling students with better and better programs.

So rather than spending a lot of taxpayer money and hoping for the best, we’d make scale-up successful by using evidence at the beginning, middle, and end of the process, to make sure that this time, we really know what we are doing. We would make sure that effective programs remain successful at scale, rather than merely hoping they will.

References

Goldstein, M., & Paulle, B. (2020, Dec. 8) Vaccine-making’s lessons for high-dosage tutoring, Part 1. The Gadfly.

Goldstein, M., & Paulle, B. (2020, Dec. 11). Vaccine-making’s lessons for high-dosage tutoring, Part IV. The Gadfly.

Neitzel, A., Lake, C., Pellegrini, M., & Slavin, R. (in press). A synthesis of quantitative research on programs for struggling readers in elementary schools. Reading Research Quarterly.

Pellegrini, M., Neitzel, A., Lake, C., & Slavin, R. (in press). Effective programs in elementary mathematics: A best-evidence synthesis. AERA Open.

Original photo by Catherine Carusso, Presidio of Monterey Public Affairs

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

A “Called Shot” for Educational Research and Impact

In the 1932 World Series, Babe Ruth stepped up to the plate and pointed to the center field fence. Everyone there understood: He was promising to hit the next pitch over the fence.

And then he did.

That one home run established Babe Ruth as the greatest baseball player ever. Even though several others have long since beaten his record of 60 home runs, no one else ever promised to hit a home run and then did it.

Educational research needs to execute a “called shot” of its own. We need to identify a clear problem, one that must be solved with some urgency, one that every citizen understands and cares about, one that government is willing and able to spend serious money to solve. And then we need to solve it, in a way that is obvious to all. I think the clear need for intensive services for students whose educations have suffered due to Covid-19 school closures provides an opportunity for our own “called shot.”

In my recent Open Letter to President-Elect Biden, I described a plan to provide up to 300,000 well-trained college-graduate tutors to work with up to 12 million students whose learning has been devastated by the Covid-19 school closures, or who are far below grade level for any reason. There are excellent reasons to do this, including making a rapid difference in the reading and mathematics achievement of vulnerable children, providing jobs to hundreds of thousands of college graduates who may otherwise be unemployed, and starting the best of these non-certified tutors on a path to teacher certification. These reasons more than justify the effort. But in today’s blog, I wanted to explain a fourth rationale, one that in the long run may be the most important of all.

A major tutoring enterprise, entirely focusing on high-quality implementation of proven programs, could be the “called shot” evidence-based education needs to establish its value to the American public.

Of course, the response to the Covid-19 pandemic is already supporting a “called shot” in medicine, the rush to produce a vaccine. At this time we do not know what the outcome will be, but throughout the world, people are closely following the progress of dozens of prominent attempts to create a safe and effective vaccine to prevent Covid-19. If this works as hoped, this will provide enormous benefits for entire populations and economies worldwide. But it could also raise the possibility that we can solve many crucial medical problems much faster than we have in the past, without compromising on strict research standards. The funding of many promising alternatives, and rigorous testing of each before they are disseminated, is very similar to what I and my colleagues have proposed for various approaches to tutoring. In both the medical case and the educational case, the size of the problem justifies this intensive, all-in approach. If all goes well with the vaccines, that will be a “called shot” for medicine, but medicine has long since proven its capability to use science to solve big problems. Curing polio, eliminating smallpox, and preventing measles come to mind as examples. In education, we need to earn this confidence, with a “called shot” of our own.

Think of it. Education researchers and leaders who support them would describe a detailed and plausible plan to solve a pressing problem of education. Then we announce that given X amount of money and Y amount of time, we will demonstrate that struggling students can perform substantially better than they would have without tutoring.

We’d know this would work, because part of the process would be identifying a) programs already proven to be effective, b) programs that already exist at some scale that would be successfully evaluated, and c) newly-designed programs that would successfully be evaluated. In each case, programs would have to meet rigorous evaluation standards before qualifying for substantial scale-up. In addition, in order to obtain funding to hire tutors, schools would have to agree to ensure that tutors use the programs with an amount and quality of training, coaching, and support at least as good as what was provided in the successful studies.

Researchers and policy makers who believe in evidence-based reform could confidently predict substantial gains, and then make good on their promises. No intervention in all of education is as effective as tutoring. Tutoring can be expensive, but it does not require a lengthy, uncertain transformation of the entire school. No sensible researcher or reformer would think that tutoring is all schools should do to improve student outcomes, but tutoring should be one element of any comprehensive plan to improve schools, and it happens to respond to the needs of post-Covid education for something that can have a dramatic, relatively quick, and relatively reliable impact.

If all went well in a large-scale tutoring intervention, the entire field of research could gain new respect, a belief among educators and the public that outcomes could be made much better than they are now by systematic applications of research, development, evaluation, and dissemination.

It is important to note that in order to be perceived to work, the tutoring “called shot” need not be proven effective across the board. By my count, there are 18 elementary reading tutoring programs with positive outcomes in randomized evaluations (see below). Let’s say 12 of them are ready for prime time and are put to the test, and 5 of those work very well at scale. That would be a tremendous success, because if we know which five approaches worked, we could make substantial progress on the problem of elementary reading failure. Just as with Covid-19 vaccines, we shouldn’t care how many vaccines failed. All that matters is that one or more of them succeeds, and can then be widely replicated.

I think it is time to do something bold to capture people’s imaginations. Let’s (figuratively) point to the center field fence, and (figuratively) hit the next pitch over it. The conditions today for such an effort are as good as they will ever be, because of universal understanding that the Covid-19 school closures deserve extraordinary investments in proven strategies. Researchers working closely with educators and political leaders can make a huge difference. We just have to make our case and insist on nothing less than whatever it takes. If a “called shot” works for tutoring, perhaps we could use similar approaches to solve other enduring problems of education.

It worked for the Babe. It should work for us, too, with much greater consequences for our children and our society than a mere home run.

*  *  *

Note: A reader of my previous blog asked what specific tutoring programs are proven effective, according to our standards. I’ve listed below reading and math tutoring programs that meet our standards of evidence. I cannot guarantee that all of these programs would be able to go to scale. We are communicating with program providers to try to assess each program’s capacity and interest in going to scale. But these programs are a good place to start in understanding where things stand today.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

An Open Letter To President-Elect Biden: A Tutoring Marshall Plan To Heal Our Students

Dear President-Elect Biden:

            Congratulations on your victory in the recent election. Your task is daunting; so much needs to be set right. I am writing to you about what I believe needs to be done in education to heal the damage done to so many children who missed school due to Covid-19 closures.

            I am aware that there are many basic things that must be done to improve schools, which have to continue to make their facilities safe for students and cope with the physical and emotional trauma that so many have experienced. Schools will be opening into a recession, so just providing ordinary services will be a challenge. Funding to enable schools to fulfill their core functions is essential, but it is not sufficient.

            Returning schools to the way they were when they closed last spring will not heal the damage students have sustained to their educational progress. This damage will be greatest to disadvantaged students in high-poverty schools, most of whom were unable to take advantage of the remote learning most schools provided. Some of these students were struggling even before schools closed, but when they re-open, millions of students will be far behind.

            Our research center at Johns Hopkins University studies the evidence on programs of all kinds for students who are at risk, especially in reading (Neitzel et al., 2020) and mathematics (Pellegrini et al., 2020). What we and many other researchers have found is that the most effective strategy for struggling students, especially in elementary schools, is one-to-one or one-to-small group tutoring. Structured tutoring programs can make a large difference in a short time, exactly what is needed to help students quickly catch up with grade level expectations.

A Tutoring Marshall Plan

            My colleagues and I have proposed a massive effort designed to provide proven tutoring services to the millions of students who desperately need it. Our proposal, based on a similar idea by Senator Coons (D-Del), would ultimately provide funding to enable as many as 300,000 tutors to be recruited, trained in proven tutoring models, and coached to ensure their effectiveness. These tutors would be required to have a college degree, but not necessarily a teaching certificate. Research has found that such tutors, using proven tutoring models with excellent professional development, can improve the achievement of students struggling in reading or mathematics as much as can teachers serving as tutors.

            The plan we are proposing is a bit like the Marshall Plan after World War II, which provided substantial funding to Western European nations devastated by the war. The idea was to put these countries on their feet quickly and effectively so that within a brief period of years, they could support themselves. In a similar fashion, a Tutoring Marshall Plan would provide intensive funding to enable Title I schools nationwide to substantially advance the achievement of their students who suffered mightily from Covid-19 school closures and related trauma. Effective tutoring is likely to enable these children to advance to the point where they can profit from ordinary grade-level instruction. We fear that without this assistance, millions of children will never catch up, and will show the negative effects of the school closures throughout their time in school and beyond.

            The Tutoring Marshall Plan will also provide employment to 300,000 college graduates, who will otherwise have difficulty entering the job market in a time of recession. These people are eager to contribute to society and to establish professional careers, but will need a first step on that ladder. Ideally, the best of the tutors will experience the joys of teaching, and might be offered accelerated certification, opening a new source of teacher candidates who will have had an opportunity to build and demonstrate their skills in school settings. Like the CCC and WPA programs in the Great Depression, these tutors will not only be helped to survive the financial crisis, but will perform essential services to the nation while building skills and confidence.

            The Tutoring Marshall Plan needs to start as soon as possible. The need is obvious, both to provide essential jobs to college graduates and to provide proven assistance to struggling students.

            Our proposal, in brief, is to ask the U.S. Congress to fund the following activities:

Spring, 2021

• Fund existing tutoring programs to build capacity to scale up their programs to serve thousands of struggling students. This would include funds for installing proven tutoring programs in about 2000 schools nationwide.
• Fund rigorous evaluations of programs that show promise, but have not been evaluated in rigorous, randomized experiments.
• Fund the development of new programs, especially in areas in which there are few proven models, such as programs for struggling students in secondary schools.

Fall, 2021 to Spring, 2022

• Provide restricted funds to Title I schools throughout the United States to enable them to hire up to 150,000 tutors to implement proven programs, across all grade levels, 1-9, and in reading and mathematics. This many tutors, mostly using small-group methods, should be able to provide tutoring services to about 6 million students each year. Schools should be asked to agree to select from among proven, effective programs. Schools would implement their chosen programs using tutors who have college degrees and experience with tutoring, teaching, or mentoring children (such as AmeriCorps graduates who were tutors, camp counselors, or Sunday school teachers).
• As new programs are completed and piloted, third-party evaluators should be funded to evaluate them in randomized experiments, adding to capacity to serve students in grades 1-9. Those programs that produce positive outcomes would then be added to the list of programs available for tutor funding, and their organizations would need to be funded to facilitate preparation for scale-up.
• Teacher training institutions and school districts should be funded to work together to design accelerated certification programs for outstanding tutors.

Fall, 2022-Spring, 2023

• Title I schools should be funded to enable them to hire a total of 300,000 tutors. Again, schools will select among proven tutoring programs, which will train, coach, and evaluate tutors across the U.S. We expect these tutors to be able to work with about 12 million struggling students each year.
• Development, evaluation, and scale-up of proven programs should continue to enrich the number and quality of proven programs adapted to the needs of all kinds of Title I schools.

            The Tutoring Marshall Plan would provide direct benefits to millions of struggling students harmed by Covid-19 school closures, in all parts of the U.S. It would provide meaningful work with a future to college graduates who might otherwise be unemployed. At the same time, it could establish a model of dramatic educational improvement based on rigorous research, contributing to knowledge and use of effective practice. If all goes well, the Tutoring Marshall Plan could demonstrate the power of scaling up proven programs and using research and development to improve the lives of children.

References

Neitzel, A., Lake, C., Pellegrini, M., & Slavin, R. (2020). A synthesis of quantitative research on programs for struggling readers in elementary schools. Available at www.bestevidence.org. Manuscript submitted for publication.

Pellegrini, M., Inns, A., Lake, C., & Slavin, R. (2020). Effective programs in elementary mathematics: A best-evidence synthesis. Available at www.bestevidence.com. Manuscript submitted for publication.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

Handling Outbreaks after COVID-19 Re-openings: The Case of Germany

By guest blogger Nathan Storey*

As schools across the U.S. are beginning to reopen in hybrid or full formats, unanticipated outbreaks of COVID are bound to occur. To help schools prepare, we have been writing about strategies schools and districts in other countries have used to combat outbreaks.

In this week’s case study, I examine how Germany has responded to outbreaks and managed school reopening nationwide.

Germany

Over one month since reopening after the summer holiday, German schools are largely still open. Critics and health experts worried in the early weeks as cases in the country appeared to increase (Morris & Weber-Steinhaus, 2020), but schools have been able to continue to operate. Now students sit in classes without masks, and children are allowed to move and interact freely on the playground.

Immediately following the reopening, 31 outbreak clusters (150 cases) were identified in the first week of schooling, and 41 schools in Berlin (out of 825 schools in the region) experienced COVID-19 cases during the first two weeks of schooling, requiring quarantines, testing, and temporary closures. Similar issues occurred across the country as schools reopened in other states. Mecklenburg-Western Pomerania, the first state to reopen, saw 800-plus students from Goethe Gymnasium in Ludwigslust sent home for quarantine after a faculty member tested positive. One hundred primary school students in Rostock district were quarantined for two weeks when a fellow student tested positive. Yet now one month later, German schools remain open. How is this possible?

Germany has focused its outbreak responses on individual student and class-level quarantines instead of shutting down entire schools. Due to active and widespread testing nationwide in the early stages of the outbreak, the country was able to get control of community-level positivity rates, paving the way for schools to reopen both in the spring, and again after summer break. Rates rose in August, but tracking enabled authorities to trace the cases to people returning from summer vacation, not from schools. At schools, outbreaks have generally been limited to one teacher or one student, who have contracted the virus from family or community members, not from within the school.

When these outbreaks occur, schools close for a day awaiting test results, but reopen quickly once affected individuals are tested negative and can return to class. At Sophie-Charlotte High School in Berlin, three days after reopening, the school received word that two students tested positive from the girls’ parents. The school in turn informed the local health authority, leading to 191 students and teachers asked to quarantine at home. Everyone was tested and two days later they received their test results. Before the week was up, school was back in session. By one estimate, due to the efficient testing and individual or class quarantines, fewer than 600 Berliner students have had to stay home for a day (out of more than 366,000 students) (Bennhold, 2020).

So far, there has been one more serious outbreak at Heinrich Hertz School in Hamburg, where a cluster of 26 students and three teachers have all received positive diagnoses, potentially infected by one of the teachers. The school moved to quarantine grades six and eight, and mask wearing rules were more strictly followed. The school and local health authorities are continuing to study the potential transmission patterns to locate the origin of the cluster.

Testing in Germany is effective because it is extensive, but targeted to those with direct contact with infections. At Heinz-Berggruen school in Berlin, a sixth grader was found to be infected after being tested even though she had no symptoms. Someone in her family had tested positive. Tracing the family member’s contacts, tests determined the source of the infection stemmed from international travel, and Heinz-Berggruen remained open, with just the infected student quarantined for two weeks. At Goethe Gymnasium in Ludwigslust, mentioned earlier, the infected teacher was sent home, and all 55 teachers were subsequently tested. The school was able to reopen less than a week later.

Some challenges have arisen. As in the US, German states are responsible for their own COVID-19 prevention measures and must make plans for the case of outbreaks. One city councilor in the Neukölln district of Berlin revealed there was confusion among parents and schools about children’s symptoms and response plans. As a result, children whose only symptoms are runny noses, for instance, have been sent home, and worries are increasing as to how effectively schools and districts will differentiate COVID-19 from flu in the winter.

The German case provides some optimism that schools can manage outbreaks and reopen successfully through careful planning and organization. Testing, contact tracing, and communication are vital, as is lowering of community positivity rates. Cases may be rising in Germany again (Loxton, 2020), but with these strategies and new national COVID management rules in place, the country is in an excellent position to address the challenge.

*Nathan Storey is a graduate student at the Johns Hopkins University School of Education

References

Barton, T., & Parekh, A. (2020, August 11). Reopening schools: Lessons from abroad. https://doi.org/10.26099/yr9j-3620

(2020, June 12). As Europe reopens schools, relief combines with risk. The New York Times. https://www.nytimes.com/2020/05/10/world/europe/reopen-schools-germany.html

Bennhold, K. (2020, August 26). Germany faces a ‘roller coaster’ as schools reopen amid Coronavirus—The New York Times. https://www.nytimes.com/2020/08/26/world/europe/germany-schools-virus-reopening.html?smid=em-share

Holcombe, M. (2020, October 5). New York City to close schools in some areas as Northeast sees rise in new cases. CNN. https://www.cnn.com/2020/10/05/health/us-coronavirus-monday/index.html

Loxton, R. (2020, October 15). What you need to know about Germany’s new coronavirus measures for autumn. The Local. https://www.thelocal.de/20201015/what-you-need-to-know-about-germanys-new-coronavirus-measures-for-autumn-and-winter

Medical Xpress. (2020, August 7). Germany closes two schools in new virus blow. https://medicalxpress.com/news/2020-08-germany-schools-virus.html

Morris, L., & Weber-Steinhaus, F. (2020, September 11). Schools have seen no coronavirus outbreaks since reopening a month ago in Germany—The Washington Post. https://www.washingtonpost.com/world/europe/covid-schools-germany/2020/09/10/309648a4-eedf-11ea-bd08-1b10132b458f_story.html

Noryskiewicz, A. (2020, August 25). Coronavirus data 2 weeks into Germany’s school year “reassures” expert. https://www.cbsnews.com/news/coronavirus-school-germany-no-outbreaks/

The Associated Press (2020, August 27). Europe is going back to school despite recent virus surge—Education Week. AP. http://www.edweek.org/ew/articles/2020/08/27/europe-is-going-back-to-school_ap.html?cmp=eml-enl-eu-news2&M=59665135&U=&UUID=4397669ca555af41d7b271f2dafac508

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

How Much Have Students Lost in The COVID-19 Shutdowns?

Everyone knows that school closures due to the COVID-19 pandemic are having a serious negative impact on student achievement, and that this impact is sure to be larger for disadvantaged students than for others. However, how large will the impact turn out to be? This is not a grim parlor game for statisticians, but could have real meaning for policy and practice. If the losses turn out to be modest comparable to the “summer slide” we are used to (but which may not exist), then one might argue that when schools open, they might continue where they left off, and students might eventually make up their losses, as they do with summer slide. If, on the other hand, losses are very large, then we need to take emergency action.

Some researchers have used data from summer losses and from other existing data on, for example, teacher strikes, to estimate COVID losses (e.g., Kuhfeld et al., 2020). But now we have concrete evidence, from a country similar to the U.S. in most ways.

A colleague came across a study that has, I believe, the first actual data on this question. It is a recent study from Belgium (Maldonado & DeWitte, 2020) that assessed COVID-19 losses among Dutch-speaking students in that country.

The news is very bad.

The researchers obtained end-of-year test scores from all sixth graders who attend publicly-funded Catholic schools, which are attended by most students in Dutch-speaking Belgium. Sixth grade is the final year of primary school, and while schools were mostly closed from March to June due to COVID, the sixth graders were brought back to their schools in late May to prepare for and take their end-of primary tests. Before returning, the sixth graders had missed about 30% of the days in their school year. They were offered on-line teaching at home, as in the U.S.

The researchers compared the June test scores to those of students in the same schools in previous years, before COVID. After adjustments for other factors, students scored an effect size of -0.19 in mathematics, and -0.29 in Dutch (reading, writing, language). Schools serving many disadvantaged students had significantly larger losses in both subjects; inequality within the schools increased by 17% in mathematics and 20% in Dutch, and inequality between schools increased by 7% in math and 18% in Dutch.

There is every reason to expect that the situation in the U.S. will be much worse than that in Belgium. Most importantly, although Belgium had one of the worst COVID-19 death rates in the world, it has largely conquered the disease by now (fall), and its schools are all open. In contrast, most U.S. schools are closed or partially closed this fall. Students are usually offered remote instruction, but many disadvantaged students lack access to technology and supervision, and even students who do have equipment and supervision do not seem to be learning much, according to anecdotal reports.

In many U.S. schools that have opened fully or partially, outbreaks of the disease are disrupting schooling, and many parents are refusing to send their children to school. Although this varies greatly by regions of the U.S., the average American student is likely to have missed several more effective months of in-person schooling by the time schools return to normal operation.

But even if average losses turn out to be no worse than those seen in Belgium, the consequences are terrifying, for Belgium as well as for the U.S. and other COVID-inflicted countries.

Effect sizes of -0.19 and -0.29 are very large. From the Belgian data on inequality, we might estimate that for disadvantaged students (those in the lowest 25% of socioeconomic status), losses could have been -0.29 in mathematics and -0.39 in Dutch. What do we have in our armamentarium that is strong enough to overcome losses this large?

In a recent blog, I compared average effect sizes from studies of various solutions currently being proposed to remedy students’ losses from COVID shutdowns: Extended school days, after-school programs, summer school, and tutoring. Only tutoring, both one-to-one and one-to-small group, in reading and mathematics, had an effect size larger than +0.10. In fact, there are several one-to-one and one-to-small group tutoring models with effect sizes of +0.40 or more, and averages are around +0.30. Research in both reading and mathematics has shown that well-trained teaching assistants using structured tutoring materials or software can obtain outcomes as good as those obtained by certified teachers as tutors. On the basis of these data, I’ve been writing about a “Marshall Plan” to hire thousands of tutors in every state to provide tutoring to students scoring far below grade level in reading and math, beginning with elementary reading (where the evidence is strongest).

I’ve also written about national programs in the Netherlands and in England to provide tutoring to struggling students. Clearly, we need a program of this kind in the U.S. And if our scores are like the Belgian scores, we need it as quickly as possible. Students who have fallen far below grade level cannot be left to struggle without timely and effective assistance, powerful enough to bring them at least to where they would have been without the COVID school closures. Otherwise, these students are likely to lose motivation, and to suffer lasting damage. An entire generation of students, harmed through no fault of their own, cannot be allowed to sink into failure and despair.

References

Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020). Projecting the potential impacts of COVID-19 school closures on academic achievement. (EdWorkingPaper: 20-226). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/cdrv-yw05

Maldonado, J. E., & DeWitte, K. (2020). The effect of school closures on standardized student test outcomes.Leuven, Belgium: University of Leuven.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

Healing Covid-19’s Educational Losses: What is the Evidence?

I’ve written several blogs (here, here, here, here, here, and here) on what schools can do when they finally open permanently, to remedy what will surely be serious harm to the educational progress of millions of students. Without doubt, the students who are suffering the most from lengthy school closures are disadvantaged students, who are most likely to lack access to remote technology or regular support when their schools have been closed.

 Recently, there have been several articles circulated in the education press (e.g., Sawchuk, 2020) and newsletters laying out the options schools might consider to greatly improve the achievement of students who lost the most, and are performing far behind grade level.

The basic problem is that if schools simply start off with usual teaching for each grade level, this may be fine for students at or just below grade level, but for those who are far below level, this is likely to add catastrophe to catastrophe. Students who cannot read the material they are being taught, or who lack the prerequisite skills for their grade level, will experience failure and frustration. So the challenge is to provide students who are far behind with intensive, additional services likely to quickly accelerate their progress, so that they can then profit from ordinary, at-grade-level lessons.

In the publications I’ve seen, there have been several solutions frequently put forward. I thought this might be a good time to review the most common prescriptions in terms of their evidence basis in rigorous experimental or quasi-experimental research.

Extra Time

One proposal is to extend the school day or school year to provide additional time for instruction. This sounds logical; if the problem is time out of school, let’s add time in school.

The effects of extra time depend, of course, on what schools provide during that additional time. Simply providing more clock hours in which typical instruction is provided makes little difference. For example, in a large Florida study (Figlio, Holden, & Ozek, 2018), high-poverty schools were given a whole hour every day for a year, for additional reading instruction. This had a small impact on reading achievement (ES=+0.09) at a cost of about $800 per student, or $300,000-$400,000 per school. Also, in a review of research on secondary reading programs by Baye, Lake, Inns & Slavin (2019), my colleagues and I examined whether remedial programs were more effective if they were provided during additional time (one class period a day more than what the control group received for one or more years) or if they were provided during regular class time (the same amount of time the control group also received). The difference was essentially zero. The extra time did not matter. What did matter was what the schools provided (here and here).

After-School Programs

Some sources suggest providing after-school programs for students experiencing difficulties. A review of research on this topic by Kidron & Lindsay (2014) examined effects of after-school programs on student achievement in reading and mathematics. The effects were essentially zero. One problem is that students often did not attend regularly, or were poorly motivated when they did attend.

Summer School

As noted in a recent blog, positive effects of summer school were found only when intensive phonics instruction was provided in grades K or 1, but even in these cases, positive effects did not last to the following spring. Summer school is also very expensive.

Tutoring

By far the most effective approach for students struggling in reading or mathematics is tutoring (see blogs here, here, and here). Outcomes for one-to-one or one-to-small group tutoring average +0.20 to +0.30 in both reading and mathematics, and there are several particular programs that routinely report outcomes of +0.40 or more. Using teaching assistants with college degrees as tutors can make tutoring very cost-effective, especially in small-group programs.

Whole-School Reforms

There are a few whole-school reforms that can have substantial impacts on reading and mathematics achievement. A recent review of our elementary school reform model, Success for All (Cheung et al., 2020), found an average effect size of +0.24 for all students across 17 studies, and an average of +0.54 for low achievers.

A secondary reform model called BARR has reported positive reading and mathematics outcomes for ninth graders (T. Borman et al., 2017)

Conclusion

Clearly, something needs to be done about students returning to in-person education who are behind grade level in reading and/or mathematics. But resources devoted to helping these students need to be focused on approaches proven to work. This is not the time to invest in plausible but unproven programs. Students need the best we have that has been repeatedly shown to work.

References

Baye, A., Lake, C., Inns, A., & Slavin, R. (2019). Effective reading programs for secondary students. Reading Research Quarterly, 54 (2), 133-166.

Borman, T., Bos, H., O’Brien, B. C., Park, S. J., & Liu, F. (2017). i3 BARR validation study impact findings: Cohorts 1 and 2. Washington, DC: American Institutes for Research.

Cheung, A., Xie, C., Zhang, T., Neitzel, A., & Slavin, R. E. (2020). Success for All: A quantitative synthesis of evaluations. Manuscript submitted for publication. (Contact us for a copy.)

Figlio, D. N., Holden, K. L., & Ozek, U. (2018). Do students benefit from longer school days? Regression discontinuity evidence from Florida’s additional hour of literacy instruction. Economics of Education Review, 67, 171-183. https://doi.org/10.1016/j.econedurev.2018.06.003

Kidron, Y., & Lindsay, J. (2014). The effects of increased learning time on student academic and nonacademic outcomes: Findings from a meta‑analytic review (REL 2014-015). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Appalachia.

Sawchuk, S. (2020, August 26). Overcoming Covid-19 learning loss. Education Week, 40 (2), 6.

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org. 

The Summertime Blues

            A long-ago rock song said it first: “There ain’t no cure for the summertime blues.”

            In the 1970s, Barbara Heyns (1978) discovered that over the summer, disadvantaged students lost a lot more of what they had learned in school than did advantaged students. Ever since then, educators have been trying to figure out how they could use time during the summer to help disadvantaged students catch up academically. I got interested in this recently because I have been trying to learn what kinds of educational interventions might be most impactful for the millions of students who have missed many months of school due to Covid-19 school closures. Along with tutoring and after school programs, summer school is routinely mentioned as a likely solution.

            Along with colleagues Chen Xie, Alan Cheung, and Amanda Neitzel, I have been looking at the literature on summer programs for disadvantaged students.

            There are two basic approaches to summer programs intended to help at-risk students. One of these, summer book reading, gives students reading assignments over the summer (e.g., Kim & Guryan, 2010). These generally have very small impacts, but on the other hand, they are relatively inexpensive.

            Of greater interest to the quest for powerful interventions to overcome Covid-19 learning losses are summer school programs in reading and mathematics. Studies of most of the summer school programs found they made little difference in outcomes. For example, an evaluation of a 5-week, six hour a day remedial program for middle school students found no significant differences in reading or math (Somers et al., 2015). However, there was one category of summer school programs that had at least a glimmer of promise. All three involved intensive, phonics-focused programs for students in kindergarten or first grade. Schachter & Jo (2005) reported substantial impacts of such a program, with a mean effect size of +1.16 on fall reading measures. However, by the following spring, a follow-up test showed a non-significant difference of +0.18. Zvoch & Stevens (2013), using similar approaches, found effect sizes of +0.60 for kindergarten and +0.78 for first grade. However, no measure of maintenance was reported. Borman & Dowling (2006) provided first graders with a 7-week reading-focused summer school. There were substantial positive effects by fall, but these disappeared by spring. The same students qualified for a second summer school experience after second grade, and this once again showed positive effects that faded by the following spring. There was no cumulative effect.

Because these studies showed no lasting impact, one might consider them a failure. However, it is important to note the impressive initial impacts, which might suggest that intensive reading instruction could be a part of a comprehensive approach for struggling readers in the early grades, if these gains were followed up during the school year with effective interventions. What summertime offers is an opportunity to use time differently (i.e., intensive phonics for young students who need it). It would make more sense to build on the apparent potential of focused summer school, rather than abandoning it based on its lack of long-term impacts.

            All by themselves, summer programs, based on the evidence we have so far “Ain’t no cure for the summertime blues.” But in next week’s blog, I discuss some ideas about how short-term interventions with powerful impacts, such as tutoring, pre-kindergarten,  and intensive phonics for students in grades K-1 in summer school, might be followed up with school-year interventions to produce long-term positive impacts. Perhaps summer school could be part of a cure for the school year blues.

References

Borman, G. D., & Dowling, Ν. M. (2006). Longitudinal achievement effects of multiyear summer school: Evidence from the Teach Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28, 25-48. doi:10.3102/01623737028001025

Heyns, B. (1978). Summer learning and the effect of schooling. New York: Academic Press.

Kim, J. S., & Guryan, J. (2010). The efficacy of a voluntary summer book reading intervention for low-income Latino children from language minority families. Journal of Educational Psychology, 102, 20-31. doi:10.1037/a0017270

Somers, M. A., Welbeck, R., Grossman, J. B., & Gooden, S. (2015). An analysis of the effects of an academic summer program for middle school students. Retrieved from ERIC website: https://files.eric.ed.gov/fulltext/ED558507.pdf

Schacter, J., & Jo, B. (2005). Learning when school is not in session: A reading summer day-camp intervention to improve the achievement of exiting first-grade students who are economically disadvantaged. Journal of Research in Reading, 28, 158-169. doi:10.1111/j.1467-9817.2005.00260.x

Zvoch, K., & Stevens, J. J. (2013). Summer school effects in a randomized field trial. Early Childhood Research Quarterly, 28(1), 24-32. doi:10.1016/j.ecresq.2012.05.002

_{Photo credit: American Education: Images of Teachers and Students in Action (CC BY-NC 4.0)}

This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.

Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.