“There is much technique to master in creating educational programs, evaluating them, and fairly summarizing their effects. There is even more technique in implementing proven programs in schools and classrooms, and in creating policies to support use of proven programs. We must care about technique because we care about children. All of us go into education to solve real problems in real classrooms. That’s the structure we are all building together over many years. None of us will live to see our structure completed, because education keeps growing in techniques and capability. But it’s useful to stop from time to time and remember why we do what we do, and for whom.”
Today, my colleagues and I are launching www.ProvenTutoring.org. The purpose of this website is to help make certain that the opportunities made available by the American Rescue Plan and other funding actually have their intended impact on the school success of America’s children.
Websites are launched every day. Why is this one special? ProvenTutoring.org introduces the ProvenTutoring Coalition, a collaboration among 14 established tutoring programs, each of which has strong evidence of its effectiveness. Together, we are capable of disseminating proven tutoring on an unprecedented scale. Collectively, our goal is to serve 100,000 tutors throughout the U.S. with top-quality materials, software, and professional development. These 100,000 tutors, some working one-to-one and some one-to-small group (up to one to five), should be able to provide proven tutoring services to about four million children each year. Each of us has prepared ambitious scale-up plans to accomplish this. We can create the capacity to meet the obvious need. Our website is intended to help districts and schools locate and adopt proven, ready-to-go strategies appropriate to their children’s needs.
What we are doing has never been attempted before in the history of American education. The federal government, along with enlightened state and local governments, is making available substantial resources to combat COVID learning losses, or “unfinished learning.” That kind of thing has happened before. Governments are making recommendations for effective use of the money, more or less based on research. That has happened before. State and local educational leaders are scrambling to figure out how to make wise use of these resources. That has happened before.
But never before has government had available a set of proven, scalable solutions to the educational problems it is trying to solve. Not just general principles, but specific, proven, scalable solutions. If a state or local superintendent calls us, we will show up, and can offer multiple options, each of which is known to be effective at improving student learning.
It has long been known that tutoring is the most effective strategy for accelerating the learning of students struggling in reading and math. Every educator knows this, of course, because when their own children are struggling in school, they get them a tutor. The research just provides evidence to back up common sense. Not all tutoring is effective; it does matter how you do it, which is why it is essential that educators use proven approaches. But “tutoring works” is not big news.
Why, then, is tutoring not universal for students who need it?
Too expensive? Not anymore. At one time tutoring was thought to require a certified teacher and one-to-one. Yet there are now effective programs that use well-trained teaching assistants with college degrees and one-to-small group tutoring, at costs starting at $600 per student per year. And the American Rescue Plan and other funding now make it possible for schools to afford even the most expensive one-to-one programs.
Not available? Not anymore. By training tutors hired locally, quality tutoring programs can operate anywhere. Tutoring should be in-person, if at all possible, but online professional development can make tutoring available anywhere (and tutoring can, of course, be offered online if necessary).
Nothing succeeds at scale in education innovation? Not anymore. It is true that educational innovations that succeed at small scale all too often fail in scale-up. Yet the reason for failure at scale is that to go to scale, programs often have to reduce professional development, materials, and standards of implementation. Yet each member of ProvenTutoring has made a commitment to provide the amount and quality of service they did in the studies that established their effectiveness. Most programs in our coalition have worked in hundreds, some in thousands of schools coast to coast, outside of research.
For Whom Does This Matter?
If you are a parent, teacher, principal, superintendent, board member, or just a member of the public interested in quality education for all children, ProvenTutoring should matter to you. Knowing that American Rescue Plan money is flowing in large part to fund proven programs known to substantially improve learning should make you happy.
If you are concerned about equity in America, ProvenTutoring should matter to you. The students who have suffered most from COVID school closures are disadvantaged and minority students, increasing gaps that already existed. The impact of ProvenTutoring will help in narrow these gaps by moving struggling students toward grade level in reading and math, and this will have a particular benefit for disadvantaged and minority students.
If you are concerned about America’s place in the world economy, ProvenTutoring should matter to you. The mediocre performance of U.S. students on international assessment (like PISA) is mostly due to the number of students scoring at the lowest levels. The U.S. workforce needs to be capable through and through. ProvenTutoring will help ensure that all students reach their full potential, and that will boost our entire economy and society.
If you believe in evidence-based reform in education, then ProvenTutoring should matter to you. It is the first-ever attempt to use proven programs to move the needle at a national scale, to solve a problem of great national importance and interest. If we can make this work, it will do for education what penicillin did for biology: Out of the lab, into policy and practice. Whatever your area of focus in education, preschool to postgraduate, the success of ProvenTutoring should matter to you as a model of how research-to-practice can happen at scale.
Who Are Our Partners?
Our partners are listed below. Check out ProvenTutoring.org starting next Monday to learn more about them.
Reading
Program
Grades
Organization
Enhanced Core Reading Instruction
K-2
U. of Oregon
Experience Corps
K-3
AARP
Future Forward
K-2
Education Analytics
Leveled Literacy Intervention (LLI)
K-2
Heineman, Fountas & Pinnell
Lindamood-Bell
1-3
Lindamood-Bell
Reading Corps
K-3
Reading & Math, Inc.
Reading Rescue
1
Benedict Silverman Foundation
Targeted Reading Instruction
K-2
U. of North Carolina
Tutoring With the Lightning Squad
K-3
Success for All Foundation
Reading, 3-5/6
ThemeReads
2-5
Success for All Foundation
Tutoring With the Lightning Squad
3-5
Success for All Foundation
Math
Program
Grades
Organization
Galaxy Math
1
Fuchs Math
Number Rockets
2-3
Fuchs Math
Whole Number Foundations (Roots, Fusion)
K-1
U. of Oregon
Math Corps
4-5
Reading & Math, Inc.
Fractions Face-Off
4
Fuchs Math
Pirate Math
3
Fuchs Math
Math Corps
6-8
Reading & Math, Inc.
Saga Math
9-10
Saga
We are looking forward to working with federal, state, and local leaders, and with everyone with a stake in children’s success, to see that this extraordinary opportunity to make a substantial difference in the school success of millions of children will prevail. Please join us in this enterprise!
Photo credit: NASA
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
Every Thursday for the past nine years, except for major holidays, I have produced my blog on educational research and practice. However, this week, I am running my blog on Monday. Why? Because on Monday, we are launching our tutoring website, the one I’ve been hinting about for weeks. And for some technical reasons I do not even understand, the website has to go out on the same day as my blog. Yes, I should have scheduled the launch for a Thursday, but here we are.
I can guarantee you that this website will completely change your perception of life, the universe, and everything, or at the very least about tutoring.
After Monday, my blogs will return to normal. Perhaps even better than normal.
Since the beginning of the COVID-19 pandemic, more than 130 biotech companies have launched major efforts to develop and test vaccines. Only four have been approved so far (Pfizer, Moderna, Johnson & Johnson, and AstraZeneca). Among the others, many have outright failed, and others are considered highly unlikely. Some of the failed vaccines are small, fringe companies, but they also include some of the largest and most successful drug companies in the world: Merck (U.S.), Glaxo-Smith-Kline (U.K.), and Sanofi (France).
Kamala Harris gets her vaccine.
Photo courtesy of NIH
If no further companies succeed, the score is something like 4 successes and 126 failures. Based on this, is the COVID vaccine a triumph of science, or a failure? Obviously, if you believe that even one of the successful programs is truly effective, you would have to agree that this is one of the most extraordinary successes in the history of medicine. In less than one year, companies were able to create, evaluate, and roll out successful vaccines, already saving hundreds of thousands of lives worldwide.
Meanwhile, Back in Education . . .
The example of COVID vaccines contrasts sharply with the way research findings are treated in education. As one example, Borman et al. (2003) reviewed research on 33 comprehensive school reform programs. Only three of these had solid evidence of effectiveness, according to the authors (one of these was our program, Success for All; see Cheung et al., in press). Actually, few of the programs failed; most had just not been evaluated adequately. Yet the response from government and educational leaders was “comprehensive school reform doesn’t work” rather than, “How wonderful! Let’s use the programs proven to work.” As a result, a federal program supporting comprehensive school reform was canceled, use of comprehensive school reform plummeted, and most CSR programs went out of operation (we survived, just barely, but the other two successful programs soon disappeared).
Similarly, the What Works Clearinghouse, and our Evidence for ESSA website (www.evidenceforessa.org), are often criticized because so few of the programs we review turn out to have significant positive outcomes in rigorous studies.
The reality is that in any field in which rigorous experiments are used to evaluate innovations, most of the innovations fail. Mature science-focused fields, like medicine and agriculture, expect this and honor it, because the only way to prevent failures is to do no experiments at all, or only flawed experiments. Without rigorous experiments, we would have no reliable successes. Also, we learn from failures, as scientists are learning from the findings of the evaluations of all 130 of the COVID vaccines.
Unfortunately, education is not a mature science-focused field, and in our field, failure to show positive effects in rigorous experiments leads to cover-ups, despair, abandonment of proven and promising approaches, or abandonment of rigorous research itself. About 20 years ago, a popular federally-funded education program was found to be ineffective in a large, randomized experiment. Supporters of this program actually got Congress to enact legislation that forbade the use of randomized experiments to evaluate this program!
Research has improved in the past two decades, and acceptance of research has improved as well. Yet we are a long way from medicine, for example, which accepts both success and failure as part of a process of using science to improve health. In our field, we need to commit to broad scale, rigorous evaluations of promising approaches, wide dissemination of programs that work, and learning from experiments that do not (yet) show positive outcomes. In this way, we could achieve the astonishing gains that take place in medicine, and learn how to produce these gains even faster using all the knowledge acquired in experiments, successful or not.
References
Borman, G. D., Hews, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 12(2), 125-230.
Cheung, A., Xie, C., Zhang, T. & Slavin, R. E. (in press). Success for All: A quantitative synthesis of evaluations. Journal of Research on Educational Effectiveness.
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just enter your email address here.
Last July, I wrote a blog about a Year 6 (fifth grade) student in England, “Richard,” who just happened to transfer to one of our Success for All schools in spring, 2020. The school staff tested him. He had no reading skills at all. None.
Because they only had a few months to prepare him to go to secondary school, the staff decided to use our Tutoring With the Lightning Squad program with Richard for 90 minutes a day for three weeks (it’s usually used 30 minutes a day for at least 60 days). He gained 2.2 grade levels. To his delight, he could read The Hodgeheg, and was looking forward to reading Harry Potter books!
Photo credit: NorthBay Media Arts
Recently, I discovered that the gains Richard made were not unique to him.
Last summer, we were approached by John Erickson, the founder of NorthBay Education, which had a campus called NorthBay at the top of the Chesapeake. For many years, NorthBay has provided week-long outdoor education and social and emotional learning experiences for sixth graders across Maryland. However, due to Covid, this was impossible, so Erickson and NorthBay Director of Education, Rick Garber, wanted to use their staff and campus to provide an extended educational experience for students who were particularly vulnerable due to their life circumstances. Erickson and Garber wanted us (at Johns Hopkins University and the Success for All Foundation) to provide daily tutoring in reading to these students. We were delighted to agree, so in October, the program got under way. The students had to participate in remote instruction, like other Baltimore students, for six hours a day, and then had a half-hour tutoring session taught by NorthBay staff, trained by SFA coaches. The students stayed all week at the camp, and then went home over each weekend.
I can’t say that all went smoothly, but after a while NorthBay was operating well.
There were two sessions, October to January and February to June. Some students stayed for only the first session and were replaced by others. Others dropped out for a variety of reasons along the way.
I just received the test scores for the 31 students who were pretested in October and posttested on March 2, on the Gray Silent Reading Test (GSRT). Because of all the coming and going, we cannot say anything scientific about the data. We do not know if the gains were representative of all students who attended, and there was no control group. However, something extraordinary happened, and I wanted to share it.
At pretest, eight of the sixth grade students tested at the beginning first grade level (1.0), and one at 1.2. The average for these nine former non-readers at posttest was 4.1. That’s a gain of 3 grade levels in four months. Every non-reader but one reached a grade equivalent of at least 3.0. The exception got to 2.8. One got to 5.8, and one to 9.2!
I had been impressed that Richard, in England, went from zero to Hodgeheg in three weeks, but he was getting the equivalent of three tutoring sessions a day. In four months, with one session a day, every one of the nine non-reading NorthBay students went from zero to Hodgeheg, and some from zero to Harry Potter.
Other students also made astonishing gains. One went from 1.8 to 5.8. One, from 2.0 to 5.0. One from 3.0 to 7.5. One from 3.5 to 9.8.
There were many things going on in this experience, of course. The students were living at a beautiful, peaceful place for four months, with caring staff, good food, and great outdoor activities. They were experiencing the Baltimore City online curriculum, with good computer linkages and plenty of on-site assistance. Tutoring is not all they were getting.
The most important conclusion from the NorthBay experience is that these kids, mostly from very difficult backgrounds, could learn to read. The NorthBay experience suggests that they always could have learned to read under the right circumstances. Had they had the opportunity to make these gains earlier in elementary school, and reached the third grade level or better by, say, third grade, perhaps they would now be at grade level in sixth grade.
The message of NorthBay is that the problems kids face in learning to read are not usually due to anything wrong with the kids. It is not due to anything wrong with the capable and heroic teachers who have done their very best. It is the system, far beyond Baltimore, that does not allocate the funding needed to provide every struggling reader whatever is necessary to learn to read.
Today, Baltimore City Public Schools is about to receive significant funding from the American Rescue Plan Act and other sources. They cannot send everyone to NorthBay, though that would be wonderful. But they can use the new funds to create the teaching and tutoring resources that made such a difference. And so can any school or district in America.
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, send your email address to thebee@bestevidence.org.
In speaking and writing about tutoring, I am often asked about where limited tutoring resources should be concentrated. My answer is this: “Make certain that every single child in America who needs it gets enough tutoring to be proficient in phonics. All other priorities are tied for second place.”
There are many pragmatic reasons why early reading should be the focus. First, the largest number of proven tutoring programs focus on grades K-2 or K-3 reading, and these programs tend to have spectacular impacts on measures of reading comprehension as well as phonics. Second, reading is fundamental. Very few students end up in special education, or are held back in grade, due to failure in any other subject, for example. Most other subjects depend on reading, of course. But also, a student’s academic self-esteem depends primarily on his or her self-perception as a reader.
Note that “early reading” in this case does not only mean reading in grades K-3. The great majority of students who are failing in reading in grades 4-12 do not have solid phonics skills, typically taught in grades K-2. There are no proven middle school reading programs in the U.S., but there are two in England, and both were adapted from K-2 programs, with a strong focus on phonics.
My point about focusing on phonics first is influenced by the fact that very poor reading skills (defined here as scoring “below basic” on NAEP) are very widespread, especially among disadvantaged and minority students. On the 2019 4th grade NAEP, 34% of all students, but 52% of Black students and 45% of Hispanic students, scored below basic. At 8th grade, it’s 27% below basic for all, 46% for Black students, and 37% for Hispanic students. Using these numbers, I’d estimate that 9.5 million elementary and 7.3 million secondary students are scoring below basic on NAEP reading, and surely need tutoring. While we wait for someone to create and evaluate secondary reading tutoring programs, could we start with the 9.5 million elementary students? This is a mighty big job in itself. We could also use proven upper-elementary tutoring programs to work with some proportion of secondary students reading far below grade level.
Even leaving aside the importance of immediate capacity, demonstrated impact, and the obvious importance of reading in elementary schools, consider the huge role of the reading gap on the most important social problem in our nation: Inequality by race and social class. In large part, racial and social class disparities cause the reading gap, but they are also caused by reading gaps. And it is possible to close the reading gaps, alongside other efforts focusing on closing economic, housing, criminal justice, and other gaps. If we made sure that every American child could read at the fifth grade level by fifth grade, the reading gap would be no longer be a major problem.
The numbers I have been citing are from 2019, before the pandemic. Now things are much worse, and there are surely a lot more than 9.5 million elementary students and 7.3 million secondary students who would score below “basic” on NAEP. Fortunately, the American Rescue Plan Act is providing substantial resources to combat Covid learning losses, and low achievement in general. Covid or no Covid, we have an opportunity and a clear pressing need to close the reading gap and provide a strong foundation for reading for today’s students.
Note that I do not mean to minimize the importance of mathematics, or of secondary school reading and math. These are crucial as well, and we need to solve these problems too. But first out of the gate, in 2021-2022, let’s make sure we use the proven tools we have at hand to solve the problem we happen to be best placed to solve, which happens to be the most important educational problem we face at this moment.
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
Over the past 20 years, there has been a major increase in the number of educational programs that have been developed, evaluated in rigorous (usually randomized) experiments, found to make a substantial difference in achievement, and then offered to schools by non-profit or for-profit organizations. Educators can easily find out about these proven programs in the federal What Works Clearinghouse, our own Evidence for ESSA website, and other sources. Yet very few of these, no matter how effective, have been widely adopted by schools. In 2015, the Every Student Succeeds Act (ESSA) defined what it means for a program to have strong, moderate, or promising evidence of effectiveness, and encouraged or even (in some cases) incentivized use of these programs. Yet even with this, few of the roughly 120,000 U.S. elementary and secondary schools regularly use any of the more than 120 proven reading or mathematics programs that meet the requirements of Evidence for ESSA and show significant positive effects.
The evidence-to-practice connection in education is much in contrast to that in, say, medicine, where medications and treatments of all kinds that have proven in rigorous experiments that they can cure or prevent diseases are usually widely adopted by medical practitioners. The recent, rapid development and successful evaluations of Covid-19 vaccines are much in the news right now, but dozens of new drugs and other treatments are validated every year, and are then adopted widely. It is certainly true that vastly more money is invested in the whole evidence-to-practice process in medicine than is true in education, but even when educational programs are found effective in randomized experiments like those required in medicine, these programs rarely enter large-scale use. Further, evidence-to-practice is common in many other fields, such as agriculture and technology. But not education.
How Tutoring Might Change Evidence-to-Practice in Education
One of the problems of evidence-to-practice in education is that we lack clear examples where programs were proven effective and then universally applied and found effective at scale. For example, evidence-to-practice was haphazard in medicine until the 20th century, when sulfa drugs, penicillin, morphine, and cures for polio, among many others, solved massive societal problems, and established the idea that research in medicine could truly bring about major change. These breakthroughs were explicitly engineered to solve health problems of great concern to the public, just as the Covid-19 vaccines were explicitly engineered to solve the pandemic.
In education, we face similar problems in the post-Covid period. Millions of students have fallen far behind due to the closure of their schools. All sorts of solutions have been proposed, but only one, tutoring, has both a solid and substantial research base and a significant number of proven, practical, cost-effective solutions.
Perhaps this is our penicillin/polio/Covid moment. We face a problem that no one can deny, of a desperate need to enable millions of students who have lost ground in the pandemic to rapidly gain in reading and math achievement. The American Rescue Plan Act (ARPA) has provided billions of dollars to solve the problem, so for once, lack of money is not an obstacle. Due to developments in educational research over the past 20 years, we now have a set of tutoring models that have been proven effective in randomized experiments. If these programs can be rapidly scaled up and applied to enough students to make a meaningful difference among struggling students nationwide, then this may serve as the example we need to establish that development, research, and dissemination can solve societal problems in education.
ProvenTutoring
My colleagues and I are organizing a demonstration exactly along these lines. Fourteen proven tutoring programs for reading and mathematics have formed a coalition, which we call ProvenTutoring. The purpose of the coalition is to make the case for proven tutoring programs, and then provide schools, districts, and states a choice of proven programs. Whichever programs are selected will then provide tutors with first-class professional development to ensure that tutoring is done well and that students receive the maximum benefit possible.
The fourteen programs had to be proven to succeed with college-educated teaching assistants as tutors (because requiring certified teachers would be impossible in a time of teacher shortages, and because evidence finds that well-supported teaching assistants get results as good as those obtained by certified teachers). Each program had to have capacity to go to substantial scale. We estimate that we can collectively serve 100,000 tutors who can serve about four million children, if all goes well.
ProvenTutoring will soon launch a website (which we will call ProvenTutoring.org) and a nationwide communications campaign to focus schools on proven tutoring as a key part of their post Covid-19 plans to combat learning losses. We will offer school districts webinars on proven tutoring, and allow users to explore specific programs to make informed choices among them. We will maintain rigorous standards of training and implementation, to make sure that the quality of implementation in practice, at scale, will be no less than the quality of implementation in the controlled experiments that established their impact.
How Successful Tutoring Could Lead to Support for Evidence-to-Practice
When districts, states, and the nation evaluate reading and math outcomes of struggling students in schools that adopted proven tutoring programs, in comparison to schools that used their ARPA money on other approaches, the outcomes should be clear. If they are, this will be a wonderful development for struggling students, and a major boost for tutoring as an intervention. However, it may also provide the example educational research needs to establish its capacity to solve big practical problems.
If ProvenTutoring is as effective as we expect, perhaps it will occur to our profession that this same strategy could apply wherever we want. This could lead to accelerated investment by government in development and evaluation of replicable programs in every crucial area of education, where robust solutions are needed. Someday, could there be ProvenAlgebra.org? ProvenScience.org? ProvenGraduation.org? Proven programs for English learners? ProvenPreschool.org? ProvenClassroomManagement.org? ProvenCivics.org? Whatever problems most need to be solved, there is no reason we cannot solve them using the same evidence-to-practice strategies that medicine, agriculture, and other fields have used so successfully for many decades.
Photo courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
Whenever I talk to educators and mention effect sizes, someone inevitably complains. “We don’t understand effect sizes,” they say. I always explain that you don’t have to understand exactly what effect sizes are, but if you do know that more of them are good and less of them are bad, assuming that the research from which they came is of equal quality, then why do you have to know precisely what they are? Sometimes I mention the car reliability rating system Consumer Reports uses, with full red circles at the top and full black circles at the bottom. Does anyone understand how they arrived at those ratings? I don’t even know, but I don’t care, because like everyone else, what I do know is that I don’t want a car with a reliability rating in the black.
People always tell me that they would like it better if I’d use “additional months of gain.” I do this when I have to, but I really do not like it, because these “months of gain” do not really mean very much, and they work very differently at the early elementary grades than they do in high schools.
So here is an idea that some people might find useful. The National Assessment of Educational Progress (NAEP) uses reading and math scales that have a theoretical standard deviation of 50. So an effect size of, say, +0.20 can be expressed as a gain equivalent to a NAEP score gain of +10 (0.20 x 50 = 10) points. That’s not really interesting yet, because most people also don’t know what NAEP scores mean.
But here’s another way to use such data that might be more fun and easier to understand. I think people could understand and care about their state’s rank on NAEP scores. So for example, the highest-scoring state on 4th grade reading is Massachusetts, with a NAEP reading score of 231 in 2019. What if the 13th state, Nebraska (222), adopted a great reading program statewide, and it gained an average effect size of +0.20. That’s equivalent to 10 NAEP points. Such a gain in effect size would make Nebraska score one point ahead of Massachusetts (if Massachusetts didn’t change). Number 1!
If we learned to speak in terms of how many ranks states would gain if they gained a given effect size, I wonder if that would give educators more understanding and respect for the findings of experiments. Even fairly small effect sizes, if replicated across a whole state, could propel a state past its traditional rivals. For example, 26th ranked Wisconsin (220) could equal neighboring 12th ranked Minnesota (222) with a statewide reading effect size gain of only +0.04. As a practical matter, Wisconsin could increase its fourth grade test scores by an effect size of +0.04, perhaps by using a program with an effect size of +0.20 with (say) the lowest-achieving fifth of its fourth graders.
If only one could get states thinking this way, the meaning and importance of effect sizes would soon become clear. And as a side benefit, perhaps if Wisconsin invested its enthusiasm and money in a “Beat Minnesota” reading campaign, as it does to try to beat the University of Minnesota’s football team, Wisconsin’s students might actually benefit. I can hear it now:
On Wisconsin, On Wisconsin,
Raise effect size high!
We are not such lazy loafers
We can beat the Golden Gophers
Point-oh-four or point-oh-eight
We’ll surpass them, just you wait!
Well, a nerd can dream, can’t he?
_______
Note: No states were harmed in the writing of this blog.
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
The American Rescue Plan was passed in the U.S. Congress this week. This $1.9 trillion bill provides funding for a lot of things I care about as a citizen, but as an educator, I’d like to focus on the portion of it allocated to healing Covid learning loss. This is $29 billion, or roughly double the usual amount spent annually on Title I. This is a major investment in the students whose educations were harmed the most by Covid school closures. These are mostly disadvantaged students and rural students who could not gain access to remote teaching, or who did not have assistance at home to take advantage of remote instruction. Data from all over the country is showing the educational damage these children have sustained.
Clearly, the new money in the ARP could make a substantial difference in the achievement and adjustment of all students returning to in-person schooling. But if educational research tells us anything at all, it tells us these two things:
Making a big difference in educational outcomes costs money.
However, lots of well-meaning uses of money do not make any perceptible difference in outcomes.
Of course, the only way to tell effective uses of new funds from ineffective uses is through rigorous research.
One of the unusual aspects of the ARP education funding is that the legislation is not very specific about how the money is to be used. This is due in part to the fact the ARP was passed using a reconciliation procedure that does not allow for much specificity. The U.S. Department of Education will be drafting guidelines for the money soon, but these guidelines are likely to be relatively flexible, because the legislation itself was not very specific.
This flexibility is likely to allow anything from very good uses of money to very poor uses. My guess is that state and district leaders, and individual principals, will have plenty of freedom to use plenty of money. How novel!
I hope states and districts will use this opportunity to clearly define what is most important to accomplish in their post-Covid planning and then insist on choosing programs, practices, and policies based on the best evidence available. This time, educators will have the opportunity to use research-proven programs not because Congress or the U.S. Department of Education tells them to, but because they care about the learning and emotional well-being of their students.
In the period following the passage of the 2015 Every Student Succeeds Act (ESSA), state, district, and building leaders learned how to use services such as the What Works Clearinghouse and our www.evidenceforessa.org website to find out the strength of the evidence supporting various programs. I hope schools will continue to use these resources to select programs that have been proven effective. I’ve written many times about the importance of using proven tutoring programs, and this is indeed the most effective strategy by far for students who are far behind in reading or math. But there are many other approaches proven to be effective, especially for disadvantaged students. There is good evidence of effectiveness not only for classroom approaches to reading and math, but also programs for creative writing, science, social-emotional learning, early childhood education, and much more. The ARP funding allows schools to invest in proven programs and find out for themselves whether they work. ARP money will not be around forever, but wouldn’t it be a great use of the money to find out what works, so that when things return to normal, school and district leaders will know more than ever before what works and what doesn’t for their particular students and their particular schools?
In the first months after all schools open for in-person learning, schools are sure to be thinking in emergency mode, about investments in tutoring and other relatively expensive but highly effective strategies. But the damage Covid has done will have long-lasting impacts, and even if schools use proven tutoring methods to help the students at the greatest risk, it is also important to build for the long haul for all students, using proven programs of all kinds. Wouldn’t it be wonderful if the terrible experience we have all been through leads to a more rational, evidence-driven approach to schooling, creating a lasting benefit not only for today’s children, but for future generations who will receive better educations than they would have before Covid?
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
If you follow my blogs, you’ll note that I have been writing recently about the ineffectiveness of summer school (here, here, and here). Along with colleagues, I wrote a review of research on summer school, which is summarized here. The reason for the ineffectiveness of summer school, I proposed, is that when summer school resembles regular school, it can be boring. Kids are sitting in school while their friends are playing outside. As a result, attendance in summer school programs intended to help struggling students can be very poor, and the motivation of those who do attend may also be poor.
However, there are two major exceptions to the otherwise dismal outcomes of studies of summer school. One is a Los Angeles study by Schacter & Jo (2005), and the other is a study by Zvoch & Stevens (2013), done in a small city in the Northwest.
Both of these studies focused on disadvantaged students in grades 1 or K-1. Both provided small-group tutoring interventions. Schacter & Jo (2005) gave students phonics instruction in groups of 15, followed by small-group tutoring. The Gates-MacGinitie reading effect size was +1.16. Zvoch & Stevens (2013) also provided group phonics instruction followed by tutoring to groups of 3 to 5. The effect size on DIBELS measures was +0.69.
The large effect sizes seen in these two studies contrast sharply with all the other studies of summer classroom programs, which had a mean effect size near zero. What this suggests is that the best instructional use of summer may be to provide one-to-one or small-group tutoring to struggling students.
In summer, 2021, the rationale for summertime tutoring is particularly strong. If current trends maintain, most teachers will have received Covid vaccines by summer, and increasing numbers of schools will open by the end of the current semester. To close schools that could be open for summer vacation seems a waste. Also, assuming the American Rescue Plan is passed (as expected), it will make a great deal of money available to serve students who have lost ground due to Covid school closures, so schools will be able to afford to pay for tutoring during the summer.
The problem with summer school is that it cannot be made mandatory, and many students will not want to attend. However, in summer 2021, providing tutoring during the summer for students who do choose to attend (and keep attending regularly) could be of great value, even if most students who need tutoring do not attend. The reason is that there are so many students who will need tutoring in September, 2021, that not all of them can be tutored right away. Providing tutoring in the summer gives some students a full dose of tutoring before school officially opens, so that schools will not be under pressure to tutor more students than they are able to serve in fall, 2021.
How Can Summer Tutoring Work?
Summertime allows schools to provide more hours of tutoring each day than would be possible during the school year. For example, teaching and tutoring were provided 2 hours a day for 7 weeks in the Schacter & Jo (2005) study, and 3½ hours per day for 5 weeks in the Zvoch & Stevens (2013) study. If tutoring were alternated with sports or music or other fun activities, one might imagine providing two or three tutoring sessions each day, for as many as 8 weeks during the summer.
These sessions might be offered during a half day, so teachers and teaching assistants might teach one morning and one afternoon session each day. In fact, tutors might provide three two-hour sessions, and reach even more students.
The tutoring methods should be ones proven effective in rigorous experiments. While any whole-class teaching should be done by teachers, teaching assistants can be trained to be excellent tutors. They will need extensive training and in-class coaching, but this is worthwhile, especially because most of these tutors will continue working with additional students during the school day starting in the fall.
Tutoring in summer 2021 will provide a pilot opportunity for schools and districts to hit the ground running in September. It will provide time and resources for providers of tutoring to greatly increase their scale of operations. And it may attract students who have been out of school for many months by offering small group, supportive tutoring with caring tutors, to help ease the transition back into school.
Summertime need not be a time for summertime blues. Instead, it can serve as a “grand opening” for a successful re-entry to school for millions of students.
References
Schacter, J., & Jo, B. (2005). Learning when school is not in session: A reading summer day-camp intervention to improve the achievement of exiting first-grade students who are economically disadvantaged. Journal of Research in Reading, 28, 158-169. Doi:10.111/j.1467-9817.2005.00260.x
Zvoch, K., & Stevens, J. J. (2013). Summer school effects in a randomized field trial. Early Childhood Research Quarterly, 28(1), 24-32. Doi:10.1016/j.ecresq.2012,05.002
This blog was developed with support from Arnold Ventures. The views expressed here do not necessarily reflect those of Arnold Ventures.
Note: If you would like to subscribe to Robert Slavin’s weekly blogs, just send your email address to thebee@bestevidence.org.
Robert Slavin's Blog 