Proven Tutoring Approaches: The Path to Universal Proficiency

There are lots of problems in education that are fundamentally difficult. Ensuring success in early reading, however, is an exception. We know what skills children need in order to succeed in reading. No area of teaching has a better basis in high-quality research. Yet the reading performance of America’s children is not improving at an adequate pace. Reading scores have hardly changed in the past decade, and gaps between white, African-American, and Hispanic students have been resistant to change.
In light of the rapid growth in the evidence base, and of the policy focus on early reading at the federal and state levels, this is shameful. We already know a great deal about how to improve early reading, and we know how to learn more. Yet our knowledge is not translating into improved practice and improved outcomes on a large enough scale.
There are lots of complex problems in education, and complex solutions. But here’s a really simple solution:

 

Over the past 30 years researchers have experimented with all sorts of approaches to improve students’ reading achievement. There are many proven and promising classroom approaches, and such programs should be used with all students in initial teaching as broadly as possible. Effective classroom instruction, universal access to eyeglasses, and other proven approaches could surely reduce the number of students who need tutors. But at the end of the day, every child must read well. And the only tool we have that can reliably make a substantial difference at scale with struggling readers is tutors, using proven one-to-one or small-group methods.

I realized again why tutors are so important in a proposal I’m making to the State of Maryland, which wants to bring all or nearly all students to “proficient” on its state test, the PARCC. “Proficient” on the PARCC is a score of 750, with a standard deviation of about 50. The state mean is currently around 740. I made a colorful chart (below) showing “bands” of scores below 750 to show how far students have to go to get to 750.

 

Each band covers an effect size of 0.20. There are several classroom reading programs with effect sizes this large, so if schools adopted them, they could move children scoring at 740 to 750. These programs can be found at www.evidenceforessa.org. But implementing these programs alone still leaves half of the state’s children not reaching “proficient.”

What about students at 720? They need 30 points, or +0.60. The best one-to-one tutoring can achieve outcomes like this, but these are the only solutions that can.

Here are mean effect sizes for various reading tutoring programs with strong evidence:

 

 

As this chart shows, one-to-one tutoring, by well-trained teachers or paraprofessionals using proven programs, can potentially have the impacts needed to bring most students scoring 720 (needing 30 points or an effect size of +0.60) to proficiency (750). Three programs have reported effect sizes of at least +0.60, and several others have approached this level. But what about students scoring below 720?

So far I’ve been sticking to established facts, studies of tutoring that are, in most cases, already being disseminated. Now I’m entering the region of well-justified supposition. Almost all studies of tutoring occupy just one year or less. But what if the lowest achievers could receive multiple years of tutoring, if necessary?

One study, over 2½ years, did find an effect size of +0.68 for one-to-one tutoring. Could we do better that that? Most likely. In addition to providing multiple years of tutoring, it should be possible to design programs to achieve one-year effect sizes of +1.00 or more. These may incorporate technology or personalized approaches specific to the needs of individual children. Using the best programs for multiple years, if necessary, could increase outcomes further. Also, as noted earlier, using proven programs other than tutoring for all students may increase outcomes for students who also receive tutoring.

But isn’t tutoring expensive? Yes it is. But it is not as expensive as the costs of reading failure: Remediation, special education, disappointment, and delinquency. If we could greatly improve the reading performance of low achievers, this would of course reduce inequities across the board. Reducing inequities in educational outcomes could reduce inequities in our entire society, an outcome of enormous importance.

Even providing a substantial amount of teacher tutoring could, by my calculations, increase total state education expenditures (in Maryland) by only about 12%. These costs could be reduced greatly or even eliminated by reducing expenditures on ineffective programs, reducing special education placements, and other savings. Having some tutoring done by part time teachers may reduce costs. Using small-group tutoring (fewer than 6 students at a time) for students with milder problems may save a great deal of money. Even at full cost, the necessary funding could be phased in over a period of 6 years at 2% a year.

The bottom line is that the low levels of achievement and high levels of gaps according to economic and racial differences could be improved a great deal using methods already proven to be effective and already widely available. Educators and policy makers are always promising policies that bring every child to proficiency: “No Child Left Behind” and “Every Student Succeeds” come to mind. Yet if these outcomes are truly possible, why shouldn’t we be pursuing them, with every resource at our disposal?

Advertisements

Love, Hope, and Evidence in Secondary Reading

I am pleased to announce that our article reviewing research on effective secondary reading programs has just been posted on the Best Evidence Encyclopedia, aka the BEE. Written with my colleagues Ariane Baye, Cynthia Lake, and Amanda Inns, our review found 64 studies of 49 reading programs for students in grades 6 to 12, which had to meet very high standards of quality. For example, 55 of the studies used random assignment to conditions.

But before I get all nerdy about the technical standards of the review, I want to reflect on what we learned. I’ve already written about one thing we learned, that simply providing more instructional time made little difference in outcomes. In 22 of the studies, students got an extra period for reading beyond what control students got for at least an entire year, yet programs (other than tutoring) that provided extra time did no better than those that did not.

If time doesn’t help struggling readers, what does? I think I can summarize our findings with three words: love, hope, and evidence.

Love and hope are exactly what students who are reading below grade level are lacking. They are no longer naive. They know exactly what it means to be a poor reader in a high-poverty secondary school (almost all of the schools in our review served disadvantaged adolescents). If you can’t read well, college is out of the question. Decent jobs without a degree are scarce. If you have no hope, you cannot be motivated, or you may be motivated in antisocial directions that give you at least a chance for money and recognition. Every child needs love, but poor readers in secondary schools are too often looking for love in all the wrong places.

The successful programs in our review were ones that give adolescents a chance to earn the hope and love they crave. One category, all studies done in England, involved one-to-one and small group tutoring. How better to build close relationships between students and caring adults than to have individual or very small group time with them? And the one-to-one or small group setting allows tutors to personalize instruction, giving students a sense of hope that this time, their efforts will pay off (as the evidence says it will).

But the largest impacts in our review came from two related programs – The Reading Edge and Talent Development High School (TDHS). These both developed in our research center at Johns Hopkins University in the 1990s, so I have to be very modest here. But beyond these individual programs, I think there is a larger message.

Both The Reading Edge (for middle schools) and TDHS (for high schools) organize students into mixed-ability cooperative teams. The team members work on activities designed to build reading comprehension and related skills. Students are frequently assessed and on the basis of those assessments, they can earn recognition for their teams. Teachers introduce lessons, and then, as students work with each other on reading activities, teachers can cruise around the class looking in on students who need encouragement or help, solving problems, and building relationships. Students are on task, eager to learn, and seeing the progress they are making, but students and teachers are laughing together, sharing easy banter, and encouraging each other. Yes, this really happens. I’ve seen it hundreds of times in secondary schools throughout the U.S. and England.

Many of the most successful programs in our review also are based on principles of love and hope. BARR, a high school program, is an excellent example. It uses block scheduling to build positive relationships among a group of students and teachers, adding regular meetings between teachers and students to review their progress in all areas, social as well as academic. The program focuses on building positive social-emotional skills and behaviors, and helping students describe their desired futures, make plans to get there, and regularly review progress on their plans with their teachers and peers. Love and hope.

California’s Expository Reading and Writing Course helps 12th graders hoping to attend California State Universities prepare to pass the test used to determine whether students have to take remedial English (a key factor in college dropout). The students work in groups, helping each other to build reading, writing, and discussion skills, and helping students to visualize a future for themselves. Love and hope.

A few technology programs showed promising outcomes, especially Achieve3000 and Read 180. These do not replace teachers and peers with technology, but instead cycle students through small group, teacher-led, and computer-assisted activities. Pure technology programs did not work so well, but models taking advantage of relationships as well as personalization did best. Love and hope.

Of course, love and hope are not sufficient. We also need evidence that students are learning more than they might have been. To produce positive achievement effects requires outstanding teaching strategies, professional development, curricular approaches, assessments, and more. Love and hope may be necessary but they are not sufficient.

Our review applied the toughest evidence standards we have ever applied. Most of the studies we reviewed did not show positive impacts on reading achievement. But the ones that did so inspire that much more confidence. The very fact that we could apply these standards and still find plenty of studies that meet them shows how much our field is maturing. This in itself fills me with hope.

And love.

Apology

In a recent blog, I wrote about work we are doing to measure the impact on reading and math performance of a citywide campaign to provide assessments and eyeglasses to every child in Baltimore, from pre-k to grade 8. I forgot to mention the name of the project, Vision for Baltimore, and neglected to say that the project operates under the authority of the Baltimore City Health Department, which has been a strong supporter. I apologize for the omission.

Vision and Blindness

If you wear reading glasses, please remove them for a moment, and continue reading.

Back to normal? For a moment, you had an experience like that of about 30% of Baltimore students. Some have myopia (nearsightedness) and some hyperopia (farsightedness), and some other problems. But few have glasses. A study in grades 2-3 found that only 6% of students had glasses in school, and 30% needed them. Kids being kids, even those who have glasses may soon lose or break them, and glasses are rarely replaced for kids in inner-city schools. As a result, some students can’t see the whiteboard, some can’t see their books, and many quietly think they are not smart because they struggle to focus on the printed word. In Maryland, students’ vision is tested only at school entry (usually pre-k), first grade, and eighth grade. If routine screenings find a problem, a note goes to parents asking them to get a formal assessment. In Baltimore, this results in about 10% of children who need glasses getting them. And then what do you think happens to those glasses between first and eighth grade?

I’ve been involved with studies of vision in inner-city schools along with colleagues Megan Collins, David Friedman, Michael Repka, and others from the Wilmer Eye Clinic at Johns Hopkins Medical Institutions, and Nancy Madden and others from the Johns Hopkins School of Education. The name of the project is Vision for Baltimore, and it operates under the authority of the Baltimore City Health Department, which has been a strong supporter. What we are finding is in one sense a privileged glimpse into the perfectly obvious. Inner city children who need glasses don’t often get them. We tested all students in grades 2-3 in 12 high-poverty Baltimore schools, and we gave those who needed them free glasses. We also followed up to make sure the students were wearing glasses, and we replaced those that were lost or broken. Students who received the glasses gained significantly on reading tests in comparison to those who never needed glasses. Of course. Yet this was the first U.S. study of its kind to show an effect of glasses on reading (two Chinese studies had found the same).

We are now doing a much larger study. A philanthropic group called Vision to Learn (VTL) wanted to provide assessments and free glasses to every elementary and middle school student in Baltimore over a three-year period. VTL has mobile vision vans, staffed with an optometrist and an optician. The vans can test all students who were found in screening to need assessment, and then provide free glasses if needed. With funding from Baltimore’s Abell Foundation and the Laura and John Arnold Foundation, we arranged to randomly assign schools to receive their vision services either in the first, second, or third year, enabling us to find the impact of these services in reading and math performance, mostly on state tests.

It will be a couple of years before we will know the results of our research, but I can tell you this much. As in our smaller study, we found that very few children already had glasses, and about 30% needed them. This fall, the first glasses are arriving, and the students are blown away. One fifth grade girl said, “Is this the way things are supposed to look?”

Now think about that girl. If she needs glasses now, she has probably needed them for years. How much damage was done to her essential early education? How much was her self-esteem damaged by learning problems due to nothing more than poor vision?

I should hasten to add that eyeglasses for students who need them are an inexpensive intervention. In the enormous quantities involved, a pair of glasses that kids are eager to wear may cost less than $20. Further, Medicaid pays for eyeglasses for all children who qualify as low income, which equates to nearly every child in Baltimore. Vision to Learn has worked out ways to make this easy to administer, so that modest funds from an existing federal program can be used for this essential service.

Vision is important. We hope our work and that of others around the U.S. will develop simple, replicable means of improving the achievement of disadvantaged children by giving them needed eyeglasses. But what I really want to talk about today is not vision, but blindness. Moral blindness. Policy blindness. Pragmatic blindness.

It so happens that vision is an excellent case to illustrate our moral, policy, and pragmatic blindness. We spend approximately $11,000 per child per year, on average, to educate a child. From all that expenditure, we want successful, capable, skilled students, who can enter higher education or the workforce with confidence and well-founded hopes of success. We want students who will follow the rules because they know that they can succeed if they do.

Yet we let $20 worth of eyeglasses stand in their way.

We spend vast amounts of money on special education, remedial services, even tutoring. Yet some proportion of the children who receive these services just needed eyeglasses instead. The policy world has tried for years to reduce special education costs and integrate children in regular classes. Many likely never needed special education to begin with.

Yet we let $20 worth of eyeglasses per child stand in our way.

We know that young people who fail in school are far more likely to become delinquent and later criminal. The costs of policing and incarceration are huge, and we need to reduce them.

Yet we let $20 worth of eyeglasses per child stand in our way.

There are lots of very difficult problems in education. This does not happen to be one of them. Can we all agree to put glasses on every disadvantaged child who needs them? This will not solve all of our problems, but if would be a heck of a start. While we’re at it, we also ought to look into hearing and other medical problems that hold kids back.

There are none so blind as those who will not see.

Correction

In an earlier version of this blog, I forgot to mention the name of the project and the authority under which is operates. I apologize for the omission.

Time Passes. Will You?

When I was in high school, one of my teachers posted a sign on her classroom wall under the clock:

Time passes. Will you?

Students spend a lot of time watching clocks, yearning for the period to be over. Yet educators and researchers often seem to believe that more time is of course beneficial to kids’ learning. Isn’t that obvious?

In a major review of secondary reading programs I am completing with my colleagues Ariane Baye, Cynthia Lake, and Amanda Inns, it turns out that the kids were right. More time, at least in remedial reading, may not be beneficial at all.

Our review identified 60 studies of extraordinary quality- mostly large-scale randomized experiments- evaluating reading programs for students in grades 6 to 12. In most of the studies, students reading 2 to 5 grade levels below expectations were randomly assigned to receive an extra class period of reading instruction every day all year, in some cases for two or three years. Students randomly assigned to the control group continued in classes such as art, music, or study hall. The strategies used in the remedial classes varied widely, including technology approaches, teaching focused on metacognitive skills (e.g., summarization, clarification, graphic organizers), teaching focused on phonics skills that should have been learned in elementary school, and other remedial approaches, all of which provided substantial additional time for reading instruction. It is also important to note that the extra-time classes were generally smaller than ordinary classes, in the range of 12 to 20 students.

In contrast, other studies provided whole class or whole school methods, many of which also focused on metacognitive skills, but none of which provided additional time.

Analyzing across all studies, setting aside five British tutoring studies, there was no effect of additional time in remedial reading. The effect size for the 22 extra-time studies was +0.08, while for 34 whole class/whole school studies, it was slightly higher, ES =+0.10. That’s an awful lot of additional teaching time for no additional learning benefit.

So what did work? Not surprisingly, one-to-one and small-group tutoring (up to one to four) were very effective. These are remedial and do usually provide additional teaching time, but in a much more intensive and personalized way.

Other approaches that showed particular promise simply made better use of existing class time. A program called The Reading Edge involves students in small mixed-ability teams where they are responsible for the reading success of all team members. A technology approach called Achieve3000 showed substantial gains for low-achieving students. A whole-school model called BARR focuses on social-emotional learning, building relationships between teachers and students, and carefully monitoring students’ progress in reading and math. Another model called ERWC prepares 12th graders to succeed on the tests used to determine whether students have to take remedial English at California State Universities.

What characterized these successful approaches? None were presented as remedial. All were exciting and personalized, and not at all like traditional instruction. All gave students social supports from peers and teachers, and reasons to hope that this time, they were going to be successful.

There is no magic to these approaches, and not every study of them found positive outcomes. But there was clearly no advantage of remedial approaches providing extra time.

In fact, according to the data, students would have done just as well to stay in art or music. And if you’d asked the kids, they’d probably agree.

Time is important, but motivation, caring, and personalization are what counts most in secondary reading, and surely in other subjects as well.

Time passes. Kids will pass, too, if we make such good use of our time with them that they won’t even notice the minutes going by.

Twenty-four Proven Programs for Struggling Readers

One of the greatest impediments to evidence-based reform in education is the belief that there are very few programs that have been rigorously evaluated and found to be effective. People often make fun of the What Works Clearinghouse (WWC), calling it the Nothing Works Clearinghouse, because in its early days there were, in fact, few programs that met WWC standards.

If you believe in the “nothing works” formulation, I’ve got astonishing news for you. You might want to find a safe place to sit, and remove any eyeglasses or sharp objects, before reading any further, to avoid accidental injury.

Ready?

I have been reviewing research on various programs for elementary struggling readers to find out how many meet the new ESSA evidence standards. The answer: at least 24. Of these, 14 met the “strong” ESSA criterion, which means that there was at least one randomized study with statistically significant positive effects. Eight met the “moderate” standard, which requires at least one quasi-experimental (i.e., matched) study with significant positive effects. Two met the “promising” standard, requiring at least one correlational study with positive effects. (For a list of struggling reader programs organized by ESSA categories, click here).

I should hasten to explain that the numbers of proven programs will be higher for struggling readers programs than for whole-class programs, because most of the struggling readers programs are one-to-one or one-to-small-group tutoring. But still, the number and diversity of proven programs is impressive. Among the 24 programs, eight used one-to-one tutoring by teachers, paraprofessionals, or volunteers. Nine used small-group tutoring by teachers or paraprofessionals. However, one used computer-assisted instruction, and five used whole-school or whole-class methods and reported significantly positive effects on the students who had been in the lowest-achieving third or quarter of the classes at pretest. Two of the 24 programs, Reading Recovery (1-1 tutoring by teachers) and Success for All (whole-school approach) are well known and have been around a long time, but many others are much less well known. Of course, one-to-one tutoring, especially by teachers, can be very expensive, but whole-school and whole-class approaches tend to be relatively inexpensive on a per-pupil basis.

Here’s my point. Schools seeking proven, practical approaches to improving outcomes for their struggling readers have a wide array of attractive alternatives. Six of them, Reading Recovery, Success for All, Sound Partners (1-1 tutoring by paraprofessionals), Lindamood (small group tutoring by teachers), Targeted Reading Intervention (1-1 tutoring by teachers), and Empower Reading (small group tutoring by teachers) all have large effect sizes from randomized experiments and have been proven in from two to 28 studies.

It is important to note that there are also many programs for struggling readers that have been evaluated and found to be ineffective, including tutoring programs. It matters a lot which program you choose.

Every school and district has children who are struggling to learn to read, and all too often their solution is to make up their own approach for these students, or to purchase materials, software, or services from vendors who can present no credible evidence of effectiveness. If there were no proven solutions, such strategies might make sense, but how can they be justified when there are so many proven alternatives?

A better use of time and energy might be for educational leaders to review the proven programs for struggling readers, seek information about their benefits and costs, speak with educators who have used them, and perhaps arrange a visit to schools using programs being considered. Then they’d have a good chance of picking an approach that is likely to work if well implemented.

Soon, we will have information about proven programs in every subject and grade level, for all types of learners. Wouldn’t this be a good time to get into the habit of using proven programs to improve student outcomes?

Joy is a Basic Skill in Secondary Reading

I have a policy of not talking about studies I’m engaged in before they are done and available, but I have an observation to make that just won’t wait.

I’m working on a review of research on secondary reading programs with colleagues Ariane Baye (University of Liege in Belgium) and Cynthia Lake (Johns Hopkins University). We have found a large number of very high-quality studies evaluating a broad range of programs. Most are large, randomized experiments.

Mostly, our review is really depressing. The great majority of studies have found no effects on learning. In particular, programs that focus on teaching middle and high school students struggling in reading in classes of 12 to 20, emphasizing meta-cognitive strategies, phonics, fluency, and/or training for teachers in what they were already doing, show few impacts on learning. Most of the studies provided daily, extra reading classes to help struggling readers build their skills, while the control group got band or art. They should have stayed in band or art.

Yet all is not dismal. Two approaches did have markedly positive effects. One was tutoring students in groups of one to four, not every day but perhaps twice a week. The other was cooperative learning, where students worked in four-member teams to help each other learn and practice reading skills. How could these approaches be so much more effective than the others?

My answer begins with a consideration of the nature of struggling adolescent readers. They are bored out of their brains. They are likely to see school as demeaning, isolating, and unrewarding. All adolescents live for their friends. They crave mastery and respect. Remedial approaches have to be fantastic to overcome the negative aspects of having to be remediated in the first place.

Tutoring can make a big difference, because groups are small enough for students to make meaningful relationships with adults and with other kids, and instruction can be personalized to meet their unique needs, to give them a real shot at mastery.

Cooperative learning, however, had a larger average effect size than tutoring. Even though cooperative learning did not require smaller class sizes and extra daily instructional periods, it was much more effective than remedial instruction. Cooperative learning gives struggling adolescent readers opportunities to work with their peers, to teach each other, to tease each other, to laugh, to be active rather than passive. To them, it means joy. And joy is a basic skill.

Of course, joy is not enough. Kids must be learning joyfully, not just joyful. Yet in our national education system, so focused on testing and accountability, we have to keep remembering who we are teaching and what they need. More of the same, a little slower and a little louder, won’t do it. Adolescents need a reason to believe that things can be better, and that school need not cut them off from their peers. They need opportunities to teach and learn from each other. School must be joyful, or it is nothing at all, for so many adolescents.

Making Evidence Primary for Secondary Readers

In the wonderful movie Awakenings, Robin Williams plays a research neuroscientist who has run out of grants and therefore applies for a clinical job at a mental hospital. In the interview, the hospital’s director asks him about his research.

“I was trying to extract myelin from millions of earthworms,” he explains.

“But that’s impossible!” says the director.

“Yes, but now we know it’s impossible,” says Robin Williams’ character.

I recently had an opportunity to recall this scene. I was traveling back to Baltimore from Europe. Whenever I make this trip, I use the eight or so uninterrupted hours to do a lot of work. This time I was reading a giant stack of Striving Readers reports, because I am working with colleagues to update a review of research on secondary reading programs.

Striving Readers, part of Reading First, was a richly funded initiative of the George W. Bush administration that gave money to states to help them adopt intensive solutions for below-level readers in middle and high schools. The states implemented a range of programs, almost all of them commercial programs designed for secondary readers. To their credit, the framers of Striving Readers required rigorous third-party evaluations of whatever the states implemented, and those were the reports I was reading. Unfortunately, it apparently did not occur to anyone to suggest that the programs have their own evidence of effectiveness prior to being implemented and evaluated as part of Striving Readers.

As you might guess from the fact that I started off this blog post with the earthworm story, the outcomes are pretty dismal. A few of the studies found statistically significant impacts, but even those found very small effect sizes, and only on some but not other measures or subgroups.

I’m sure I and others will learn more as we get further into these reports, which are very high-quality evaluations with rich measures of implementation as well as outcomes. But I wanted to make one observation at this point.

Striving Readers was a serious, well-meaning attempt to solve a very important problem faced by far too many secondary students: difficulties with reading. I’m glad the Department of Education was willing to make such an investment. But next time anyone thinks of doing something on the scale of Striving Readers, I hope they will provide preference points in the application process for applicants who propose to use approaches with solid evidence of effectiveness. I also hope government will continue to fund development and evaluation of programs to address enduring problems of education, so that when they do start providing incentives for using proven programs, there will be many to choose from.

Just like the earthworm research in Awakenings, finding out conclusively what doesn’t work is a contribution to science. But in education, how many times do we have to learn what doesn’t work before we start supporting programs that we know do work? It’s time to recognize on a broad scale that programs proven to work in rigorous evaluations are more likely than other approaches to work again if implemented well in similar settings. Even earthworms learn from experience. Shouldn’t we do the same?