Twenty-four Proven Programs for Struggling Readers

One of the greatest impediments to evidence-based reform in education is the belief that there are very few programs that have been rigorously evaluated and found to be effective. People often make fun of the What Works Clearinghouse (WWC), calling it the Nothing Works Clearinghouse, because in its early days there were, in fact, few programs that met WWC standards.

If you believe in the “nothing works” formulation, I’ve got astonishing news for you. You might want to find a safe place to sit, and remove any eyeglasses or sharp objects, before reading any further, to avoid accidental injury.

Ready?

I have been reviewing research on various programs for elementary struggling readers to find out how many meet the new ESSA evidence standards. The answer: at least 24. Of these, 14 met the “strong” ESSA criterion, which means that there was at least one randomized study with statistically significant positive effects. Eight met the “moderate” standard, which requires at least one quasi-experimental (i.e., matched) study with significant positive effects. Two met the “promising” standard, requiring at least one correlational study with positive effects. (For a list of struggling reader programs organized by ESSA categories, click here).

I should hasten to explain that the numbers of proven programs will be higher for struggling readers programs than for whole-class programs, because most of the struggling readers programs are one-to-one or one-to-small-group tutoring. But still, the number and diversity of proven programs is impressive. Among the 24 programs, eight used one-to-one tutoring by teachers, paraprofessionals, or volunteers. Nine used small-group tutoring by teachers or paraprofessionals. However, one used computer-assisted instruction, and five used whole-school or whole-class methods and reported significantly positive effects on the students who had been in the lowest-achieving third or quarter of the classes at pretest. Two of the 24 programs, Reading Recovery (1-1 tutoring by teachers) and Success for All (whole-school approach) are well known and have been around a long time, but many others are much less well known. Of course, one-to-one tutoring, especially by teachers, can be very expensive, but whole-school and whole-class approaches tend to be relatively inexpensive on a per-pupil basis.

Here’s my point. Schools seeking proven, practical approaches to improving outcomes for their struggling readers have a wide array of attractive alternatives. Six of them, Reading Recovery, Success for All, Sound Partners (1-1 tutoring by paraprofessionals), Lindamood (small group tutoring by teachers), Targeted Reading Intervention (1-1 tutoring by teachers), and Empower Reading (small group tutoring by teachers) all have large effect sizes from randomized experiments and have been proven in from two to 28 studies.

It is important to note that there are also many programs for struggling readers that have been evaluated and found to be ineffective, including tutoring programs. It matters a lot which program you choose.

Every school and district has children who are struggling to learn to read, and all too often their solution is to make up their own approach for these students, or to purchase materials, software, or services from vendors who can present no credible evidence of effectiveness. If there were no proven solutions, such strategies might make sense, but how can they be justified when there are so many proven alternatives?

A better use of time and energy might be for educational leaders to review the proven programs for struggling readers, seek information about their benefits and costs, speak with educators who have used them, and perhaps arrange a visit to schools using programs being considered. Then they’d have a good chance of picking an approach that is likely to work if well implemented.

Soon, we will have information about proven programs in every subject and grade level, for all types of learners. Wouldn’t this be a good time to get into the habit of using proven programs to improve student outcomes?

Advertisements

Why Leave Learning to Chance?

Every year about four million kindergartners enter America’s schools. They’re all excited, eager and confident, because that’s the nature of kindergartners, but unfortunately, we adults know better. We know that among those wonderful five year olds, 65% will reach fourth grade reading below the “proficient” level on the National Assessment of Educational Progress (NAEP), and 31% will not even reach the “basic” level. We know which students in which neighborhoods are most likely to have these problems. Since 1980, the story has hardly changed.

Today, I’m writing this blog from an airplane flying from Baltimore to San Francisco. Flying was a risky business long ago, but today the chances are infinitesimal that my airplane will crash.

So here’s a question. Why is it ok to leave the reading success of children to chance? Why don’t we treat reading success the way we treat air safety, as something to ensure no matter what?

If you think we don’t yet know how to ensure the reading success of all children, you might be right, but I can tell you that we absolutely do know how to ensure a much higher level of success than we have now, with today’s teachers and today’s schools. I was recently reviewing research evaluating reading programs, and I found more than 60 different programs with moderate to strong evidence of effectiveness: one-to-one and one-to-small group tutoring, classroom methods, school-wide reforms, and technology. Over time, it’s certain that these approaches, and combinations of them, could become more and more effective, and we could approach 100% success.

Getting to 100% will require more than just better instruction. We are doing a study in high-poverty schools in Baltimore and found that while at least 21% of second and third graders need glasses, only 6% have them. I’m sure there are similar stories relating to hearing, dental, health, and mental health. Absenteeism is another blocker, and there are more. If we want to get to 100%, we have to deal with all of these.

Well sure, you might say, but how could we afford all of this? Fortunately, the most widespread reading problems can be solved inexpensively. The average annual per-pupil cost in the U.S. is about $11,000. The annual cost of our proven Success for All reading program is around $100 additional, or less than 1% of what we are already spending. Two pairs of eyeglasses — one to take home and one to leave at school — including the eye exam and glasses replacement, costs less than $50. Proven tutoring models provided by paraprofessionals can cost as little as $400 per student, but even at $2000 for one-to-one tutoring, that’s 18% of average per-pupil cost, and for only a minority of the class.

These modest expenditures on proven programs quickly pay back their costs in terms of reducing special education and retention, much less long-term benefits to children and society. Yet none of the 60 proven and promising programs I found is in truly widespread use.

On my airplane, of course, the situation is quite different. Pilots are carefully and extensively trained in proven methods. Technology is constantly developing to provide information and automated assistance to ensure safety and effectiveness. Back-up systems ensure that if things go wrong despite the best of preparation, disaster will not result. All of these systems are constantly evolving in response to development, evaluation, and implementation of innovations.

The reading success of a child is a very serious matter. It simply makes no sense to treat it any less seriously than we treat air safety. Just as on airplanes, we need systems to monitor children’s success, not to punish teachers but to know when and how to intervene if trouble arises.

Perhaps someday, we’ll put Boeing or Lockheed Martin in charge of our schools, and charge them with getting us as close as possible to 100% success in reading. I can see it now.

Proven approaches to:

Phonemic awareness? Check
Phonics? Check
Vocabulary? Check
Fluency? Check
Comprehension? Check
Vision? Check
Hearing? Check
Tutoring backup? Check

Ready for takeoff!

Of course we can solve this problem. All we have to do is to decide it must be solved and then do it. It is neither efficient nor ethical to keep accepting the number of reading disasters we experience in our schools.

Evidence-Based Practice: It’s About the Kids

2013-04-10-BlogImageReading.jpg

Many years ago, I heard a heart-rending story. There was a fourth grader in a school in Southern Maryland who had not learned to read. I’ll call him Patrick. A proven reading program, our Success for All model, came to the school and replaced the school’s haphazard reading approach with a systematic, phonetic model. By the end of the school year, Patrick was reading near grade level.

Toward the end of the year, Patrick’s mother came to the school to thank his teacher for what she’d done for him. She showed Patrick’s teacher a box in which Patrick had saved every one of his phonetic readers. “Patrick calls this his treasure box,” she said. “He says he is going to keep these books forever, so that if he ever has a child of his own, he can teach him how to read.”

Here is the importance of this story. If you follow my blogs in the Huffington Post, or other writings on evidence-based practice, they often sound a little dry, full of effect sizes and wonkiness. Yet all of those effect sizes and policy proposals mean nothing unless they are changing the lives of children.

Traditional educational practices are perhaps fine for most kids, but there are millions of kids like Patrick who are not succeeding in school but could be, if they experienced proven programs and practices. Patrick, at age 10, had the foresight to prepare to help his own child someday avoid the pain and humiliation he had experienced. Why is it so hard for caring grownups in positions of authority to come to the same understanding?

There is no problem in education we know more about than early reading failure. There are proven one-to-one and small-group tutoring programs, classroom interventions, and whole-school approaches like Success for All. They differ in costs, impacts, and practicability in various settings, but it is clear that reading failure can be prevented or remediated before third grade for nearly all children. Yet most struggling young readers do not receive any of these programs.

Patrick must be about 30 by now. Perhaps he has a child of his own. Wherever he is, I’m certain he remembers how close he came to a life of illiteracy and failure. I wonder if he still has his treasure box with the books inside it.

Patrick probably does not know where those books came from, the research supporting their use, or the effect sizes from the many evaluations. He doesn’t need to be a researcher to understand what happened to him. What he does know is that somehow, someone cared enough to give him an opportunity to learn to read, using a program proven to be effective.

Why does what happened to Patrick have to be such a rare occurrence? If you understand what the evidence means and you see educators and policy makers continuing to ignore it, shouldn’t you be furious?

Lessons from Innovators: Children’s Learning Initiative

2013-02-19-HP4Image21813.jpg

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the two foundations, will highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

Today’s post focuses on the Children’s Literacy Initiative (CLI). It is based on conversations between the Forum and CLI’s Executive Director, Kelly Hunter, on what it takes to maintain fidelity to a complex model in light of constant change in urban school districts. A summary of her comments is as follows.

Plan for change and stick to your core. School systems are in constant flux and developers must be prepared for instability. The Children’s Literacy Initiative (CLI) tries to do that by using training, coaching and other supports to promote quality teaching to ensure that students in low performing, urban districts are proficient readers, writers and thinkers by the end of third grade. They are currently attempting to scale their effort to four new districts, Camden, Chicago, Newark and Philadelphia. This is easier said than done. Such districts experience frequent teacher and administrator turnover, school closures and mergers, and charter formation. Hunter suggests that if you want to implement with fidelity you first have to take a long and hard look at your model, make decisions about what is core, and then message those core ingredients in a way that respects where schools are coming from. Hunter notes, “We realized that we were struggling with messaging our change model. Even though research shows quality teaching is the number one school factor, funders and others were focused on other reforms that are sexy today. We didn’t want to focus on being negative or bad mouthing other reforms. We just knew we had to be strong in our position, share the research, and stay clear about our message and core ingredients.”

Identify the right champions. Kelly and her partners at CLI have learned that regional superintendents are a critical ingredient for sustained change. These area leaders have considerable influence over principals. “At the beginning,” Hunter notes, “we would get central office and schools to sign off, but not the regional superintendents. Then we would be off and running but all of a sudden the regionals were messaging something different than what we were doing.” When regional leaders began to understand the importance of fidelity and appreciate the core ingredients, they were then able to share their enthusiasm with principals or set standards to reinforce values and practices consistent with the model.

Partner to multiply resources and minimize obstacles. As they push towards scale, leaders at CLI have also learned the importance of cultivating new and varied partnerships. In addition to district staff, especially important partners include local funders and other program providers. Local funders are essential from a sustainability standpoint. It is also critical to partner with other entities that provide related services or technical assistance within a building or district – even when they involve a different subject matter or grade. These partnerships can allow for more comprehensive and coherent supports across disciplines and grade levels and minimize confusion among and competing demands on district staff. “It’s about enhancing what we are doing, not changing it,” comments Hunter. For example, in one i3 school in West Philadelphia, Drexel University was providing coaching services in math while CLI was providing literacy coaching. By working together, they were able to make coaching across these topics more consistent and communication more streamlined.

Scale back to scale up. Implementing innovative practices is complicated and labor intensive. Regional knowledge is necessary to help align external needs and resources with your own organizations’ demands and capacities. Networking locally is a great way to learn about a school, community or district, and to identify key stakeholders, funders, and advocates. But building this knowledge and these relationships takes staff, time, and energy. To address this challenge, CLI revisited their initial plan and decided to concentrate energy and resources on implementing the model deeply in four cities rather than spread themselves thinly across ten. According to Hunter, “we knew that in some communities, we didn’t have enough local influence, networking and outreach to raise the dollars and implement the model with fidelity. We were chasing dollars and our model was being compromised. Ultimately that compromises student achievement.” Instead, she says, “over time we hope to build our presence in and around our four hubs and eventually serve as a model for other communities as they scale to surrounding schools and districts.”

Lessons from Innovators: Children’s Learning Initiative

2013-02-19-HP4Image21813.jpg

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the two foundations, will highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

Today’s post focuses on the Children’s Literacy Initiative (CLI). It is based on conversations between the Forum and CLI’s Executive Director, Kelly Hunter, on what it takes to maintain fidelity to a complex model in light of constant change in urban school districts. A summary of her comments is as follows.

Plan for change and stick to your core. School systems are in constant flux and developers must be prepared for instability. The Children’s Literacy Initiative (CLI) tries to do that by using training, coaching and other supports to promote quality teaching to ensure that students in low performing, urban districts are proficient readers, writers and thinkers by the end of third grade. They are currently attempting to scale their effort to four new districts, Camden, Chicago, Newark and Philadelphia. This is easier said than done. Such districts experience frequent teacher and administrator turnover, school closures and mergers, and charter formation. Hunter suggests that if you want to implement with fidelity you first have to take a long and hard look at your model, make decisions about what is core, and then message those core ingredients in a way that respects where schools are coming from. Hunter notes, “We realized that we were struggling with messaging our change model. Even though research shows quality teaching is the number one school factor, funders and others were focused on other reforms that are sexy today. We didn’t want to focus on being negative or bad mouthing other reforms. We just knew we had to be strong in our position, share the research, and stay clear about our message and core ingredients.”

Identify the right champions. Kelly and her partners at CLI have learned that regional superintendents are a critical ingredient for sustained change. These area leaders have considerable influence over principals. “At the beginning,” Hunter notes, “we would get central office and schools to sign off, but not the regional superintendents. Then we would be off and running but all of a sudden the regionals were messaging something different than what we were doing.” When regional leaders began to understand the importance of fidelity and appreciate the core ingredients, they were then able to share their enthusiasm with principals or set standards to reinforce values and practices consistent with the model.

Partner to multiply resources and minimize obstacles. As they push towards scale, leaders at CLI have also learned the importance of cultivating new and varied partnerships. In addition to district staff, especially important partners include local funders and other program providers. Local funders are essential from a sustainability standpoint. It is also critical to partner with other entities that provide related services or technical assistance within a building or district – even when they involve a different subject matter or grade. These partnerships can allow for more comprehensive and coherent supports across disciplines and grade levels and minimize confusion among and competing demands on district staff. “It’s about enhancing what we are doing, not changing it,” comments Hunter. For example, in one i3 school in West Philadelphia, Drexel University was providing coaching services in math while CLI was providing literacy coaching. By working together, they were able to make coaching across these topics more consistent and communication more streamlined.

Scale back to scale up. Implementing innovative practices is complicated and labor intensive. Regional knowledge is necessary to help align external needs and resources with your own organizations’ demands and capacities. Networking locally is a great way to learn about a school, community or district, and to identify key stakeholders, funders, and advocates. But building this knowledge and these relationships takes staff, time, and energy. To address this challenge, CLI revisited their initial plan and decided to concentrate energy and resources on implementing the model deeply in four cities rather than spread themselves thinly across ten. According to Hunter, “we knew that in some communities, we didn’t have enough local influence, networking and outreach to raise the dollars and implement the model with fidelity. We were chasing dollars and our model was being compromised. Ultimately that compromises student achievement.” Instead, she says, “over time we hope to build our presence in and around our four hubs and eventually serve as a model for other communities as they scale to surrounding schools and districts.”

Reading by Third Grade: Thinking Beyond Retention

Everyone agrees that children who read well by third grade are far more likely than those who do not to succeed in later education and in life. For example, a recent report from the Annie E. Casey Foundation found that poor readers in third grade were four times more likely than adequate readers to eventually drop out of school. Based on this kind of information, many states have passed or are considering legislation requiring that students not reading at an established level by the end of third grade to repeat the grade. Usually, the legislation also includes some funding for schools to help struggling students meet the standard.

These state initiatives raise important questions. Are they good for the students who were retained? And are they good for the overall school system?

Retention has been studied for many years, and the research is reasonably clear. Children who repeat a grade do very poorly in reading and, ultimately, graduation rates compared similarly to their low-achieving age mates who were promoted. In comparison to their new grade mates, they show a short-term gain because they are older when they take the test, but this advantage wears off within a few years.

Reading-by-third-grade policies could end up being beneficial, however, because they provide substantial incentives to use effective strategies to prevent reading failure. Holding a child back is very expensive, incurring an extra year of per-pupil cost (roughly $10,000). Many states or districts have as many as 50 percent of third graders who might not meet the standards, so a school of 500 students holding back 50 percent of third graders would be failing about 35 children at a cost of $350,000 per year.

School leaders now have whatever their state gives them to reduce retentions, and they have a good reason to spend whatever it takes to reduce their retention rates and save a lot of money; $350,000 is a lot of money to solve a well-defined problem concentrated in pre-K through third grade.

There is no aspect of school improvement more extensively studied than preventing reading failure in the early grades. There are many proven programs, especially one-to-one and small-group tutoring, cooperative learning, and whole-school reform models.

But many schools in reading-by-third-grade states are simply hiring reading specialists to tutor children one-to-one. While tutoring can be very effective, in most schools one reading specialist cannot possibly tutor enough students to put them all over the line.

The research would suggest that different types of intervention are needed for different children depending on their age and how far behind they are, as in the made-up curve below. In this curve, students in the green section are likely to be reading at grade level, though high-quality teaching is still needed to keep them on track. Those in the yellow section are not on track for success, but are not too far from the criterion. Those in the orange and red zones need intensive help to reach grade level.

2013-01-28-HP2Graph1.jpg

A school strategy only focused on the orange and red zones, with one-to-one tutoring, would not be very effective, as they have such a long way to go. Using tutors just for students in the yellow zone is ethically questionable and is still unlikely to get to enough children.

Instead, schools need a thoughtful, integrated strategy to get the maximum number of students to the standard. This could involve using proven but relatively inexpensive strategies for all students, high-quality small group tutoring for students in the yellow and orange zones, and high-quality one-to-one tutoring for children in the red zone, as illustrated below.

2013-01-28-HP2Graph2.jpg

If schools use the reading-by-third-grade movement as an opportunity to use proven practices throughout the primary grades, they can reap substantial savings by avoiding unnecessary retentions, and most importantly, they can make a life-changing difference for all of their students.