Why Leave Learning to Chance?

Every year about four million kindergartners enter America’s schools. They’re all excited, eager and confident, because that’s the nature of kindergartners, but unfortunately, we adults know better. We know that among those wonderful five year olds, 65% will reach fourth grade reading below the “proficient” level on the National Assessment of Educational Progress (NAEP), and 31% will not even reach the “basic” level. We know which students in which neighborhoods are most likely to have these problems. Since 1980, the story has hardly changed.

Today, I’m writing this blog from an airplane flying from Baltimore to San Francisco. Flying was a risky business long ago, but today the chances are infinitesimal that my airplane will crash.

So here’s a question. Why is it ok to leave the reading success of children to chance? Why don’t we treat reading success the way we treat air safety, as something to ensure no matter what?

If you think we don’t yet know how to ensure the reading success of all children, you might be right, but I can tell you that we absolutely do know how to ensure a much higher level of success than we have now, with today’s teachers and today’s schools. I was recently reviewing research evaluating reading programs, and I found more than 60 different programs with moderate to strong evidence of effectiveness: one-to-one and one-to-small group tutoring, classroom methods, school-wide reforms, and technology. Over time, it’s certain that these approaches, and combinations of them, could become more and more effective, and we could approach 100% success.

Getting to 100% will require more than just better instruction. We are doing a study in high-poverty schools in Baltimore and found that while at least 21% of second and third graders need glasses, only 6% have them. I’m sure there are similar stories relating to hearing, dental, health, and mental health. Absenteeism is another blocker, and there are more. If we want to get to 100%, we have to deal with all of these.

Well sure, you might say, but how could we afford all of this? Fortunately, the most widespread reading problems can be solved inexpensively. The average annual per-pupil cost in the U.S. is about $11,000. The annual cost of our proven Success for All reading program is around $100 additional, or less than 1% of what we are already spending. Two pairs of eyeglasses — one to take home and one to leave at school — including the eye exam and glasses replacement, costs less than $50. Proven tutoring models provided by paraprofessionals can cost as little as $400 per student, but even at $2000 for one-to-one tutoring, that’s 18% of average per-pupil cost, and for only a minority of the class.

These modest expenditures on proven programs quickly pay back their costs in terms of reducing special education and retention, much less long-term benefits to children and society. Yet none of the 60 proven and promising programs I found is in truly widespread use.

On my airplane, of course, the situation is quite different. Pilots are carefully and extensively trained in proven methods. Technology is constantly developing to provide information and automated assistance to ensure safety and effectiveness. Back-up systems ensure that if things go wrong despite the best of preparation, disaster will not result. All of these systems are constantly evolving in response to development, evaluation, and implementation of innovations.

The reading success of a child is a very serious matter. It simply makes no sense to treat it any less seriously than we treat air safety. Just as on airplanes, we need systems to monitor children’s success, not to punish teachers but to know when and how to intervene if trouble arises.

Perhaps someday, we’ll put Boeing or Lockheed Martin in charge of our schools, and charge them with getting us as close as possible to 100% success in reading. I can see it now.

Proven approaches to:

Phonemic awareness? Check
Phonics? Check
Vocabulary? Check
Fluency? Check
Comprehension? Check
Vision? Check
Hearing? Check
Tutoring backup? Check

Ready for takeoff!

Of course we can solve this problem. All we have to do is to decide it must be solved and then do it. It is neither efficient nor ethical to keep accepting the number of reading disasters we experience in our schools.

Evidence-Based Reform Is Irreversible

Today’s blog is a celebration of a milestone, of a sort. This is my 100th blog on evidence-based research in education in the Huffington Post. If you add on blogs I used to do for Education Week, it’s about 260, but 100 is a rounder number. This milestone provides an occasion to step back and reflect on where we are in evidence-based reform.

Evidence-based reform has its problems and pitfalls. It may be that Congress will abolish the Investing in Innovation (i3) program. It may be that many promising programs will turn out not to make a significant difference. It may be that pushback will increase from people who oppose evidence for any of several reasons. But in education, the evidence genie is out of the bottle, and it is never returning. I say this with confidence because I have never heard of a field that embraced evidence as a basis for policy and practice and then abandoned it. It is too powerful an idea, and at the end of the day, evidence provides information to government, educators and citizens to enable schools to use best practices to progressively improve outcomes for children. This is not, or should not be, a partisan issue. Everyone shares a concern for solid, provable outcomes. No one wants to spend money without seeing a result.

As recently as ten years ago, it was not at all clear that evidence-based reform would prevail. There were so few programs that met high or moderate standards of evidence that the What Works Clearinghouse was widely called the “Nothing Works Clearinghouse.” Too few researchers were capable of carrying out sophisticated cluster randomized trials. Federal and foundation investment in R & D was insufficient. Since then, however, progress has been remarkable. Investing in Innovation (i3), the Institute of Education Sciences (IES) at the U.S. Department of Education, and other funders have supported a great deal of development and evaluation of new programs, as well as building evaluation capacity across the country. i3 also accelerated the dissemination of proven models. Congress added a new category of allowable uses for School Improvement Grants (SIG) focused on proven, whole-school reform models. The Title II Supporting Effective Educator Development (SEED) program now requires applicants to show evidence of effectiveness. Any of these individual developments could be reversed, but the broader movement has a momentum now that goes beyond any single program or set of programs.

For a long time, education has paid little attention to evidence. There are still far too few researchers, educators and policy-makers involved in the evidence movement. Yet there are enough people who are committed to evidence to keep the movement going through good times and bad. Opponents abound, but who wants to stand up for ignorance? I am optimistic that the evidence movement will prevail. As Winston Churchill said, “Americans will always do the right thing, after having exhausted all the other possibilities.” I think we are at that point in education.

Columbus and Replicability

Happy Columbus Day!

Columbus is revered among researchers because:

1. He didn’t know where he was going;
2. He didn’t know where he was when he got there; and
3. He did it all on government money.

Columbus gets a lot of abuse these days, and for good reason. He was a terrible person. However, people also say that he didn’t actually discover America. Leif Erikson had been here earlier, they say, and of course the Indians were already here.

What Columbus did discover was not America per se, but a replicable and openly published route to America. And that’s what made him justifiably famous. In research, as in discovery, what matters is replicability, the ability to show that you can do something again, and to tell others how they can do the same. Columbus was indisputably the first to do that (Leif Erikson kept his voyage secret).

Replicability is the hallmark of science. In science, if you can’t do it again, it didn’t happen. In fact, there is a popular science humor magazine called the Journal of Irreproducible Results, named for this principle.

As important as replication is in all of science, it is rare in educational research. It’s difficult to get funding to do replications, and if you manage to replicate a finding, journal editors are likely to dismiss it (“What does this add to the literature?” they say). Yet as evidence-based reform in education advances, the need for replication increases. This is a problem because, for example, the majority of programs with at least one study that met What Works Clearinghouse standards had exactly one study that did so.

Soon, results will become available for the first and largest cohort of projects funded by the Investing in Innovation (i3) program. Some of these will show positive effects and some will show outcomes close enough to significance to be worth trying again. I hope there will be opportunities for these programs to replicate and hopefully improve their outcomes, so we can expand our armamentarium of replicable and effective approaches to enduring problems of education.

We really should celebrate Columbus Day on November 3rd when Columbus returned to the New World. The day he reached the New World was a significant event, but it wasn’t really important until he showed that he (and anyone else) could do it again.

Beware of Do-It-Yourself Assessments

Faithful readers of this blog, and followers of the Best Evidence Encyclopedia (BEE), will know that I am always cautioning readers of program evaluations to pay no attention to findings from measures overly aligned with the experimental but not the control treatment. For example, when researchers teach a set of vocabulary words to the experimental students (but not the controls), it is not surprising to find strong impacts. Unfortunately this happens all too often, but we carefully winnow such measures out of our BEE reviews.

In a recent paper written with my colleague Alan Cheung, we looked at 645 studies accepted across all BEE reviews done so far to find out which methodological factors are associated with excessive, improbable effect sizes. In an earlier blog I wrote about the profound impact of sample size: small studies get (improbably) big effect sizes.

Another important factor, however, was the use of experimenter-made measures. Even after our careful, conservative weeding out of studies with over-aligned measures, we were surprised to find out that effect sizes on measures made by experimenters were twice as high as effect sizes on measures made by someone else (usually standardized tests).

It may be going too far to suggest that no one should ever use or accept experimenter-made measures, no matter how fair they appear to be to the experimental and control groups. However, what it does say is that we need to be very cautious in accepting experimenter-made measures. Standardized tests are far from perfect, but they are almost always fair to experimental and control groups, as control teachers can be assumed to be trying as hard as experimental teachers to improve outcomes on these measures. This may not be so on experimental-made tests.

I’m all for do-it-yourself cooking, home repairs, and other projects. But when it comes to do-it-yourself educational measurement, let the reader beware!

Spell it Éxito or Success — That’s What Hispanic Students Need

I want to wish everyone a happy and reflective Hispanic Heritage Month (September 15 – October 15). This must be a disturbing time for Hispanics, with presidential candidates competing to say terrible and ignorant things about them.

Americans love to speak with pride about our nation of immigrants, often as a prelude to telling their own immigration story. Our history does record the struggles immigrants had to overcome to find their place in America, but the sad part is that the struggles are often due to our own government, our own people and even (in many cases) immigrants from the same places who got here a little earlier.

In the long run, I think it is certain that Hispanics will become successful, respected and accepted, as the many middle class Hispanics already are. However, our schools could do a lot to help ease the transition so that Hispanics and other immigrants can reach economic security and social acceptance much faster.

The key educational issue with Spanish-dominant Hispanics and other English learners is figuring out how to help them learn English without slowing down their learning of everything else they need to learn in school. For decades, there have been highly contentious and political battles about whether Spanish-dominant children should be taught to read first in Spanish and then transitioned to English by about the third grade, or whether they should be taught in English from the outset. Along with colleagues, I did a large, randomized experiment comparing these strategies. The result? By fourth grade, there was no significant difference between the two groups.

Our findings confirmed a growing sense among advocates for English learners that fighting about language of instruction is not as important as ensuring quality of instruction. Along with colleague Alan Cheung, I wrote a review of research on effective reading programs for English learners. There were many proven and promising programs. Yet still today, there remains far more discussion about language than about effective instruction.

To me, this is one more area in which research and development in education can shift the debate and lead to genuine progress. So often, opposing sides make passionate arguments for or against some aspect of practice. Both sides are only interested in research that supports their preconceived position, and reject all other evidence. Yet often, the evidence points to solutions that are effectively neutral on the contentious issue. In the case of Spanish-dominant students, there is plenty of rigorous research to indicate that they benefit from cooperative learning, one-to-one and small-group tutoring, and instruction in metacognitive and learning-to-learn skills, for example. None of these solutions depends much on the language of instruction, so they do not get the attention the fight over bilingual education gets. Yet if everyone can stay focused on what is best for children rather than winning an argument, we’re likely to see the outcomes that everyone wants.

I’m not saying that language of instruction is unimportant, and there needs to be further research on how to help students who start off with limited English to succeed in school and maintain their home language. But whether you call it éxito or success, that’s what Hispanic kids need in school, whichever language is emphasized in the first few years.