What Would Evidence-Based Policy Look Like in Education?

Note: Steve Fleischman, deputy executive officer at Education Northwest, writes this guest post.

Is evidence-based policy an oxymoron? Is it possible to have evidence serve as a guide rather than merely as a justification for policy? I think there are two ways in which evidence can play a key role in school improvement.

The first is that evidence can help us identify high-leverage problems that create policy priorities. These are problems that, if solved, would reduce a large percentage of the variance between good and bad outcomes for kids. In 2004, for example, Paul Barton suggested a list of 14 factors correlated to high student achievement, including out-of-school factors such as hunger and nutrition, reading to children, and student mobility. He also listed in-school factors such as teacher quality, rigor of curriculum, and school safety. Barton noted that low-income and minority children are at a disadvantage in most of these areas.

Evidence-based policy would dictate that we concentrate on these, or other similar high-leverage conditions, to improve education–particularly for our most disadvantaged students. Imagine if instead of spending billions of dollars over the next 10 years on a thousand different efforts we concentrated policy on making sure that students master reading early, have successful transitions from middle to high school, and stay in school at least through high school graduation. After all, evidence from a 2010 study by the Annie E. Casey Foundation points to the “make or break” nature of mastering reading by grade 3 for children’s future educational development; ACT provides evidence that the level of academic achievement attained by eighth-grade students “has a larger impact on their college and career readiness by the time they graduate than anything that happens academically in high school;” and the Everyone Graduates Center describes the devastating consequences that dropping out of high school has on both individuals and society.

Second, evidence-based policymakers can also insist that we judge proposed solutions to high-leverage problems against proof that they can get the job done. Let’s say we are talking about the goal of having all students reading by grade 3. Policymakers should not care whether it is charters, vouchers, homeschooling, non-union or unionized schools, reading programs, or professional development that achieves the desired goal. They should only care about demonstrated results. I agree with Rick Hess when he argues that, “The proper measure of whether proposals are consistent with public schooling ought not be whether power, politics, or finances shift, but whether we are doing a better job of educating all children so they master essential knowledge and skills, develop their gifts, and are prepared for the duties of citizenship.”

I was struck several years ago while reading “Polio: An American Story” how, led by science and the commitments of policy leaders, our entire nation was mobilized in a multi-decade effort to eradicate the dreaded disease. Even Lucy and Desi and other celebrities of the 1950s were engaged in the cause. Today, by combining science and policy, the Bill & Melinda Gates Foundation has extended the fight against polio around the world. Imagine if evidence-based policy could similarly mobilize our entire nation to accomplish a few critical educational outcomes: all children reading by grade 3, successful transitions to high school, and significant reductions in dropout rates. What would our education system look like then?
Steve Fleischman
Education Northwest (educationnorthwest.org), a nonprofit headquartered in Portland, Ore., conducts research, evaluation, technical assistance, training, and strategic communications activities to promote evidence-informed education policy and practice.

The Unmet Promise of Education Technology

In the mid-2000s, the U.S. Department of Education commissioned a large, randomized evaluation of the most widely used computer-assisted instruction (CAI) programs in elementary reading and middle and high school math. Schools were randomly assigned to use one of several CAI programs. The results (published here and here) were dismal. In both subjects and all grade levels, achievement levels were virtually identical for the students who experienced CAI and those who did not. This finding was consistent with the conclusions of recent reviews of research on CAI in reading and math, which find that the higher the quality of the research (e.g. random assignment of large samples), the lower the estimate of CAI effects. In a nation that worships technology and spends billions on technology in schools, the study should have been a wake-up call, but it was hardly covered by the general press (and not so much by the education press). Earlier this month, Matt Richtel of The New York Times did a good job of describing the pressure and appeal for school leaders to adopt technologies that promise improved instruction, efficiency, and results but don’t necessarily have the data to justify the cost.

How could modern CAI programs fail to make much difference in student learning? One clue in the federal study is in the fact that children did not spend many hours on the computers over the course of the year. Perhaps a bigger dose would have a larger effect, although studies of this possibility do not generally find a dosage effect.

Another explanation for both the limited hours of use of CAI (also found in many studies) and the limited impact may be that traditional CAI is just inconsistent with traditional teaching, and is therefore not valued by teachers or integrated very well into daily teaching.

Whatever the explanation, the modest impact of modern CAI programs creates a paradox. On one hand, it is clear that breakthroughs in educational practice (and therefore policy) will involve technology. Excellent professional development can help teachers get better results, but I believe that outstanding, sustainable improvements in daily teaching are going to depend on the extraordinary capabilities of technology. Yet I’d be the first to admit that the track record for technology as it has been used in schools so far is not so great.

I think the greatest promise for innovation in teaching using technology is in applications that fully integrate the two. Currently, some of the best evidence for modern technologies is for programs that cycle children through integrated activities, both with and without technology. Evidence also supports the use of embedded multimedia, where teachers use brief bits of video integrated into their lessons to build motivation and understanding with powerful visuals. Computer-assisted tutoring and small-group tutorials similarly combine the strengths of teachers and technology, and have shown very positive outcomes. A recent study in England showed that use of self-paced electronic response devices to provide immediate feedback to students and teachers increased math learning.

It’s time to rethink the role of technology in instruction, to ask not how technology can mimic (and replace) what good teachers do, but how it can support good teaching. Kids, especially elementary kids, learn in large part because they want to please valued adults, and they learn in collaboration with each other. No computer can replace a teacher’s empathy, enthusiasm, or ability to understand and respond to students’ interests and needs. Yet research in reading and math suggests that technology has enormous potential to add interest, visual images, organization, and assessment to teachers’ lessons and to cooperative interactions among students. The task is to figure out how teachers, peers, and technology can all work together to create effective classrooms.

Research and Innovation: The Way Forward in Education

Fifty-four years ago, America was galvanized when the Soviet Union put a satellite into space. We responded as we always do when we have a national consensus on an important goal: We innovated. We invested heavily not only in rockets, but also in education, to prepare our entire nation to be second to none. I’m old enough to remember how exciting it was to feel a part of the national response to Sputnik. We knew that America would regain its leadership, and it was all up to us kids!

In education today, we wait in vain for the “Sputnik moment,” the time when our leaders decide that falling behind our international peers in academic achievement is no longer acceptable. Instead of investing in research and innovation, as we did in the wake of Sputnik, our leaders today try to solve our educational problems by fiddling with management solutions, governance solutions, and assessment solutions that do not fundamentally change what happens between teachers and students. These policies may be beneficial, but they don’t scare the Finns or the Chinese or even the Canadians who outperform our students. The reason it was Neil Armstrong and not Nikolai Armstronganoff who landed on the moon was that we invested in targeted, relentless research and development. We did not manage our way to the moon, we invented our way to the moon. Dramatic improvements in medicine, agriculture, and technology happened the same way. And so it must be in education.

Sputnik: Advancing Education through Innovation and Evidence is a new blog dedicated to disseminating news and information on research and development in education that could transform teaching and improve student outcomes on a scale that matters. In addition to reporting on research itself, it will focus on policy developments relevant to research and innovation in education. Guest bloggers will present their perspectives on how research and innovation can play a greater role in policy and practice.

This is an exciting time for those who share a belief in research and innovation as the way forward in education. I hope you’ll join me in exploring the outer limits of education reform.