Proven Programs Don’t Implement Themselves

One of the criticisms often leveled at evidence-based reform in education is this: Programs may be proven effective in controlled experiments, but on a larger scale, they won’t be implemented with care and therefore won’t work. I have seen many awful implementations of programs that have been successful elsewhere and I agree that this is a problem. Proven programs don’t implement themselves.

How can we ensure widespread, effective, intelligent use of proven programs? After many years of wrestling with this question, I have a set of principles for ensuring high-quality implementations of proven programs, which I will now reveal to one and all.

1. Make sure teachers chose to implement the program. When anyone is forced to do something, they often do a poor job of it. Work with volunteers. If a program works with individual teachers, let them opt in. If it works with schools, let the teachers vote (Success for All* requires 75% in favor). After you demonstrate local success, then come back and offer the program again to those who chose not to do the program, but start with people who are committed and positive about implementation.

2. The school is the unit of change. It’s very hard for isolated teachers to do serious innovation. Schools taking on programs usually get much better results. In secondary schools, departments may take on this function.

3. Make sure the program itself is well specified. Teachers should have a clear idea of what it is, and have manuals, videos, student materials, and other aids to quality implementation.

4. Provide plenty of training and, even more importantly, follow-up support. Real change is hard, and teachers need both top-quality initial training and visits over time from skilled coaches, who give feedback and help teachers stay on track.

5. Assess implementation and student outcomes. Every few months, look at how teachers are implementing the approach and give them friendly feedback. Look at student data to see that kids are benefiting from the program, and share the data with the teachers.

6. Engage implementers with each other. Teachers implementing new programs need opportunities to share ideas, visit each other’s classes, ask each other for help, and take joint responsibility for outstanding outcomes.

7. Plan for the long haul. The change process goes on forever. If you want quality implementation, plan on sticking at it for a long time, to help school staffs continue to grow in sophistication and skill.

A very wise businessman I know lives by the principle that a mediocre plan well implemented always beats a great plan poorly implemented. I don’t know if that applies in education, but I do know that a great plan implemented with care, fidelity, and intelligence is the only thing that makes a difference. Whatever national or local policies we adopt must make sure that proven programs are outstandingly implemented, especially with the kids who need them most.

For the latest on evidence-based education, follow me on twitter: @RobertSlavin

* Robert Slavin is Chairman of the Board of the Success for All Foundation

When Every Student Has a Computer at Home

The history of technology in education is one of schools running to catch up with technology developments outside of school. Learning from our past follies in this area, perhaps now it is time we anticipate how ubiquitous computer access can be achieved and then exploited to benefit all children.

Computers have been used in schools since the early 1970s, and today there are at least a few in virtually every school, and most students now have access at home. But, the fact that not all students have their own computers at home limits instructional uses of computers in as well as out of school. A teacher cannot, for example, assign homework requiring a computer if only 20 out of her 25 students have working computers at home.

A number of developments are about to change this. One is the rapid adoption of smart phones and tablet computers, which are making electronic access more common and less expensive at home. The second is the impending extinction of the paper textbook, which may soon create an enormous economic incentive for school districts or states to give away inexpensive computers to students, so that they can replace paper textbooks with cheaper e-texts.

When every student has a computer, the educational uses of technology can be transformed, especially if innovators seize the opportunity to create exciting, interactive software rather than simply converting linear textbooks into their electronic equivalents.

Ubiquitous access to computers could solve one of the key limitations on educational computer apps, the fact that devices available only during the school day conflict with the structure of school, which typically has teachers and classrooms designed for group teaching, not individualized, self-paced work. As a result, actual use of computers documented by research is far less than what advocates or providers might hope, and this may explain the limited impacts of computer-assisted instruction on reading and math learning.

If teachers could assign daily homework on computers, students could on their own do work that fills in gaps in their learning. Students performing ahead of the class could go into advanced topics. Just as kids play games not only with the computers, but also with others virtually anywhere else, if students have just read an assigned book or learned about a given topic, they could engage in guided discussions online about that book or topic with e-pals anywhere in the world.

Ubiquitous computers at home could enable students to watch video content at home linked to topics they are learning in school. Why take class time to show a video when it can be viewed as homework? This opens up new uses for already-produced as well as new educational programming while also offering teachers more time on task in the classroom.

Despite the rapid approach of ubiquitous computer availability at home, I am not aware of much research or development anticipating this eventuality. There are thousands of educational apps, games, videos, and other content being created each year, of course, but is anyone weaving these into complete approaches to reading, math, science, or social studies instruction and evaluating the outcomes? It is time for the education R&D infrastructure to jump in with both feet to ensure that we are not just creating new products for young consumers. Guided by research and a mission to improve outcomes for kids, we can shift the market’s focus to effective tools for students eager to learn in the technological world they are already inhabit.

For the latest on evidence-based education, follow me on twitter: @RobertSlavin

Are Evidence and Innovation in Conflict?

I do a fair amount of speaking on the importance of evidence-based reform in education, and I hear a disturbing objection to this idea: insisting on evidence for educational programs will slow down the process of innovation.

At one level, this is an astonishing argument. If we don’t know if innovations work, why should we care if they are not widely used? Actually, at least 99% of the textbooks, software, and professional development approaches adopted by schools today have never been evaluated successfully in comparison to control groups, so complaining about evidence-based reform is a bit premature.

However, there is a genuine dilemma. Technology innovations, for example, develop very rapidly, and it takes time to do proper evaluations. If government insisted that schools use only proven programs, there would be a danger that promising innovations might never get established, or might be obsolete by the time the report appears.

The solution to this dilemma is to encourage but not require schools to adopt proven programs. Government could offer special incentive funds (as in the Obey-Porter Comprehensive School Reform program of the late 1990s) or by awarding 5 to 10 competitive preference points for schools that commit to implementing proven programs with fidelity. Beyond encouraging schools themselves, these incentives would encourage developers and innovators to create and evaluate promising approaches, knowing that if they work, many schools may adopt them.

The whole idea of evidence-based reform is to go beyond ideas and create a system to support the development, evaluation, and dissemination of effective innovations. Wise policies would support all steps in this process, including step one: the crucial development and piloting of promising innovations.

Shifting Government from “Who Gets What” to “What Works”

Every political science student knows the old adage that the focus of government is “Who gets what?” That is, government takes in taxes and then distributes benefits, and contending groups pressure government to increase the proportion of those benefits delivered to their constituents.

The “who gets what” dynamic exists as much in education as anywhere else, and perhaps even more, as education (like the military) is overwhelmingly a government-funded operation. Whenever the annual budget numbers come out for the U.S. Department of Education, advocacy groups scan every figure looking for gains or losses in their favored line items. There is nothing wrong with this, but the laser focus on line items may be distracting us from a focus on the more important question: are we getting better able to solve the enduring problems of education? Do we know more about “what works” and how to use public funds to support proven approaches?

The Obama Administration’s Investing in Innovation (i3) initiative is an excellent example of a “what works” approach rather than a “who gets what” mentality. It is funding a broad array of educational innovations to scale up proven ones and help developers of new approaches build capacity and effectiveness. Senator Bennet is advancing a proposal to create a set-aside in i3 for a new ARPA-ED initiative, modeled after the Defense Department’s successful DARPA. Unique to this proposal, discussed this week at the American Enterprise Institute, is that it would explicitly avoid “who gets what” structure and would focus directly on creating groundbreaking new technological capabilities in education. In fact, when prodded about how this would benefit rural communities, the Department of Education’s Jim Shelton boldly indicated that that kind of small thinking would not transform the way we educate kids in this country, and we instead need to focus on capabilities above special interests.

The i3 funding is unprecedented, but it is still a tiny slice of federal education funding. Over time, it is sure to increase the number of proven, replicable programs from which schools can choose. If you view the $150 million per year i3 and a potential set-aside for ARPA-ED initiatives as a strategy to improve the impact of the $14 billion Title I program, you have to conclude that i3 and ARPA-ED are extremely cost-effective investments. Yet in the media, percentage increases in big line items like Title I are widely reported and debated, while the smaller line items, like research, development, and dissemination, are lost in the small print.

Government reports budget numbers annually and National Assessment of Educational Progress data every few years. Perhaps in addition to these reports, the federal government should publish a regular report on how much more we’ve learned over the past period about how to improve student learning and other outcomes. Maybe focusing on and identifying advances in proven capacity to improve student outcomes would encourage legislators to invest in this capacity and encourage educators to use it, perhaps moving education policy toward more of a “what works” focus.

I am more hopeful than ever that we are headed in the right direction, but we have a way to go to make evidence the centerpiece of a new reality of teaching children effectively.

For the latest on evidence-based education, follow me on twitter: @RobertSlavin

Disclosure: Dr. Slavin’s organization, the Success for All Foundation, is a recipient of federal Investing in Innovation funds.