First, Do No Harm: The Blind Duchess

One of the great strengths of the evidence movement in education has been its bipartisan nature. Democrats and Republicans, liberals and conservatives have equal reasons to want to know what works, and to try to ensure that government funds will be spent primarily on programs and practices known to work from rigorous experiments. Politics plays a legitimate role in determining how evidence is put to use and what values should underpin policies in education, but whatever one’s politics, everyone should agree that it’s essential to know what works.

Yet while it’s easy to conclude that we should promote what does work, it’s not so easy to decide what to do in areas in which there is insufficient evidence. We want to gradually replace programs and practices not known to work with those that do have strong evidence, but what do we do while the evidence base is growing?

I recently took a tour of Chatsworth, a huge, ornate great house that since the 1600s has been the family seat of the Dukes of Devonshire, one of the wealthiest families in England. Our guide told us about a famous duchess, Georgiana (a distant ancestor of Lady Diana). In the late 1700s, Georgiana suffered from irritated eyes. Her physician had her bathe her eyes in a mixture of milk and vinegar, and then applied leeches. As a consequence, she went blind.

The duchess’ physician ignored the first principle of medicine, stated in the Hippocratic Oath that every doctor swears: “First, do no harm.” I think it is safe to assume that the Duchess of Devonshire could have had any doctor in Europe, and that the one she chose was considered one of the best. Yet even a duke or duchess or a king or queen could not obtain the kind of routine medical care we take for granted today. But what their doctors could at least do was to take care to avoid making things worse. Recall that around the same time, King George III suffered from insanity, perhaps caused by his physicians, and George Washington was killed by his leech-using doctors.

Today, in education, we face a different set of problems, but we must start with the Hippocratic principle: First, do no harm. But for us, doing no harm is less than straightforward.

In educational practice, we have a growing but still modest number of proven interventions. As I’ve noted previously, our Evidence for ESSA website contains approximately 100 reading and math programs for grades K-12 that meet current ESSA evidence standards. That’s impressive, but it is still a smaller number of proven programs than we’d like, especially in secondary schools and in mathematics. We are now working on the category of science, which has fewer proven programs, and we know that writing will have fewer still.

In all of education research, there are very few programs known to do actual harm, so we don’t really have to worry too much about the Duchess of Devonshire’s problem. What we have instead is a growing number of proven and promising programs and a very large number of programs that have not been evaluated at all, or not well enough to meet current standards, or with mixed outcomes.

For educators, “First, do no harm” may be taken to mean, “use programs proven to be effective when they exist, but stick with promising approaches until better ones have been validated.” That is, in areas in which there are many programs with strong, positive evidence of effectiveness, select one of these and implement it with care. But in areas in which few programs exist, use the best available, rather than insisting on perfect evidence.

One example of what I am talking about is after-school programs. Under federal funding called 21st Century Community Learning Centers (21st CCLC), after-school programs have been widespread. Several years ago, an evaluation of 21st CCLC found few benefits for student achievement, and there are few if any proven models in broad scale use. So how should the federal government respond?

I would argue that the principle of “First, do no harm” would support continuing but significantly modifying 21st CCLC or other after-school funding. Federal support for after school programs might be reformed to focus on development and evaluation of programs that improve achievement outcomes. In this way, federal dollars continue to support a popular and perhaps useful service, but more importantly they support R&D to find out which forms of that service produce the desired outcomes. The same approach might be applied to career and technical education and many other areas in which there is substantial federal, state, or local investment, but little evidence of what works. In each case, funds currently supporting popular but unproven services could be shifted to supporting development, evaluation, and dissemination of proven, effective strategies designed to meet the activity’s goal.

Instead of potentially harming students or taking away funding altogether, such a strategy could open up new areas of inquiry that would be sure to eventually create and validate effective programs where they do not exist today.

In education, “First, do no harm” should not justify abandonment of whole areas of education services that lack a sufficient selection of proven approaches. Instead, it means supplementing service dollars with R&D dollars to find out what works. We cannot justify the kinds of treatment the Duchess of Devonshire received for her irritated eye, but we also cannot justify using her case to give up on the search for effective treatments.

This blog is sponsored by the Laura and John Arnold Foundation

First, Do No Harm: The Blind Duchess

Today, in education, we face a different set of problems, but we must start with the Hippocratic principle: First, do no harm. But for us, doing no harm is less than straightforward.

This blog is sponsored by the Laura and John Arnold Foundation

Research and Practice: “Tear Down This Wall”

I was recently in Berlin. Today, it’s a lively, entirely normal European capital. But the first time I saw it, it was 1970, and the wall still divided it. Like most tourists, I went through Checkpoint Charlie to the east side. The two sides were utterly different. West Berlin was pleasant, safe, and attractive. East Berlin was a different world. On my recent trip, I met a young researcher who grew up in West Berlin. He recalls his father being taken in for questioning because he accidentally brought a West Berlin newspaper across the border. Western people could visit, but western newspapers could get you arrested.

I remember John F. Kennedy’s “Ich bin ein Berliner” speech, and Ronald Reagan’s “Mr. Gorbechev, tear down this wall.” And one day, for reasons no one seems to understand, the wall was gone. Even today, I find it thrilling and incredible to walk down Unter den Linden under the Brandenburg Gate. Not so long ago, this was impossible, even fatal.

The reason I bring up the Berlin Wall is that I want to use it as an analogy to another wall of less geopolitical consequence, perhaps, but very important to our profession. This is the wall between research and practice.

It is not my intention to disrespect the worlds on either side of the research/practice wall. People on both sides care deeply about children and bring enormous knowledge, skill, and effort to improving educational outcomes. In fact, that’s what is so sad about this wall. People on both sides have so much to teach and learn from the other, but all too often, they don’t.

What has been happening in recent years is that the federal government, at least, has been reinforcing the research/practice divide in many ways, at least until the passage of the Every Student Succeeds Act (ESSA) (more on this later). On one hand, government has invested in high-quality educational research and development, especially through Investing in Innovation (i3) and the Institute of Education Sciences (IES). As a result, over on the research side of the wall there is a growing stockpile of rigorously evaluated, ready-to-implement education programs for most subjects and grade levels.

On the practice side of the wall, however, government has implemented national policies that may or may not have a basis in research, but definitely do not focus on use of proven programs. Examples include accountability, teacher evaluation, and Common Core. Even federal School Improvement Grants (SIG) for the lowest-achieving 5% of schools in each state had loads of detailed requirements for schools to follow but said nothing at all about using proven programs or practices, until a proven whole-school reform option was permitted as one of six alternatives at the very end of No Child Left Behind. The huge Race to the Top funding program was similarly explicit about standards, assessments, teacher evaluations, and other issues, but said nothing about use of proven programs.

On the research side of the wall, developers and researchers were being encouraged by the U.S. Department of Education to write their findings clearly and “scale up” their findings to presumably eager potential adopters on the practice side. Yet the very same department was, at the same time, keeping education leaders on the practice side of the wall scrambling to meet federal standards to obtain Race to the Top, School Improvement Grants, and other funding, none of which had anything much to do with the evidence base building up on the research side of the wall. The problem posed by the Berlin Wall was not going to be resolved by sneaking well-written West Berlin newspapers into East Berlin, or East Berlin newspapers into West Berlin. Rather, someone had to tear down the wall.

The Every Student Succeeds Act (ESSA) is one attempt to tear down the research/practice wall. Its definitions of strong, moderate, and promising levels of evidence, and provision of funding incentives for using proven programs (especially in applications for school improvement), could go a long way toward tearing down the research/practice wall, but it’s too soon to tell. So far, these definitions are just words on a page. It will take national, state, and local leadership to truly make evidence central to education policy and practice.

On National Public Radio, I recently heard recorded recollections from people who were in Berlin the day the wall came down. One of them really stuck with me. West Berliners had climbed to the top of the wall and were singing and cheering as gaps were opened. Then, an East German man headed for a gap. The nearby soldiers, unsure what to do, pointed their rifles at him and told him to stop. He put his hands in the air. The West Germans on the wall fell silent, anxiously watching.

A soldier went to find the captain. The captain came out of a guardhouse and walked over to the East German man. He put his arm around his shoulders and personally walked him through the gap in the wall.

That’s leadership. That’s courage. It’s what we need to tear down our wall: leaders at all levels who actively encourage the world of research and the world of practice to become one. To do it by personal and public examples, so that educators can understand that the rules have changed, and that communication between research and practice, and use of proven programs and practices, will be encouraged and facilitated.

Our wall can come down. It’s only a question of leadership, and commitment to better outcomes for children.

This blog is sponsored by the Laura and John Arnold Foundation

The Age of Evidence

In 1909, most people outside of cities had never seen an automobile. Those that existed frequently broke down, and there were few mechanics. Roads were poor, fuel was difficult to obtain, and spare parts were scarce. The automobile industry had not agreed on the best form of propulsion, so steam-powered cars, electric cars, and diesel cars shared the road with gasoline-powered cars. The high cost of cars made them a rich man’s hobby and a curiosity rather than a practical necessity for most people.

Yet despite all of these limitations, anyone with eyes to see knew that the automobile was the future.

I believe that evidence in education is at a similar point in its development. There are still not enough proven programs in all fields and grade levels. Educators are just now beginning to understand what proven programs can do for their children. Old fashioned textbooks and software lacking a scintilla of evidence still dominate the market. Many schools that do adopt proven programs may still not get promised outcomes because they shortchange professional development, planning, or other resources.

Despite all of these problems, any educator or policy maker with eyes to see knows that evidence is the future.

There are many indicators that the Age of Evidence is upon us. Here are some I’d point to.

· The ESSA evidence standards. The definitions in the ESSA law of strong, moderate, and promising levels of evidence and incentives to use programs that meet them are not yet affecting practice on a large scale, but they are certainly leading to substantial discussion about evidence among state, district, and school leaders. In the long run, this discussion may be as important as the law itself in promoting the use of evidence.

· The availability of many more proven programs. Our Evidence for ESSA website found approximately 100 K-12 reading and math programs meeting one of the top three ESSA standards. Many more are in the pipeline.

· Political support for evidence is growing and non-partisan. Note that the ESSA standards were passed with bipartisan support in a Republican Congress. This is a good indication that evidence is becoming a consensus “good government” theme, not just something that professors do.

· We’ve tried everything else. Despite their commendable support for research, both the G.W. Bush and the Obama administrations mainly focused on policies that ignored the existence of proven programs. Progress in student performance was disappointing. Perhaps next time, we’ll try using what works.

Any of these indicators could experience setbacks or reversals, but in all of modern history, it’s hard to think of cases in which, once the evidence/innovation genie is out of the bottle, it is forced back inside. Progress toward the Age of Evidence may be slower or more uneven than we’d like, but this is an idea that once planted tends to persist, and to change institutions.

If we have proven, better ways to teach reading or math or science, to increase graduation rates and college and career readiness, or to build students’ social and emotional skills and improve classroom behavior, then sooner or later policy and practice must take this evidence into account. When it does, it will kick off a virtuous cycle in which a taste for evidence among education leaders leads to substantial investments in R&D by government and the private sector. This will lead to creation and successful evaluation of better and better educational programs, which will progressively add to the taste for evidence, feeding the whole cycle.

The German philosopher Schopenhauer once said that every new idea is first ridiculed, then vehemently opposed, and then accepted as self-evident. I think we are nearing a turning point, where resistance to the idea of evidence of effectiveness as a driver in education is beginning to give way to a sense that of course any school should be using proven programs. Who would argue otherwise?

Other fields, such as medicine, agriculture, and technology, including automotive technology, long ago reached a point of no return, when innovation and evidence of effectiveness began to expand rapidly. Because education is mostly a creature of government, it has been slower to change, but change is coming. And when this point of no return arrives, we’ll never look back. As new teaching approaches, new uses of technology, new strategies for engaging students with each other, new ways of simulating scientific, mathematical, and social processes, and new ways of accommodating student differences are created, successfully evaluated, and disseminated, education will become an exciting, constantly evolving field. And no one will even remember a time when this was not the case.

In 1909, the problems of automotive engineering were daunting, but there was only one way things were going to go. True progress has no reverse gear. So it will be in education, as our Age of Evidence dawns.

This blog is sponsored by the Laura and John Arnold Foundation

High-Reliability Organizations

I’m writing this blog from the inside of an airplane high above the Atlantic. I have total confidence that my plane will deliver me safely to Europe. It’s astonishing. The people who run every aspect of this plane are ordinary folk. I knew a guy in college who spent his entire career as a pilot for the very airline I’m flying today. He was competent, smart, and very, very careful. But he was not expected to make things up as he went along. He liked to repeat an old saying: “There are old pilots and there are bold pilots, but there are no old, bold pilots.”

When I was younger, I recall that airplane crashes were relatively common. These were always prominently reported in the news. But today, airplane disasters not caused by terrorists or crazy people are extremely rare. The reason is that air disasters are so catastrophic that airlines have adopted procedures in every aspect of their operation to ensure that planes arrive safely at their destinations. Every system important to safety is checked and rechecked, with technology and humans backing each other up. I happen to have a nephew who is studying to be an aircraft mechanic. His course is extremely rigorous. Most people don’t make it through. His final test, he says, will have 80 questions. The minimum acceptable score: 80. His brother is a nuclear engineer on a navy submarine. Same kind of training, same requirement for success. No room for error. The need for such care in airplanes and submarines is obvious. But why not in education?

My friend and colleague Sam Stringfield had this idea many years ago. Based on it, he and a Welsh colleague, David Reynolds, created what they called “high-reliability schools.” They evaluated them in Wales, and found substantially greater gains in schools using this approach than in control schools.

Despite its success, the high-reliability idea did not catch hold in education. Yet any student who is unnecessarily failing in school is a catastrophe waiting to happen. You don’t need a lot of data tables to be convinced that students not reading well by third grade are headed for big trouble. They are disproportionately likely to end up in special education, to repeat one or more grades, to drop out of high school, and to get into behavioral difficulties and problems with the law. Each of these outcomes is hugely damaging to the student and hugely expensive to the taxpayer.

Yet there is no problem in all of education that is better researched than early reading failure. There are many proven strategies known to greatly reduce reading failure: whole school methods, small group, individual tutoring, technology, and more. Our Evidence for ESSA web site lists dozens of proven approaches. It is probably already the case that any school could identify students at risk of reading failure in kindergarten or first grade and then apply proven, easily available methods conscientiously to ensure that virtually every child will succeed in reading.

The point here is that if we wanted to, we could treat early reading the way airlines and submarines treat safety, as a life or death issue.

If schools accepted the high-reliability challenge for early reading, here is what they would do. First, they’d adopt proven pre-reading programs for pre-kindergarten, and then proven beginning reading programs for grades K-3. Teachers of these grades would receive extensive professional development and then in-class coaching to help them use these proven strategies as well as they were used in the research that validated them, or better.

Starting in kindergarten, we’d start to assess students in early reading skills, so we’d know which students need assistance in which specific skills. We’d continue to assess all students over time to be sure that all are on a path to success. The assessments would include vision and hearing so that problems in these areas are solved.

Each school would have staff trained and equipped to provide an array of services for students who are in need of additional help. These would include small-group tutoring for students with mild problems, and one-to-one tutoring for more serious problems. Multiple proven programs, each focusing on distinct problems, would be ready to deploy for students who need them. Students who need eyeglasses, hearing accommodations, or other health assistance would be treated. Students who are English learners would receive assistance with language and reading.

The point is, each school would be committed to ensuring the success of every child, and would be prepared to do so. Like my high-reliability nephews, the goal of every person in every school would be zero failures. Not just fewer. Zero.

There is no question that this goal could be accomplished. The only issue is whether it could be accomplished at a cost that would be politically acceptable. My guess is that a full-scale, replicable schoolwide strategy to ensure zero reading failures in high-poverty schools could add about $200 per child per year, from grades pre-K to 3. A lot of money, you say? Recall from a previous blog that the average per-pupil cost in the U.S. is approximately $11,000. What if it were $11,200, just for a few years? The near-term savings in special education and retentions, much less longer-term costs of delinquency and dropout, would more than return this investment.

But more than cost-effectiveness, there is a moral imperative here. Failing children who could succeed is simply wrong. We could greatly reduce or eliminate this problem, just as the aircraft industry has done. Our society must come to see school failure as the catastrophe that it is, and to use whatever proven methods are needed to make reading failure a problem of the past.

This blog is sponsored by the Laura and John Arnold Foundation

Teachers’ Roles in Evidence-Based Reform

Long ago, I was a special education teacher in Oregon. Someone at that time came up with the idea that since learning was harder in the earlier stages of a skill than it was later on, progress on advanced levels of a skill should be more rapid. They gave us teachers “six-cycle graph paper” to graph individual children’s progress on each skill we taught them. The graph paper was marked to show logarithmic growth, where the units were large at the low levels of a skill and small at the high levels, so a progress chart would show as a straight line.

Does this make any sense to you? Well, it did not make any sense to me and my fellow-teachers, either. Yet it was presented to us as mandatory, a state or district requirement.

We tried, but we just couldn’t make the six-cycle graph paper work. It required a massive amount of paperwork and calculation, taking us away from teaching. We staged a small revolt, and within less than a month, as I recall, six-cycle graph paper was only used for kids to color on.

Six-cycle graph paper was a foolish idea from the beginning. Every teacher in my school, and (I’d guess) in every school could see that it was unworkable and hopeless. But nobody asked us. No research supporting its use was even presented to us, if it existed (which I very much doubt). Further, and perhaps most damaging, the six-cycle graph paper experience eroded our faith in our own school administration and in innovation itself, and it made us less likely to implement other innovations that might be more promising.

After my time as a teacher, I went to graduate school, and then I began working on development, evaluation, and dissemination of cooperative learning strategies in elementary and middle schools. My colleagues and I began our dissemination efforts by doing large, voluntary workshops for teachers. Any teacher of any subject or grade level could attend, so we were usually working with just a few teachers from any one school. We rapidly learned that this kind of scattershot professional development was extremely popular, but it did not stick very well in the schools. The problem was that isolated teachers had difficulty maintaining an innovation without the support of their administration and peers.

Nell Duke, today a well-known researcher, tells a story that perfectly illustrates the problem. As a young elementary teacher, she read about cooperative learning and implemented it in her class with great eagerness and success. However, her principal was not amused. “Miss Duke,” he said, “what in blazes are you doing in there?” She enthusiastically explained cooperative learning, told him about the research on it, and explained how excited and productive the students were.

“I suppose that’s all right,” he said. “But can’t you get them to do it quietly?”

Having individual teachers be the unit of innovation or dissemination made it difficult to ensure that teachers had understanding and supportive administrators, coaching, or other ongoing support, and many of them either failed due to implementation problems that could have easily been remedied, or they succumbed to pressure from the administration or peers over time to conform with what the rest of the school was doing.

From my experience, widespread and impactful use of proven programs is not likely to succeed if it is imposed upon teachers or if it is sprinkled across the landscape as Johnny Appleseed did. How could the whole school become the unit of dissemination with the active participation of the teachers?

When we began Success for All in 1987, my colleagues and I hit upon a formula that we still use today with great success. Success for All (SFA) is a whole-school approach, intended to work in schools that serve mostly disadvantaged elementary and middle schools. Such schools can be assumed to have already tried and failed with many innovations, and they may be suspicious that this is just one more.

Our solution, starting with our very first SFA school in Baltimore, was to introduce the program to the administration and teaching staff and then let them vote by secret ballot as to whether or not they wished to participate. A positive vote of 80% was required for us to enter the school.

The voting does many things. First, it convinces the whole staff that they truly have a choice, and that for once they are not being pushed into something they did not select. Second, the process leading up to the vote already helps to get teachers thinking about how they can work with peers to improve the whole school, not just their personal classroom.

Votes are usually positive, but the voting process may reveal issues teachers want resolved before they take on an innovation. For example, teachers may say, “We’ll support this program, but we’ll need additional planning time. Can you promise that?” If the principal agrees, the vote is likely to be positive. When a school cannot arrive at a positive vote it is usually the case that there are serious problems in the school, such as a lack of trust between teachers and administrators. It is probably a good idea to delay starting a major innovation until such problems are worked out.

When starting a schoolwide innovation in a given district or region, it makes a lot of sense to start with schools that eagerly adopt the model. The vote helps identify such schools. Other schools that are less eager can then see how things go with the early adopters and come into the project later on. Within schools, the few reluctant teachers (less than 20%), seeing their peers voting in favor of SFA, are usually willing to give it a try, and may gain enthusiasm over time.

Beyond the voting itself, treating schools as the unit of implementation provides schools with new strategies for improving their whole teaching staff. It gives all teachers a common language, common tools, access to joint training and in-class coaching, and peer assistance. Principals may use distributed leadership, involving teachers and other staff in committees to plan school strategies to solve common problems (such as behavior/attendance, parent/community involvement, onboarding of new teachers, and teaching strategies for particular topics and grade levels). Instead of just one or two leaders in a school, every teacher becomes a leader in some area of expertise or interest.

A school that has chosen a proven program and is implementing it with understanding, enthusiasm, and the participation of all school staff is a fun, satisfying place to work. It can retain its teachers, instead of seeing them “promoted” to less challenging parts of town, because teachers prefer to work in supportive, successful environments in which their ideas and leadership are sought after and appreciated.

In addition to voting, teachers can play a key role in adapting the program to meet local circumstances, needs, and resources. They may introduce innovations after they have mastered the basics, and these innovations may catch on in the school, district, or the entire network of SFA schools nationally. Further, schools often pilot new strategies or materials and provide feedback to program developers.

I believe that whatever innovations developers are trying to disseminate, teachers should have the opportunity to choose (or not choose) them as a total school or distinct section of a school, such as the math department or the primary team. This mode of dissemination preserves teachers’ rights to participate in essential decisions about their own school without requiring that each school or teacher reinvent the wheel.

Or the six-cycle graph paper.

This blog is sponsored by the Laura and John Arnold Foundation