In the 1978 movie “Superman,” Lois Lane, star journalist, crash-lands in a helicopter on top of a 50-story skyscraper. The helicopter is hanging by a strut to the edge of the roof, and Lois is hanging on to a microphone cord. Finally, the cord breaks, and Lois falls 45 floors before (of course) she is swooped up by Superman, who flies her back to the roof and sets her down gently. Then he says to her:
“I hope this doesn’t put you off of flying. Statistically speaking, it is the safest form of travel.”
She faints.
I’ve often had the very same problem whenever I do public speaking. As soon as I mention statistics, some of the audience faints dead away. Or perhaps they are falling asleep. But either way, saying the word “statistics” is not usually a good way to make friends and influence people.
The fact is, most people don’t like statistics. Or more accurately, people don’t like statistics except when the statistical findings agree with their prejudices. At an IES meeting several years ago, a well-respected superintendent was invited to speak to what is perhaps the nerdiest, most statistically-minded group in all of education, except for an SREE conference. He actually said, without the slightest indication of humor or irony, that “GOOD research is that which confirms what I have always believed. BAD research is that which disagrees with what I have always believed.” I’d guess that the great majority of superintendents and other educational leaders would agree, even if few would say so out loud to an IES meeting.
If educational leaders only attend to statistics that confirm their prior beliefs, one might argue that, well, at least they do attend to SOME research. But research in an applied field like education is of value only if it leads to positive changes in practice. If influential educators only respect research that confirms their previous beliefs, then they never change their practices or policies because of research, and policies and practices stay the same forever, or change only due to politics, marketing, and fads. Which is exactly how most change does in fact happen in education. If you wonder why educational outcomes change so slowly, if at all, you need look no further than this.
Why is it that educators pay so little attention to research, whatever its outcomes, much in contrast to the situation in many other fields? Some people argue that, unlike medicine, where doctors are well trained in research, educators lack such training. Yet agriculture makes far more practical use of evidence than education does, and most farmers, while outstanding in their fields, are not known for their research savvy.
Farmers are, however, very savvy business owners, and they can clearly see that their financial success depends on using seeds, stock, methods, fertilizers, and insecticides proven to be effective, cost-effective, and sustainable. Similarly, research plays a crucial role in technology, engineering, materials science, and every applied field in which better methods, with proven outcomes, lead to increased profits.
So one major reason for limited use of research in education is that adopting proven methods in education rarely leads to enhanced profit. Even in parts of the educational enterprise where profit is involved, economic success still depends far more on politics, marketing, and fads, than on evidence. Outcomes of adopting proven programs or practices may not have an obvious impact on overall school outcomes because achievement is invariably tangled up with factors such as social class of children and schools’ abilities to attract skilled teachers and principals. Ask parents whether they would rather have their child to go to a school in which all students have educated, upper-middle class parents, or to a school that uses proven instructional strategies in every subject and grade level. The problem is that there are only so many educated, upper-middle class parents to go around, so schools and parents often focus on getting the best possible demographics in their school rather than on adopting proven teaching methods.
How can education begin to make the rapid, irreversible improvements characteristic of agriculture, technology, and medicine? The answer has to take into account the fundamental fact that education is a government monopoly. I’m not arguing whether or not this is a good thing, but it is certain to be true for many years, perhaps forever. The parts of education that are not part of government are private schools, and these are very few in number (charter schools are funded by government, of course).
Because government funds nearly all schools, it has both the responsibility and the financial capacity to do whatever is feasible to make schools as effective as it possibly can. This is true of all levels of government, federal, state, and local. Because it is in charge of all federal research funding, the federal government is the most logical organization to lead any efforts to increase use of proven programs and practices in education, but forward-looking state and local government could also play a major role if they chose to do so.
Government can and must take on the role that profit plays in other research-focused fields, such as agriculture, medicine, and engineering. As I’ve argued many times, government should use national funding to incentivize schools to adopt proven programs. For example, the federal government could provide funding to schools to enable them to pay the costs of adopting programs found to be effective in rigorous research. Under ESSA, it is already doing this, but right now the main focus is only on Title I school improvement grants. These go to schools that are among the lowest performers in their states. School improvement is a good place to start, but it affects a modest number of extremely disadvantaged schools. Such schools do need substantial funding and expertise to make the substantial gains they are asked to make, but they are so unlike the majority of Title I schools that they are not sufficient examples of what evidence-based reform could achieve. Making all Title I schools eligible for incentive funding to implement proven programs, or at least working toward this goal over time, would arouse the interest and enthusiasm of a much greater set of schools, virtually all of which need major changes in practices to reach national standards.
To make this policy work, the federal government would need to add considerably to the funding it provides for educational research and development, and it would need to rigorously evaluate programs that show the greatest promise to make large, pragmatically important differences in schools’ outcomes in key areas, such as reading, mathematics, science, and English for English learners. One way to do this cost-effectively would be to allow districts (or consortia of districts) to put forward pairs of matched schools for potential funding. Districts or consortia awarded grants might then be evaluated by federal contractors, who would randomly assign one school in each pair to receive the program, while the pair members not selected would serve as a control group. In this way, programs that had been found effective in initial research might have their evaluations replicated many times, at a very low evaluation cost. This pair evaluation design could greatly increase the number of schools using proven programs, and could add substantially to the set of programs known to be effective. This design could also give many more districts experience with top-quality experimental research, building support for the idea that research is of value to educators and students.
Getting back to Superman and Lois Lane, it is only natural to expect that Lois might be reluctant to get on another helicopter anytime soon, no matter what the evidence says. However, when we are making decisions on behalf of children, it’s not enough to just pay attention to our own personal experience. Listen to Superman. The evidence matters.
This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.
Robert Slavin's Blog 