CDC Told to Avoid Use of “Evidence-Based”: Is the Earth Flat Again?

In this blog series, I generally try to stay non-partisan, avoiding issues that, though important, do not relate to evidence-based reform in education. However, the current administration has just crossed that line.

In a December 16 article in the Washington Post, Lena Sun and Juliet Eilperin reported that the Trump Administration has prohibited employees of the Centers for Disease Control and Prevention (CDC) from using seven words or phrases in their reports. Two of these are “evidence-based” and “science-based.” Admittedly, this relates to health, not education, but who could imagine that education will not be next?

I’m not sure exactly why “evidence-based” and “science-based” are included among a set of banned words that otherwise consist of words such as “fetus,” “transgender,” and “diversity” that have more obvious political overtones. The banning of “evidence-based” and “science-based” is particularly upsetting because evidence, especially in medicine, has up to now been such a non-partisan, good-government concept. Ultimately, Republicans and Democrats and their family members and friends get sick or injured, or want to prevent disease, and perhaps as a result, evidence-based health care has been popular on both sides of the aisle. In education, Republican House Speaker Paul Ryan and Democratic Senator Patty Murray have worked together as forceful advocates for evidence-based reform, as have many others. George W. Bush and Barak Obama both took personal and proactive roles in advancing evidence in education.

You have to go back a long time to find governments banning evidence itself. Perhaps you have to go back to Pope Paul V, whose Cardinal Bellarmine ordered Galileo in 1615 to: “…abandon completely the opinion that the sun stands still at the center of the world and the Earth moves…”

In fear for his life, Galileo agreed, but in 1633, Galileo was accused of breaking his promise. He was threatened with torture, and had to agree again to the Pope’s demand. He was placed under house arrest for the rest of his life.

After his 1633 banishment, Galileo was said to have muttered, “E pur si muove” (And yet it moves). If he did (historians doubt it), he was expressing defiance, but also a key principle of science: “Proven principles remain true even if we are no longer allowed to speak of them.”

The CDC officials were offered a new formulation to use instead of “evidence-based” and “science-based.” It was: “CDC bases its recommendations on science in consideration with community standards and wishes.”

This is of course the antithesis of evidence or science. Does the Earth circle the sun in some states or counties, but it’s the other way around in others? Who decides which scientific principles apply in a given location? Does objective science have any role at all or are every community’s beliefs as valid as every other’s? Adopting the ban would hold back research and applications of settled research, harming millions of potential beneficiaries and making the U.S. a laughingstock among advanced nations. Banning the words “evidence-based” and “science-based” will not change scientific reality. Yet it will perhaps slow down funding for research and dissemination of proven treatments, and that would be disastrous, both in medicine and in education. I hope and expect that scientists in both fields will continue to find the truth and make it known whatever the consequences, and that our leaders of both parties see the folly of this action and reverse it immediately.

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.

 

Advertisements

Evidence and Freedom

In 1776, a small group of American patriots had a vision of a government of, by, and for the people, and they risked their lives to make it so. Their commitment to liberty was not just ideological, it was also pragmatic. They knew that people who were empowered to make their own decisions were more likely to be committed to the implementation of those decisions. The same should apply to education today.

One of the most important aspects of the Every Student Succeeds Act (ESSA) is how it balances evidence with freedom. The Act defines proven programs and mentions evidence 60 times. It encourages use of proven programs throughout. It provides for additional preference points for proposals in seven areas that meet evidence requirements. Yet only in the area of school improvement for the lowest 5% of schools does it require use of proven programs. This is probably a good thing.

Americans, even more than other people, don’t like to be told what to do. If the evidence movement turns into a set of mandates, telling educators which programs they can or cannot implement, it will probably be doomed. Even when evidence for or against given programs is solid and widely replicated, many political forces opposing evidence-based reform would surely come into play if educators felt compelled to use certain programs and avoid others.

Years ago, I had an experience that reinforced my view that teachers respond better to proven practices if they are free to choose them. I was doing a cooperative learning workshop in a large urban district. A surly-looking teacher raised her hand. “Do we have to do this?” she asked. “Of course not” I answered. “These are ideas for you to use or not, as you wish”

“In this district,” said the teacher “if we’re not required to use something, we’re not allowed to do it.”

How can we avoid compulsion? The answer is easy. Federal, state, and local policies need to provide incentives for schools to use certain programs with strong evidence of effectiveness from rigorous experiments, but not mandates to do so. That’s what ESSA will do in several areas. Incentives may mean providing a few points on competitive grant proposals, or modest financial incentives, for schools that adopt proven programs. These incentives should be enough to get educators’ attention, but not enough to force them to pick a given program.

Incentives should cause educators to eagerly volunteer to use proven programs, to raise their hands, not their hackles. They could lead educators to learn more about the proven programs available to them and about the research process itself. This in turn could encourage political leaders to support education R & D, as educators and the public at large begin to clamor for more programs and better research.

Government cannot and should not try to get 3 million teachers in 100,000 schools in 14,000 districts to use any particular set of programs, no matter what their evidence of effectiveness. What it can and should do is set in motion policies that gradually expand the availability, adoption, and spread of proven programs, eventually pushing less effective approaches to improve or disappear. Development and evaluation of promising programs continues in ESSA, in the new Education Innovation Research (EIR), which along with R & D funded by other agencies will continuously add to the set of proven programs ready for adoption. As the number and quality of proven programs grow, educators will become more and more comfortable about using them.

From our nation’s founding, freedom to make informed choices has been an essential foundation stone of our system of governance. So it should be in education policy.

Evidence can inform key decisions for children, and government can encourage and incent adoption of proven programs. However, educators need the freedom to do what is right for their children, guided but not steered by valid and useful research.

What the Election Might Mean for Evidence-Based Education

Like everyone else in America, I awoke on Wednesday to a new era. Not only was Donald Trump elected president, but the Republicans retained control of both houses of Congress. This election will surely have a powerful impact on issues that the president-elect and other Republicans campaigned on, but education was hardly discussed. The New York Times summarized Mr. Trump’s education positions in an October 31 article. Mr. Trump has spoken in favor of charters and other school choice plans, incentive pay for teachers, and not much else. A Trump administration will probably appoint a conservative Secretary of Education, and that person would have considerable influence on what happens next.

What might this mean for evidence-based reform in education? Hopefully, the new administration will embrace evidence, as embodied in the Every Student Succeeds Act (ESSA). Why? Because the Congress that passed ESSA less than a year ago is more or less the same Congress that was just elected. Significantly, Senators Rob Portman (R-Ohio), Michael Bennet (D-Colorado), and Patty Murray (D-Washington), some of the major champions of evidence in the Senate, were all just re-elected. Senator Lamar Alexander (R-Tennessee), a key architect of ESSA, is still in office. In the absence of a major push from the new executive branch, the Congress seems likely to continue its bipartisan support for the ESSA law.

Or so I fervently hope.

Evidence has not been a partisan issue and it will hopefully remain bipartisan. Everyone has an interest in seeing that education dollars are spent wisely to benefit children. The evidence movement has advanced far enough to offer real hope that step-by-step progress can take place in education as increasingly effective methods, materials, and software become available, as a direct outcome of research and development. Evidence-based reform has strengthened through red and blue administrations. It should continue to grow through the new administration.

Or so I fervently hope.

The Wonderful Reputation of Educational Research

Back in 1993, Carl Kaestle memorably wrote about the “awful reputation of educational research.” At the time, he was right. But that was 23 years ago. In the interim, educational research has made extraordinary advances. It is now admired by researchers in many other fields and by policy makers in many areas of government. As indicated by the importance of evidence in the Every Student Succeeds Act (ESSA), evidence is starting to make more of a difference in policy and practice. There is still a long, long way to go, but the trend is hugely positive.

In a recent article for the Brookings Institution, Ruth Curran Neild, acting director of the Institute of Education Sciences (IES), argued that educational research is on the right track. The one thing it lacks, she says, is adequate funding. I totally agree. Of course there are improvements that could be made to education policies and practices, but the part of the education field working on using science to improve outcomes for children is very much going in the right direction. Many are frustrated that it is not getting there fast enough, but we need more wind in our sails, not a change of course.

I was listening recently to an NPR broadcast about a new center for research on immunological treatments for cancer. The interviewer asked how their center could possibly make much difference with a grant of only $250 million. The director sheepishly agreed this was a problem, but hoped they could nevertheless make a contribution. If only we in education had conversations like this – ever!

What has radically changed over the past 15 years is that there is now far more support than there once was for randomized evaluations of replicable programs and practices, and as a result we are collectively building a strong set of studies that use the kinds of designs common in medicine and agriculture but not, until recently, in education. My colleagues and I constantly update reviews of research on educational interventions in the main areas of practice at the Best Evidence Encyclopedia website. Where once randomized studies were rare, they are becoming the norm. We recently published a review of research on early childhood programs, in which we located 32 studies of 22 different programs. Twenty-nine of the studies used randomized designs, thanks primarily to funding and leadership from a federal investment called Preschool Curriculum Evaluation Research (PCER). We are working on a review of research on secondary reading programs. Due to the federal Striving Readers program, which invested in evaluations of a wide variety of school interventions, our review is now dominated by randomized studies. Studies of programs for struggling elementary readers are now overwhelmingly randomized. The Investing in Innovation (i3) program requires randomized evaluations in its validation and scale-up grants and encourages them in its development grants, and this is increasing the prevalence of randomized studies across all studies of programs for students from grades pre-K to 12. The National Science Foundation has begun to fund scale-up projects that require random assignment, as have a few private foundations.

Random assignment is the hallmark of rigorous science. From a methodological standpoint, random assignment is crucial because only when students, teachers, or schools are randomly assigned to treatment or control conditions can readers be sure that any differences observed at posttest are truly the result of the treatments, and not of self-selection or other bias. But more than this, use of random assignment establishes a field as serious about its science. Studies that use random assignment are called “gold standard,” because there is no better design in existence. Yes, there are better and worse randomized studies, better and worse measures, and so on. Mixed methods studies can usefully add insight to the numbers. Replication is very important in establishing effectiveness. And there are certainly circumstances in which randomization is impossible or impractical, and a well-done quasi-experiment will do. But all this being said, the use of randomization moves the science of education forward and gives educational leaders reliable information on which to make decisions.

The most telling criticism of randomized experiments is that they are expensive. Yes, they can be. Encouragement and funding from IES and the Laura and John Arnold Foundation is increasing the use of inexpensive experiments in situations in which treatments and (usually) measures are already being paid for by government or other sources, so only funding for the evaluation is needed. But these experiments are only possible in special circumstances. In others, someone has to come up with serious funding to support randomized designs.

This brings us back to Ruth Neild’s main point. We know what needs to be done in educational research. We need to develop a wide variety of promising innovations, subject them to rigorous, ultimately randomized experiments, and then disseminate those programs found to be effective. We have systems in place to do all of these things. We just need a lot more funding to do them faster and better.

I don’t know if the increases in the quality of research in education are understood by policy makers, or how much this quality matters for funding. But education now has a case to make that it deserves much greater funding. Educational research is no longer just of interest to the academics who do it. It is producing answers that matter for children, and that should justify funding in line with our field’s new, wonderful reputation.

Money and Evidence

Many years ago, I spent a few days testifying in a funding equity case in Alabama. At the end of my testimony, the main lawyer for the plaintiffs drove me to the airport. “I think we’re going to win this case,” he said, “But will it help my clients?”

The lawyer’s question has haunted me ever since. In Alabama, then and now, there are enormous inequities in education funding in rich and poor districts due to differences in property tax receipts in different districts. There are corresponding differences in student outcomes. The same is true in most states. To a greater or lesser degree, most states and the federal government provide some funding to reduce inequalities, but in most places it is still the case that poor districts have to tax themselves at a higher rate to produce education funding that is significantly lower than that of their wealthier neighbors.

Funding inequities are worse than wrong, they are repugnant. When I travel in other countries and try to describe our system, it usually takes me a while to get people outside the U.S. to even understand what I am saying. “So schools in poor areas get less than those in wealthy ones? Surely that cannot be true.” In fact, it is true in the U.S., but in all of our peer countries, national or at least regional funding policies ensure basic equality in school funding, and in most cases I know about they then add additional funding on top of equalized funding for schools serving many children in poverty. For example, England has long had equal funding, and the Conservative government added “Pupil Premium” funding in which each disadvantaged child brings additional funds to his or her school. Pupil Premium is sort of like Title I in the U.S., if you can imagine Title I adding resources on top of equal funding, which it does in only a few U.S. states.

So let’s accept the idea that funding inequity is a BAD THING. Now consider this: Would eliminating funding inequities eliminate achievement gaps in U.S. schools? This gets back to the lawyer’s question. If we somehow won a national “case” that required equalizing school funding, would the “clients” benefit?

More money for disadvantaged schools would certainly be welcome, and it would certainly create the possibility of major advances. But in order to maximize the impact of significant additional funding, it all depends on what schools do with the added dollars. Of course you’d have to increase teachers’ salaries and reduce class sizes to draw highly qualified teachers into disadvantaged schools. But you’d also have to spend a significant portion of new funds to help schools implement proven programs with fidelity and verve.

Again, England offers an interesting model. Twenty years ago, achievement in England was very unequal, despite equal funding. Children of immigrants from Pakistan and Bangladesh, Africans, Afro-Caribbeans, and other minorities performed well below White British children. The Labour government implemented a massive effort to change this, starting with the London Challenge and continuing with a Manchester Challenge and a Black Country Challenge in the post-industrial Midlands. Each “challenge” provided substantial professional development to school staffs, as well as organizing achievement data to show school leaders that other schools with exactly the same demographic challenges were achieving far better results.

Today, children of Pakistani and Bangladeshi immigrants are scoring at the English mean. Children of African and Afro-Caribbean immigrants are just below the English mean. Policy makers in England are now turning their attention to White working-class boys. But the persistent and substantial gaps we see as so resistant to change in the U.S. are essentially gone in England.

Today, we are getting even smarter about how to turn dollars into enhanced achievement, due to investments by the Institute of Education Sciences (IES) and Investing in Innovation (i3) program in the U.S. and the Education Endowment Foundation (EEF) in England. In both countries, however, we lack the funding to put into place what we know how to do on a large enough scale to matter, but this need not always be the case.

Funding matters. No one can make chicken soup out of chicken feathers, as we say in Baltimore. But funding in itself will not solve our achievement gap. Funding needs to be spent on specific, high-impact investments to make a big difference.

Evidence-Based Resources Needed for Flint

I’m sure you are aware of the catastrophe in Flint, Michigan. 100,000 citizens of that city were exposed to water contaminated by lead and other toxic chemicals, and were then lied to about it by their state and local political leaders. Flint was the topic of the cover story in the February 1 Time magazine (“The Poisoning of an American City”). The story has been reported coast to coast, and in one of the presidential debates, Hillary Clinton noted the perfectly obvious, that this could never have happened in a middle-class suburb.

I won’t retell the whole story, but in essence, Flint, which is primarily under the political control of unelected state managers, made two disastrous decisions. First, in response to an increase in costs for clean Lake Huron water, Flint leaders decided to pump water from the nearby Flint River (“a sewer,” a resident noted). Second, the same leaders failed to add chemicals to keep the polluted water from dissolving lead in old pipes, also to save money. The result was drinking water with lead levels far higher than national standards. Early on, a courageous pediatrician, Dr. Mona Hanna-Attisha, noted elevated lead levels in young children and many other symptoms of poisoning, including anemia, rashes, hand tremors, and seizures. She persistently complained to anyone who would listen, but no action was taken until a September 2015 study found that children under five had twice the lead levels prevailing before the change in water sources.

Lead poisoning is devastating to the nervous systems of infants. It leads to diminished cognitive functioning, as measured by IQ tests and school performance, across a person’s entire life. Even low-level exposure has destructive effects. Dr. Hanna-Attisha noted, “If you were going to put something in a population to keep them down for generations to come, it would be lead.” It so happens that Baltimore has also long struggled with lead poisoning due to peeling paint in poor sections of the city. Freddie Gray, the young man who died in a police van, had lead poisoning as a child, performed poorly in school, and as a result was supporting himself with petty crime and drug-dealing. That’s how he got into the van in the first place.

In a January 30 article in the New York Times, the heroine of the Flint story, Dr. Hanna-Attisha, was asked what should be done for the roughly 8,000 children under 6 who were exposed to lead-contaminated water for up to two years. She said, “We have to throw every single evidence-based resource at these kids, starting now.” Since the effects of lead poisoning itself cannot be reversed, what she was referring to was proven early childhood programs, visiting nurse programs, and nutrition improvements. She might have added tutoring for children who need it, eyeglasses, and proven comprehensive school reforms, among many other interventions.

The kids in Flint certainly deserve special attention and immediate effective intervention. But in what disadvantaged area in the U.S., or anywhere, would we not say the same? Flint is remarkable because it is acute, and because the political betrayals expose the system that disregards the needs of disadvantaged people. But the chronic problems of poverty in America need just the same solutions.

There are approximately 52,000 Title I schools in the U.S. I would suggest, for starters, that every principal of every one of these schools, and every mayor, governor, legislator, and national, state, and local education leader, frame and display Dr. Hanna-Attisha’s statement in their office:

“We have to throw every single evidence-based resource at these kids, starting now.”

And then they should do it. Starting now.

Evidence at Risk in House Spending Bill

The House Appropriations Committee marked up its spending bill yesterday for fiscal year 2016 for the Departments of Labor, Health and Human Services and Education. The spending levels in the bill put forward by the majority reduce Department of Education funding by $2.8 billion, mostly by eliminating approximately two dozen programs and severely cutting back several others, so it is no surprise that the bill passed through the Committee along party lines.

I can’t speak for all of the affected programs, but I do want to address what some of these proposed cuts could do. In a word, they would devastate the movement toward evidence as a basis for policy and practice in education.

First, the House bill would eliminate Investing in Innovation (i3). i3 has been the flagship for “tiered evidence” initiatives, providing large scale-up grants for programs that already have substantial evidence of effectiveness, smaller “validation” grants for programs with some evidence to build up their evidence base, and much smaller “development” grants for programs worth developing, piloting, and evaluating. At $120 million per year, i3 costs about 50¢ per taxpayer. What we get for 50¢ per year is a wide variety of promising programs at all grade levels and in all subjects, serving thousands of mostly high-poverty schools nationwide. We get evidence on the effectiveness of these programs, which tells us which are ready for broader use in our schools. The evidence from i3 informs the whole $630-billion public education enterprise, especially the $15-billion Title I program. That is, i3 costs 2¢ for every $100 spent on public education. Congresswoman Chellie Pingree of Maine offered an amendment to restore i3 and increase its funding level to $300 million, which is what the president had proposed. The process of offering the amendment gave members the opportunity to discuss the importance of i3, but in the end it was withdrawn (a not-uncommon procedural move when the amendment does not have an offset and/or is not expected to pass).

Second, the House proposal would significantly reduce funding for the Institute of Education Sciences (IES). IES commissions a wide variety of educational research, data collection, communications about evidence, and standard-setting for evidence, at a very modest cost. In this case, Congressman Mike Honda of California offered and withdrew an amendment to restore IES funding to its FY15 level of $574 million.

Finally, the main target of the proposed cuts was discretionary programs, which provide direct services to students. Districts, states, and other entities have to apply for these pots of money (as distinct from funds such as Title I or IDEA that are distributed by formula). Examples include Striving Readers (for struggling secondary readers); School Improvement Grants, or SIG (for low-performing schools); Preschool Development Grants; Mathematics and Science Partnerships; Ready to Learn (educational television); and several others.

These discretionary funding sources are the programs that could most easily be focused on evidence. One practical example is SIG, which recently added a category of approved expenditures consisting of whole-school reform programs with at least moderate evidence of effectiveness, which includes having been tested against a control group in at least one rigorous experiment. As another example, Title II SEED grants for professional development now require that programs adopted under SEED funding have at least moderate evidence of effectiveness. Congresswoman Rosa DeLauro offered an amendment to reinstate many of these programs, and it failed along party lines.

Adding evidence as a requirement or encouraging use of proven programs is much easier with discretionary programs than with formula grants. Yet if the House bill were to become law, there would be very few discretionary programs left.

The House proposal would greatly reduce national capacity to find out what works and what does not, and to scale up proven programs and practices. I very much hope our leaders in Congress will rethink this strategy and retain funding for the government programs mostly likely to help all of us learn — policy makers, educators, and kids alike.