Fund What Works!

2015-02-05-HP6202_05_2015.jpg

President Obama’s recently released budget may or may not make its way into law, but it has already made its way into the hearts of those who believe that government programs must be held to account for producing the outcomes they are intended to produce. Red or blue, everyone should want government dollars to make a difference.

For PK-12 education, evidence appears in a starring role in the president’s proposal. To me, the most exciting innovation in the budget is something called Leveraging What Works. This is a proposal for a competitive grant program in which the U.S. Department of Education would reward school districts that use their existing federal formula funds for activities that have a strong or moderate evidence base as defined in the recent EDGAR changes, a proposal I have advocated on this blog. The total amount requested, $100 million per year, is tiny compared with current expenditures, but it would be a start on the idea that federal dollars should increasingly go to proven programs and practices. Hopefully, this idea would go along with policies encouraging adding preference points to competitive proposals that commit to using proven approaches.

The proposal would add $50 million for School Improvement Grants (SIG) for states to use to encourage school districts to implement proven strategies. If the standards for “proven” are rigorous, this would be another way to use modest funding to spur effective use of much larger existing funding.

The Investing in Innovation (i3) program is showing its value in moving along the pipeline of educational innovations from development to validation to scale-up. The budget would more than double annual funding for this crucial program. It would also double funding for early education research and evaluation and would invest $20 million a year to test strategies to improve transitions from preschool to third grade. It adds $70 million to the Social Innovation Fund (SIF) to test promising solutions to a broad array of problems relating to children and youth.

There is much more in the remarkable budget. I hope our Congress can put aside partisanship and agree on a compelling new direction for government: Fund What Works!

Advertisements

Seeds, Bricks, and Sand: Stages of School-Reform Readiness

2015-01-15-HP57Seeds_Sands.jpg

Every school, no matter how effective at improving student outcomes, could probably be even more effective, and some schools have a particularly long way to go. Various proven reform models for whole schools, particular subjects, or specific purposes stand ready to help all of these schools improve. Yet schools vary a great deal in terms of readiness for particular approaches to reform.

A metaphor for three types of schools in terms of readiness for reform is seeds, bricks, sand. The “seeds” metaphor implies an environment so conducive to reform that anything can grow there. The staff and leadership of the school are capable, aware of research, participating in professional development, well-coordinated, cohesive, and unafraid of change. Such a school may be able to create and evaluate its own reform methods and sustain and improve them over time, perhaps with general advice from consultants. “Bricks” schools are also positively oriented toward change, but are unlikely to invent effective reforms themselves. Such schools have committed and hard-working teachers and leaders who have not had the time or resources to become reform experts themselves, but are welcoming to proven models. The “bricks” metaphor implies that if someone brings the bricks and a set of plans to the site, a durable edifice can be built and maintained.

A “sand” school, on the other hand, is one that is not ready for reform, and building on this site is like building a sand castle, which will wash away with the next tide. In such schools the staff and leadership may be at odds with each other, may not believe that children can learn any more than they do now, or may have experienced failure with previous reforms. These schools may need serious restructuring.

The usefulness of the “seeds-bricks-sand” categories is in understanding how to help schools adopt and sustain proven programs. The great majority of Title I schools, in my experience, are “bricks” schools, ready, willing, and able to implement well-defined, research-proven programs, but unlikely to have the inclination to invent their own school-wide approach. Others are clearly in the “sand” category. Yet Title I schools in trouble are frequently given “seeds” advice. Until now, schools receiving substantial funding under the current School Improvement Grants (SIG) have been routinely given consultants to help them work out their own school-wide reform designs, rather than being helped to adopt proven programs. There are schools that can benefit from such strategies, but they are rarely the ones that are persistently low achieving, as all SIG schools are. Recent proposed regulations would offer SIG schools the option of adopting proven, whole-school reform models, a welcome and long overdue change.

Use of proven, well-structured reforms needs to be expanded in all schools. For those in the greatest difficulty, this need is urgent. The new SIG regulations would allow schools capable of implementing (but not inventing) proven, effective reforms a chance to turn themselves around.

Whole-school reform is difficult and expensive, and when it fails, the consequences for children as well as educators can be dire and long-lasting. Failed initiatives not only waste money, but they undermine the belief that high-poverty schools can be saved. We need to get smarter about targeting interventions to specific types of schools to increase the likelihood that investments in reform truly pay off for kids and for our society.

Are Proven Educational Innovations Ready for Prime Time?

These are good times for evidence-based reform in education. Due in particular to Investing in Innovation (i3) and the Institute of Education Sciences (IES), the number of proven programs in all subjects and grade levels in increasing, and as i3 programs come to the ends of their evaluations, the number of proven programs should accelerate (even though there are also sure to be many disappointments).

The increasing numbers of programs proven effective in rigorous research creates new opportunities for policy and practice in education. Already, School Improvement Grants (SIG) are newly introducing an option for schools to choose proven, comprehensive reform models. Other areas of policy may also soon begin to encourage or incentivize use of programs with strong evidence.

If these changes in policy begin to happen, it will begin to matter whether educational programs proven to be effective are in fact ready for prime time, meaning that they are ready to be disseminated and supported in the form they existed when they were successfully evaluated. It would be catastrophic if educators and policy makers began looking on the What Works Clearinghouse, for example, or looking for programs that meet the EDGAR standards for strong or moderate evidence of effectiveness, and found many program that were unavailable, or unrealistic, or impractical.

Much as providing evidence of effectiveness is an advance in education, there is a real need for a determination of the degree to which programs are also ready for widespread implementation.

Some indicators of readiness for prime time would be easy to assess. For example, programs that lack a web site, do not offer materials or training, or otherwise do not exist in anything like the form in which they were evaluated cannot be considered ready for implementation. Some programs used procedures in their evaluation that could never be replicated, such as science programs that provide each experimental class with enough graduate students to monitor and assist every lab group. Some proven technology products run on hardware that no longer exists.

Many studies use measures of learning outcomes that are closely aligned with what was taught in the experimental but not the control group. Such studies might be excluded on the basis that the overaligned measure does not have meaning beyond the experiment itself.

Educators who choose to use proven programs have a right to be confident that the programs they have selected are, if implemented well, likely to result in enhanced performance on measures they care about. Finding a lot of programs that cannot be implemented under ordinary circumstances and with meaningful measures will diminish interest in evidence-based reform.

Evidence-based reform itself is ready for prime time in education, but its future depends on whether it is perceived to produce genuine benefits for children. We need to make sure that the proven programs we offer to educators meet their needs and those of the students, not just scientific standards.

Thank You, Jim Shelton

2014-10-16-HP5010_16_2014.jpg

Everyone who works to advance evidence-based reform in education was saddened to learn that Jim Shelton will be leaving the U.S. Department of Education. Jim is currently the Department’s Deputy Secretary, and before that he was the Assistant Deputy Secretary for the Office of Innovation and Improvement (OII), the home of the Investing in Innovation (i3) program. Under his leadership, i3 created a unique approach to speeding up innovation in education, supporting development of new programs, evaluation of promising programs, and scale-up of already proven programs. Jim has provided the intellectual leadership for i3, understanding better than anyone the process by which proven and replicable programs could gradually infuse government-funded programs in education, starting with competitive programs (such as School Improvement Grants) and then moving on to formula programs (such as Title I).

Jim has been a tireless advocate for evidence, but even more so for kids. He understands that the children who need the best schools need the best programs. Better teachers? Smaller classes? Better parent support? Sure, they need these too, but until core classroom practices are better in every class, millions of children will continue to fail each year.

Jim is a powerful speaker and a powerful advocate for his ideas. I don’t know where he’s going next, but I’m certain that wherever it is, he will carry on the fight for children and for evidence so that the programs we provide children actually work.

Evidence to the Rescue! Whole School Option for SIG

2014-09-11-HP48.jpg

The U. S. Department of Education has just released draft regulations that will substantially change School Improvement Grants (SIG). Within the document are two pages that could change the world. Really.

Up to now, the very low achieving schools that qualified for SIG funding had to choose among four options. Two of these, school closure and restart, are rarely used. Turnaround requires firing the principal and at least 50% of the staff. So most SIG schools have opted for transformation, which still requires firing the principal in most cases and implementing a lengthy list of changes.

The new regs keep these four options, with a few tweaks, and they add three more. One is a “state-determined” model to be devised by state education departments, which requires approval by the U.S. Secretary of Education, and another emphasizes pre-kindergarten programs as part of elementary school reform. These are good ideas, but not revolutionary.

The revolutionary option is an “evidence-based whole-school reform option.” Schools that choose it have to implement whole-school reform models created and disseminated by external organizations. That’s good, but not quite revolutionary in itself.

Here comes the revolutionary part. Be sure you’re sitting down before reading on.

The whole-school reform models SIG schools may choose must have a high level of evidence of effectiveness. At least two studies that meet tough What Works Clearinghouse (WWC) standards must have documented positive effects on major learning outcomes. The Department will review programs in advance and will determine if they meet the standards. Full disclosure: Our Success for All program will most likely qualify. But this is a lot bigger than a good thing for Success for All.

This is the first time in all of education history, going back to Socrates, when evidence of effectiveness has been a requirement for anything in practice. Yes, the equally revolutionary Investing in Innovation (i3) program requires evidence of effectiveness for its large validation and scale-up grants, and the scale-up grants, in particular, have collectively benefitted thousands of schools. However, the focus of i3 is on building capacity among developers, not on direct service. In contrast, the SIG whole-school option moves evidence-based reform into a new arena. For the first time, a major federal program that does provide direct service to schools is offering them an option that requires substantial evidence of effectiveness.

This gives schools that apply for and receive SIG funding a new set of possibilities to take on a proven whole-school strategy without having to fire their principals or others. Assuming whole-school models were proven in schools like theirs, and assuming the organizations that made and evaluated them have a lot of experience working with similar schools, their students are likely to benefit, and that’s all good.

If all goes well, the whole-school option will demonstrate the power of evidence-based reform. Proven models will show well in evaluations of SIG, because their capacity to show positive outcomes is what got them on the list. If this happens, one might expect that other parts of federal, state, and local education would also begin demanding evidence of effectiveness for innovative programs. Seeing this, perhaps government and private funders will ramp up their investments in research and development, to create and validate additional models for a wide variety of purposes. Imagine, dare I say it, that Title I might begin to encourage schools to use proven approaches. Or Title II. Or IDEA. Or technology applications. Or STEM. Or. . . just about any area of education in which outcomes can be measured.

If we start with the SIG whole-school option, the evidence-based reform movement could get completely out of hand. Wouldn’t that be great?

Six Low-Cost or Free Ways to Make American Education the Best in the World

2014-06-12-HP38Image_06_12_2014.jpg

It does not take a political genius to know that for the foreseeable future, American education is not going to be rescued by a grand influx of new money. Certainly in the near term, the slow economic recovery, gridlock in Washington, and other factors mean that the path to substantial improvement in outcomes is going to be paved not with new gold, but with better use of the gold that’s already there.

No problem.

We already spend a lot of money on education. The task right now is to change how we spend federal, state, and local resources so that more money is spent on programs and practices known to make a difference rather than on investments with zero or unknown impacts on learning. Here are my top six suggestions for how to spend our education resources more effectively. (I’ll go into more details on these in future blogs).

1. Provide incentives for schools and districts to implement programs with strong evidence of effectiveness in competitive grants. In competitive grants in all parts of federal and state government, offer “competitive preference points” for applicants who promise to adopt and effectively implement programs proven to be effective. For example, schools proposing to implement programs identified as having “strong evidence of effectiveness” under the new EDGAR definitions might receive four extra points on a 100-point scale, while those meeting the criteria for “moderate evidence of effectiveness” might receive two points. Readers of this blog have seen me make this recommendation many times. Perfect example: School Improvement Grants for low-achieving schools. Cost: zero.

2. Provide incentives for schools and districts to implement programs with strong evidence of effectiveness in formula grants. The big money in federal and state education funding is in formula grants that go to districts and schools based on, for example, levels of poverty, rather than competitive applications. The classic example is Title I. Schools have great freedom in how they use these funds, so how can they be encouraged to use them in more effective ways? The answer is to provide additional incentive funding if schools or districts commit to using proven programs with their allotted formula funds. For example, if schools agree to use a portion of their (formula-driven) Title I funds on a proven program, they may qualify for additional funds (not from the formula pot). This was the idea behind the Obey-Porter Comprehensive School Reform initiative of the late 1990s, which encouraged thousands of Title I schools to adopt whole-school reform models. Cost: This strategy could be done at a cost of perhaps 1% of the current $15 billion annual Title I budget.

3. Offer commitment to proven programs as an alternative to use of value-added teacher evaluation models. A central part of the current administration’s policies is incentivizing states and districts to adopt teacher evaluation plans that combine principal ratings of teachers with value-added scores based on students’ state reading and math tests. This is a required part of Race to the Top in those states that received this funding, and it is a required element of state applications for a waiver of elements of No Child Left Behind.

In practice, current teacher evaluation policies are intended to do two things. First, they insist that schools identify extremely ineffective teachers and help them find other futures. If done fairly and consistently, few oppose this aspect of teacher evaluation. Principals have evaluated teachers and identified those with serious deficits forever, and I am not arguing against continuing this type of evaluation.

The second purpose of the teacher evaluation policies is to improve teaching and learning for all teachers. This is the expensive and contentious part of the policies; in most states it requires a combination of frequent, structured observation by principals and “value-added” assessments of a given teacher’s students. The technical difficulties of both are substantial, and no study has yet shown any benefit to student learning as a result of going through the whole ordeal.

If the goal is better teaching and learning, why not require that all reform approaches meet the same evidence standards? If a school proposes to use a schoolwide strategy that (unlike current teacher evaluation policies) has strong evidence of effectiveness, the school should be permitted, even encouraged, to suspend aspects of the new model as long as it is implementing proven alternatives with fidelity and good outcomes. Cost: Modest, assuming proven programs are similar in cost to the expensive new teacher evaluation strategies.

4. Train and equip paraprofessionals as tutors. The most common expenditure of Title I funds is on paraprofessionals or aides, educators who do not usually have teaching degrees but perform all sorts of functions within schools other than class teaching. Paraprofessionals can be wonderful and capable people, but evidence in the U.S. and U.K. consistently finds that as they are most commonly used, they make little difference in student learning.

Yet there is also extensive evidence that paraprofessionals can be very effective if 
they are trained to provide well-structured one-to-one or one-to-small group tutoring to students who are struggling in reading and math. Paraprofessionals are a multi-billion dollar army eager and capable of making more of a difference. Let’s empower them to do so. Cost: Minimal (just training and materials).

5. Encourage schools to use Supplemental Educational Services (SES) funding on proven programs. As part of No Child Left Behind, Title I schools had to use substantial portions of their Title I dollars to provide Supplemental Educational Services (SES) to students in schools failing to meet standards. Study after study has found SES to be ineffective, and expenditures on SES are waning, yet they remain as a significant element of Title I funding, even in states with waivers. If districts could be encouraged to use SES funds on programs with evidence of effectiveness in improving achievement (such as training paraprofessionals and teachers to be tutors in reading and/or math), outcomes are sure to improve. Cost: Minimal.

6. Invest in research and development to identify effective uses of universal access to tablets or computers. Despite economic and political hard times, schools everywhere are moving rapidly toward providing universal, all-student access to tablets or computers. There is a lot of talk about blended learning, flipped learning, and so on, but little actual research and development is going on that is likely to identify effective and replicable classroom strategies likely to make good use of these powerful tools. As it has done many times before, American education is about to spend billions on technology without first knowing which applications actually work. Setting aside a tiny percentage of the costs of the hardware and software, we could fund many innovators to create and rigorously evaluate approaches using all-student technology access, before we get stuck on ineffective solutions (again). Cost: modest.

* * *

I’m sure there are many more ways we could shift existing funds to advance
American education, but they all come down to one common recommendation: use what works. Collectively, the six strategies I’ve outlined, and others like them, could catapult American education to the top on international comparisons, greatly reduce education gaps, and prepare our students for the demands of a technological economy, all at little or no net cost, if we’re willing to also stop making ineffective investments. Moreover, all of these six prescriptions could be substantially underway in the next two years, during the remainder of the current administration. All could be done by the Department of Education alone, without congressional action. And again, I’m sure that others have many other examples of low-cost and no-cost solutions that I haven’t thought of or haven’t addressed here.

A revolution in American education does not necessarily require money, but it does require courage, leadership, and resolve. Those are resources our nation has in abundance. Let’s put them to work.

School Improvement Grants Embrace Evidence of Effectiveness

2014-02-20-HPImage022014.jpg

Despite all of the exciting gains made by evidence-based reform in recent years, all of the progress so far has been limited to development, evaluation, and scale-up of proven programs outside of mainline education policy or funding. Title I, Title II, Race to the Top, School Improvement Grants, and other large, influential funding sources for reform have hardly been touched by the growth of proven, replicable programs sponsored primarily by the Institute of Education Sciences and Investing in Innovation (i3). Until the evidence movement crosses over from R&D to the real world of policy and practice, it will remain the domain of academics and policy wonks, not a real force for change.

In the recently passed Omnibus budget, however, appears a first modest step over the R&D/policy divide. This is a new provision in congressional authorization of School Improvement Grants (SIG). Up until now, SIG schools (ones that have suffered from very low achievement levels for many years) had to choose among four models, all of which require major changes in staffing. Each SIG school is expected to develop its own model of reform, usually with the help of consultants. The problem has been that each of the hundreds of schools receiving (substantial) SIG funding has to create its own never-before-tested path to reform, and then try to implement it with quality in a school that has just experienced a substantial turnover of its leadership and staff.

The “Fifth Option” recently introduced by Congress adds a new alternative. SIG schools can choose to adopt a “proven whole-school reform model” that meets at least a moderate level of evidence support, which includes having been tested against a control group in at least one rigorous experiment. The fifth option will let schools keep their leaders and staffs, but adopt a schoolwide approach that has been used in many similar schools and found to be effective.

The Omnibus bill was passed too late in the year to apply this fifth option to the 2014-2015 school year, and the U. S. Department of Education, as well as individual states, have a lot of work to do to prepare new regulations and supports for schools applying for SIG funds under this new option in 2015-2016.

However, the fifth option makes an important statement that has not been made previously. In a major school improvement (not R&D) funding program, the fifth option says “use what works.” Wisely, it does not mandate the use of any specific programs, but by highlighting evidence-proven approaches, it puts the government behind the idea that federal funding should whenever possible be used to help educators use programs with strong evidence of effectiveness. This could be the start of something beautiful.