EDGAR and the Two-Point Conversion


Once upon a time, there was a football player named EDGAR. His team was in the state championship. It was the fourth quarter, and they were down by seven points. But just as time ran out, EDGAR ran around the opposing line and scored a touchdown.

EDGAR’s coach now had a dilemma. Should he try a safe kick for one point, putting the game into overtime, or go for a much more difficult two-point conversion, one chance to score from the five-yard line?

Evidence-based reform faces a similar dilemma. The U.S. Department of Education proposed several months ago some additions to EDGAR, not a football player but a stultifyingly boring doorstopper of a book of regulations for grants and contracts. These new regulations, as I noted in an earlier blog, are really exciting, at least to evidence wonks. They define four levels of evidence for educational programs: Strong evidence of effectiveness, moderate evidence of effectiveness, evidence of promise, and strong-theory. These definitions are similar to those used in Investing in Innovation (i3) to qualify proposals for scale-up (strong), validation (moderate), or development grants (evidence of promise or strong theory).

Here’s where the two-point conversion comes in. Readers of this blog may recall that I have long advocated the provision of, say, two competitive preference points in discretionary grants for proposals promising to use proven programs when such programs are available. The idea is that two points on a scale of 100 would greatly increase the interest of grant writers in proven programs without heavy handedly requiring their use. No grant writer ignores two points, but school leaders may have good reasons to prefer programs that have not yet been successfully evaluated. In those cases, the schools would be free to forego the competitive preference points. Still, the two points would telegraph the government’s support for the use of proven programs without undermining local control. Requests for proposals routinely include competitive preference points for criteria a lot less important than whether the program schools are proposing to use have been proven to work. Why not provide at least this much encouragement for potential innovators, at no additional cost to the government?

Offering two points for proposing to use proven programs would bring about a major conversion of education reform. It would focus attention on the evidence and make school and district leaders aware that effective options are available to them and are ok, even encouraged, by the powers that be.

As in my fictitious football game, it’s EDGAR who sets up the two-point conversion. The new definitions in EDGAR make it relatively easy for the Department of Education to put competitive preference points or other encouragements for use of proven programs into requests for proposals. EDGARS’s end run does not win the game, but it creates a condition in which the game can be won – by a two point conversion.

Lessons from Innovators: eMINTS


The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the Forum for Youth Investment, highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

This blog, based on an interview between the Forum for Youth Investment and eMINTS leadership, focuses on what the eMINTS National Center has learned about scaling a technology intervention in the sometimes low-tech environment of schools. eMINTS provides professional development for K-12 educators using interactive group sessions and in-classroom coaching/mentoring to help teachers integrate technology into their teaching. Their i3 validation grant focuses on expanding an intensive professional development model for teachers in rural middle schools.

Plan for success. Thinking about scale and sustainability starts long before a funding stream is coming to an end. As Lorie Kaplan, Executive Director of the eMINTS National Center notes, “Even in the very early stages of developing and testing your innovation, it is important to ask yourself – if we are successful, is this innovation scalable?” Even prior to receiving the i3 award, eMINTS leadership began considering the options for expanding their work to all 50 states. Five years ago, they went through Joel Barker’s Implications Wheel process – a software-enhanced group process for discovering and mapping the implications of change. They have goals to expand both nationally and internationally. The process required them to look at emerging trends, their strategic objectives, innovations they could make, and changes to policy and practice required for change.

Expand with fidelity and flexibility. Even as they envisioned a program that reached across the country and internationally (they currently have a partnership with a school in New South Wales Australia), eMINTS staff were aware of the broad implications of scaling. They knew that trying to grow too fast without a support structure would be devastating to the fidelity of the program. “It is critical to identify which parts of the program must be held constant from one implementation to the next in order to maintain quality, while also maintaining flexibility to adapt successfully to each new context,” said Christie Terry, eMINTS Associate Director. Building in local support strategies has allowed the program to scale effectively. These supports vary depending on the locale. In Utah, where they have had more requests for training than they had the capacity to offer, for example, they began to approve local experts to train trainers. Hoping to build on that success, they are exploring establishing regional hubs and have tried to secure additional funding to support that process. Kaplan notes that although they didn’t receive a second i3 grant to develop regional hubs in school districts in Utah, Delaware and Alabama, the three districts are committed to the concept and hope to push on to develop regional hubs even without the funding.

Make the case for value early and often. Many initiatives with sound evidence and great materials bump up against shrinking school budgets. In some cases, the funding stream that seeds a project is no longer available two or three years down the road. In other cases, schools silo interventions and miss opportunities to fund programs through alternative sources. eMINTS faced both problems as they began to scale their intervention. They found that schools didn’t have a dedicated funding stream to support the model, especially after a key federal technology funding stream was defunded by the Department of Education. To cope, they had to work with schools to understand the multi-faceted value of the program and find resources that connect to the goals of eMINTS. For example, they have had to demonstrate how technology impacts instruction in a variety of ways – from increasing student motivation and achievement, to developing 21st century skills, to supporting common core implementation. Similarly, they have had to work with schools to understand the full intervention. “We had to convince schools that eMINTS isn’t just about hardware and equipment,” says Kaplan. “The professional development components of the model are as critical as the technology going into the classrooms. Unless you have the essential supports for how to use the technology well, you aren’t going to get anything out of it.” Once they were able to make the leap from equipment to the broader initiative, schools began to see beyond technology-only funding streams and are more supportive of the program overall.

Be high tech and high touch. Educators need personal attention to succeed in trying and implementing new program – even when they are high tech. One major component of the eMINTS professional development program is the in-classroom coaching that teachers receive on a regular basis. This one-on-one time with their trainer allows teachers to have support as they implement the new technologies and teaching strategies into their own context. The eMINTS staff use a wide variety of free, online collaboration tools, such as Skype, Moodle, Google Apps, and Edmodo, to do more personalized, just-in-time coaching and mentoring with teachers. According to Terry, “It’s important to be available to teachers as they encounter challenges to help them move past barriers and sustain their enthusiasm. Using these types of tools gives teachers concrete ways to build and sustain their personal learning networks.”

Use Evidence to Improve Title I, Not to Threaten It


Among many educators and policy makers, the idea of evidence-based reform is scary. They fear that rigorous evaluations will fail to support their favorite government programs. What if studies find few benefits of Title I or other longstanding government programs focused on disadvantaged children?

If the evidence movement comes to be seen as the Grim Reaper, intent on stamping out funding programs, it is doomed. Government spending on disadvantaged children needs to be made more impactful, not abandoned. Title I, for example, is not a specific school or classroom intervention, it is a funding source that supports more and less effective initiatives. The goal of research and development should not be to determine whether Title I “works,” but to inform Title I directors, principals, and teachers about which investments of Title I dollars pay off in enhanced student achievement and which do not. R & D needs to create and evaluate new reading and math approaches, interventions for English learners and struggling readers, technology applications, whole-school reforms, and other methodologies. Those that work can then be encouraged as alternatives for Title I schools everywhere to use. As these strategies become more and more widely used, Title I itself will become more and more effective.

As effective uses of Title I funds become more common, less effective uses will fall away. For example, research has long found that traditional uses of paraprofessionals add little to student learning. However, there are several proven programs that use paraprofessionals to tutor struggling readers one-to-one or in small groups. If Title I schools are encouraged to use paras in new ways, less effective uses will be likely to diminish.

With time and continued investment, the evidence base for effective practices will grow, and Title I can further encourage the use of proven approaches. It can use incentives as well as information, but should not mandate the use of particular programs. Title I or other federal programs can invest in R & D, disseminate information on effective practices, help providers of proven programs to go to scale, provide funding to help schools adopt proven approaches, and so on, helping Title I transition from a focus on remediation to a focus on innovation and forward-looking practice. All educators support improving the outcomes of Title I, and I believe they would come to support a process capable of bringing proven innovations to disadvantaged schools.

If evidence-based reform is seen as a way to improve, not threaten, Title I and other federal funding programs, educators and policy makers can come to see it as an ally, not a threat. In a sense, it does not matter how effective Title I is today, as long as it is clearly getting better every year. Creating, evaluating, and disseminating effective approaches for use within Title I is the best way to ensure that Title I gets better outcomes. It maintains the key funding source for reform, and it builds support for evidence among those who care about disadvantaged kids.