Put International Lessons to the Test in U.S. Schools

In a November 10 Sputnik I wrote some cautionary thoughts about what we can and cannot learn from international comparisons to improve educational policies. My old friend Marc Tucker, in his December 20 blog called Top Performers, took me to task, saying that by suggesting we try out ideas from abroad in our own schools before adopting them wholesale, I was “looking for my keys where the light was better” rather than where they might actually be.

In my blog I was completely agreeing with Marc that we can learn a lot from other countries. I work part-time in England and am very familiar with education there and elsewhere in Europe. There is indeed much we can learn in other countries. In fact, we already are: the hot off the press Quality Counts report from Education Week found that “Education officials in 29 states reported that their agency uses international education comparisons to inform their reform strategies or identify ‘best practices.'” Where I take issue with Marc is in his apparent belief that if we study what successful nations do, we can just plunk their policies down in our context and all will be well. Marc seems to think that international comparisons have proven that our main efforts need to be directed toward improving teacher quality. He might very well be right. I’d love to see teacher salaries doubled, teacher education dramatically improved, induction enhanced, and so on, and perhaps these policies would solve our problems by making teaching a more attractive profession, bringing higher-quality students into teaching, and providing excellent professional development and support to help existing and new teachers to be effective and to want to stay in the profession. Frankly, however, there isn’t a U.S. educator or policy maker who didn’t already know that these would be great ideas long before we ever heard of Finland.

But how do we cause all of these things to happen in our society, with our kids? Which of these policies are not only effective, but most cost-effective? Is it too much to ask that whatever ideas we glean from observing Finland or Singapore or Japan be tested in Minnesota or Massachusetts or Mississippi, so we can learn how they work here? And in the meantime, might we also increase use of programs and practices that have been proven to work in the U.S., and develop and evaluate more of them?

America’s strength in every field, from medicine to agriculture to satellites, lies in its extraordinary capacity in research and development. This is true in education as much as in other areas; the products of U.S. educational R & D are much sought after in other countries. While other countries can give us good ideas and benchmarks to evaluate our students’ performance, let’s also build on our strengths.

Advertisements

Breaking Down Red-Tape Barriers to College

Updated

As college application season is coming to a close, parents and kids are embarking on a more daunting task: figuring out how to pay for college. Unfortunately, difficulties in navigating the financial aid process can result in many students forgoing college altogether. Could there be a better way to help kids get beyond this single but life altering barrier?

Stanford researcher Eric Bettinger did a study recently in which H&R Block took data from peoples’ tax forms to fill out a FAFSA (Free Application for Federal Student Aid) for their high school seniors. The cost of doing this was trivial, yet the benefits were huge. Children of parents randomly assigned to have their FAFSA done by H&R Block were significantly more likely to go to college than kids randomly assigned to a control group. To my knowledge, there is no more effective way of increasing the college attendance of kids who might or might not go, and this one costs almost nothing.

It so happens that my son was going into a Master’s program in Florida and had to fill out a FAFSA. Knowing about the H&R Block study, I suggested he take it to H&R Block office near him that had just done his tax forms. Needless to say, they didn’t provide the service. I’ve since learned that even though H&R Block paid for Bettinger’s study (with the help of grants from the Bill & Melinda Gates Foundation, the National Science Foundation, and other sources), their offices rarely offer the FAFSA service.

I tell this story because I think it speaks volumes about inequities and idiocies in American education. First, it exposes one of the many enormous benefits kids get if they just have the good sense to be born to middle-class, literate parents (who can help them fill out a FAFSA). Second, why is it that school districts or colleges themselves cannot provide the service H&R Block was experimenting with (but later decided not to offer)? Third, if it does take H&R Block or other tax preparers to do a FAFSA, then why can’t every low income parent of a high school kid hoping to go to college get a voucher to have their local tax preparer help them fill out a FAFSA form?

This is not my field, so perhaps all of these things are being done, BUT I STRONGLY DOUBT IT. Instead, my rather confident guess is that the system is happily cranking along, effectively barring deserving, capable, and promising young people from a brighter future because it’s no one’s job to solve this little FAFSA problem. We spend billions, by the way, in financial aid and elaborate programs to help able, disadvantaged kids go to college. It’s not that we’re unwilling to spend money. It’s just that we’re unwilling to follow the evidence until we find solutions to the core problems of our society.

Gold-Standard Program Evaluations, on a Shoestring Budget

Note: This is a guest post by Jon Baron, President of the Coalition for Evidence-Based Policy, and Chairman of the National Board for Education Sciences

In today’s tough economic climate, quality evaluations of education reforms – to determine which are truly effective in improving student achievement, graduation rates, and other key outcomes – are especially important. They enable us to focus our limited resources on strategies that have been proven to work.

Well-conducted randomized controlled trials are generally recognized as the most reliable method (the “gold standard”) for evaluating a program’s effectiveness. However, widespread misconceptions about what such studies involve – including their cost – have often limited their use by education officials.

In plain language: Randomized controlled trials in education are studies that randomly assign a sample of students, teachers, or schools to a group that participates in the program (“the program group”) or to a group that does not (“the control group”). With a sufficiently large sample, this process helps ensure that the two groups are equivalent, so that any difference in their outcomes over time – such as student achievement – can be attributed to the program, and not to other factors.

Such studies are often perceived as being too costly and administratively burdensome to be practical in most educational settings. In fact, however, it is often possible to conduct such a study at low cost and burden if the study can measure outcomes using state test scores or other administrative data that are already collected for other purposes. Costs are reduced by eliminating what is typically the study’s most labor-intensive and costly component: locating the individual sample members at various points in time after program completion, and administering tests or interviews to obtain their outcome data. In some cases, the only remaining cost is the researcher’s time to analyze the data.

For example, the following are two recent randomized trials that were conducted at low cost, yet produced findings of policy and practical importance:

Roland Fryer, recent winner of the MacArthur “Genius” Award, conducted an evaluation of New York City’s $75 million Teacher Incentive Program in which 396 of the city’s lowest-performing public schools were randomly assigned to an incentive group, which could receive an annual bonus of up to $3000 per teacher if the school increased student achievement and other key outcomes, or a control group. Three years after random assignment, the study found that the incentives had no effect on student achievement, attendance, graduation rates, behavior, GPA, or other outcomes. Based in part on these results, the city recently ended the program, freeing up resources for other efforts to improve student outcomes.

The study’s cost: Approximately $50,000. The low cost was possible because the study measured all outcomes using state test scores and other administrative records already collected for other purposes.
Eric Bettinger and Rachel Baker conducted an evaluation of InsideTrack college coaching – a widely-implemented mentoring program for college students designed to prevent them from dropping out of school. This was a well-conducted trial, which randomized more than 13,000 students at eight colleges. The study found that the program produced a 14 percent increase in college persistence for at least two years, and a 13 percent increase in likelihood of graduating college.

The study’s cost: Less than $20,000. The low cost was possible because the study measured its key outcomes using administrative data that the colleges already collected for other purposes – i.e., their enrollment and graduation records – rather than by collecting new data through individual surveys.
In recent years, federal and state policy, as well as improvements in information technology, have greatly increased the availability of high-quality administrative data on student achievement and other key educational outcomes. Thus, it has become more feasible than ever before to conduct gold-standard randomized evaluations on a shoestring budget. Equipped with reliable evidence, education officials can have much greater confidence that their spending decisions will produce important improvements in student outcomes.

-Jon Baron

The Coalition for Evidence-Based Policy is a nonprofit, nonpartisan organization whose mission is to increase government effectiveness through the use of rigorous evidence about “what works.”

Education Innovation: What It Is and Why We Need More of It

NOTE: This is a guest post from Jim Shelton, Assistant Deputy Secretary of the Office of Innovation and Improvement at the U.S. Department of Education.

Whether for reasons of economic growth, competitiveness, social justice or return on tax-payer investment, there is little rational argument over the need for significant improvement in U.S. educational outcomes. Further, it is irrefutable that the country has made limited improvement on most educational outcomes over the last several decades, especially when considered in the context of the increased investment over the same period. In fact, the total cost of producing each successful high school and college graduate has increased substantially over time instead of decreasing – creating what some argue is an inverted learning curve.

This analysis stands in stark contrast to the many anecdotes of teachers, schools and occasionally whole systems “beating the odds” by producing educational outcomes well beyond “reasonable” expectations. And, therein lies the challenge and the rationale for a very specific definition of educational innovation.

Education not only needs new ideas and inventions that shatter the performance expectations of today’s status quo; to make a meaningful impact, these new solutions must also “scale”, that is grow large enough, to serve millions of students and teachers or large portions of specific under-served populations. True educational innovations are those products, processes, strategies and approaches that improve significantly upon the status quo and reach scale.

Shelton graphic.JPG

Systems and programs at the local, state and national level, in their quest to improve, should be in the business of identifying and scaling what works. Yet, we traditionally have lacked the discipline, infrastructure, and incentives to systematically identify breakthroughs, vet them and support their broad adoption – a process referred to as field scans. Programs like the Department of Education’s Investing in Innovation Fund (i3) are designed as field scans; but i3 is tiny in comparison to both the need and the opportunity. To achieve our objectives, larger funding streams will need to drive the identification, evaluation, and adoption of effective educational innovations.

Field scans are only one of three connected pathways to education innovation, and they build on the most recognized pathway – basic and applied research. The time to produce usable tools and resources from this pathway can be long – just as in medicine where development and approval of new drugs and devices can take 12-15 years – but, with more and better leveraged resources, more focus, and more discipline, this pathway can accelerate our understanding of teaching and learning and production of performance enhancing practices and tools.

The third pathway focuses specifically on accelerating transformational breakthroughs, which require a different approach – directed development. Directed development processes identify cutting edge research and technology (technology generically, not specifically referring to software or hardware) and use a uniquely focused approach to accelerate the pace at which specific game changing innovations reach learners and teachers. Directed development within the federal government is most associated with DARPA (the Defense Advanced Research Projects Agency), which used this unique and aggressive model of R&D to produce technologies that underlie the Internet, GPS, and the unmanned aircraft (drone). Education presents numerous opportunities for such work. For example: (1) providing teachers with tools that identify each student’s needs and interests and match them to the optimal instructional resources or (2) cost-effectively achieving the 2 standard deviations of improvement that one-to-one human tutors generate. In 2010, the President’s Council of Advisors on Science and Technology recommended the creation of an ARPA for Education to pursue directed development in these and other areas of critical need and opportunity.

Each of these pathways -the field scan, basic and applied research and directed development – will be essential to improving and ultimately transforming learning from cradle through career. If done well, we will redefine “the possible” and reclaim American educational leadership while addressing inequity at home and abroad. At that point, we may be able to rely on a simpler definition of innovation:

“An innovation is one of those things that society looks at and says, if we make this part of the way we live and work, it will change the way we live and work.”

-Dean Kamen

-Jim Shelton

Note: The Office of Innovation and Improvement at the U.S. Department of Education administers more than 25 discretionary grant programs, including the Investing in Innovation Program, Charter Schools Program, and Technology in Education.