Whadja Do In School Today?

Every parent of a four or five year old knows the drill. Your child comes home after pre-kindergarten or kindergarten. “Whadja do today?” you  say with eager anticipation, thinking of all the friends your child must have made, the stories your child heard, the songs your child sang, the projects or dress-up or phonics or math, or…well, anything.

“Nuffin,” your child says, wandering out of range to avoid further interrogation.

You know your child did a whole lot more than “nuffin.” But how can you find out so that you can build on what the teacher did each day?

One answer is something our group at the Success for All Foundation created utilizing Investing in Innovation (i3) funding with partners at Sesame Workshop, Sirius Thinking, and Johns Hopkins University. We call it Home Links. Home Links are 10-15 minute videos, akin to short television shows, that parents and children watch together, 4 evenings a week. Each show uses content from Sesame Street and animations we have made with Sirius Thinking, so they are a bit like Sesame Street shows themselves, with one huge difference: the content of the shows reflects the activities that children and teachers were doing that day in school.

The Home Links give kids reinforcement and extension of vocabulary and skills they learned that day, and that’s important. But more important, they tell parents what’s happening in school. When a show contains skits about fall, the letter V, counting to five, and singing traditional songs, the parents know that all of these things are happening in school. Our surveys found that 96% of the time, a parent, grandparent, or other relative watches with the child. At the end of each show there is music and movement, and parents tell us they dance with their children, and they love the closeness and fun. But parents also now know how to support their children’s learning. If the topic is markets, they know to point out interesting things when they next are at the market with their child. If the letter is T, they know to point out things that begin with T. If the math segment is on shapes, parents know to ask children about shapes they see in daily life. Home should not be another classroom, but it’s the ideal place for a child to learn that the things he or she is learning in school are important to his or her parents and exist in his or her community. It also helps children understand that knowing about and learning about those things brings pride and builds curiosity.

Home Links are sent home on DVDs each day. We are now looking for funding to make an online version so families can download Home Links to digital devices such as phones and tablets.

Right now, Home Links are being used in approximately 300 preschool and kindergarten classes already using our proven Success for All whole-school approach. In the future, we hope to disseminate Home Links to preschools and kindergartens whether or not they use Success for All.

When this happens, more and more parents won’t have to ask, “Whadja do in school today?” They’ll know. And they’ll know how to build on what they find out.

And that ain’t nuffin’.

 

The Investing in Innovation (i3) program is a federal competitive grant program at the U.S. Department of Education, within the Office of Innovation and Improvement (OII). It provides funding to support local education agencies or nonprofit organizations in partnership with LEAs and/or schools to expand and develop innovative practices that can serve as models of best practices and to identify and document best practices that can be shared and taken to scale in the areas of improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

More information on the i3 program can be found here.

More information on Success for All Foundation’s grant Around the Corner: A Technology-Enhanced Approach to Early Literacy can be found here.

Advertisements

What Schools in One Place Can Learn from Schools Elsewhere

In a recent blog, I responded to an article by Lisbeth Schorr and Srik Gopal about their concerns that the findings of randomized experiments will not generalize from one set of schools to another. I got a lot of supportive response to the blog, but I realize that I left out a key point.

The missing point was this: the idea that effective programs readily generalize from one place to another is not theoretical. It happens all the time. I try to avoid talking about our own programs, but in this case, it’s unavoidable. Our Success for All program started almost 30 years ago, working with African American students in Baltimore. We got terrific results with those first schools. But our first dissemination schools beyond Baltimore included a Philadelphia school primarily serving Cambodian immigrants, rural schools in the South, small town schools in the Midwest, and so on. We had to adapt and refine our approaches for these different circumstances, but we found positive effects across a very wide range of settings and circumstances. Over the years, some of our most successful schools have been ones serving a Native Americans, such as a school in the Arizona desert and a school in far northern Quebec. Another category of schools where we see outstanding success is ones serving Hispanic students, including English language learners, as in the Alhambra district in Phoenix and a charter school near Los Angeles. One of our most successful districts anywhere is in small-city Steubenville, Ohio. We have established a successful network of SFA schools in England and Wales, where we have extraordinary schools primarily serving Pakistani, African, and disadvantaged White students in a very different policy context from the one we face in the U.S. And yes, we continue to find great results in Baltimore and in cities that resemble our original home, such as Detroit.

The ability to generalize from one set of schools to others is not at all limited to Success for All. Reading Recovery, for example, has had success in every kind of school, in countries throughout the world. Direct Instruction has also been successful in a wide array of types of schools. In fact, I’d argue that it is rare to find programs that have been proven to be effective in rigorous research that then fail to generalize to other schools, even ones that are quite different. Of course, there is great variation in outcomes in any set of schools using any innovative program, but that variation has to do with leadership, local support, resources, and so on, not with a fundamental limitation on generalizability to additional populations.

How is it possible that programs initially designed for one setting and population so often generalize to others? My answer would be that in most fundamental regards, the closer you get to the classroom, the more schools begin to resemble each other. Individual students do not all learn the same way, but every classroom contains a range of students who have a predictable set of needs. Any effective program has to be able to meet those needs, wherever the school happens to be located. For example, every classroom has some number of kids who are confident, curious, and capable, some number who are struggling, some number who are shy and quiet, some number who are troublemakers. Most contain students who are not native speakers of English. Any effective program has to have a workable plan for each of these types of students, even if the proportions of each may vary from classroom to classroom and school to school.

There are reasonable adaptations necessary for different school contexts, of course. There are schools where attendance is a big issue and others where it can be assumed, schools where safety is a major concern and others where it is less so. Schools in rural areas have different needs from those in urban or suburban ones, and obviously schools with many recent immigrants have different needs from those in which all students are native speakers of English. Involving parents effectively looks different in different places, and there are schools in which eyeglasses and other health concerns can be assumed to be taken care of and others where they are major impediments to success. But after the necessary accommodations are made, you come down to a teacher and twenty to thirty children who need to be motivated, to be guided, to have their individual needs met, and to have their time used to greatest effect. You need to have an effective plan to manage diverse needs and to inspire kids to see their own possibilities. You need to fire children’s imaginations and help them use their minds well to write and solve problems and imagine their own futures. These needs exist equally in Peru and Poughkeepsie, in the Arizona desert or the valleys of Wales, in Detroit or Eastern Kentucky, in California or Maine.

Disregarding evidence from randomized experiments because it does not always replicate is a recipe for the status quo, as far as the eye can see. And the status quo is unacceptable. In my experience, the reason programs fail to replicate is that they were never all that successful in the first place, or because they attempt to replicate a form of a model much less robust than the one they researched.

Generalization can happen. It happens all the time. It has to be planned for, designed for, not just assumed, but it can and does happen. Rather than using failure to replicate as a stick to beat evidence-based policy, let’s agree that we can learn to replicate, and then use every tool at hand to do so. There are so many vulnerable children who need better educations, and we cannot be distracted by arguments that “nothing replicates” that are contradicted by many examples throughout the world.

An Exploded View of Comprehensive School Reform

Recently, I had to order a part for an electric lawnmower. I enjoyed looking at the exploded view (similar to the one above) on the manufacturer’s web site. What struck me about it was that so many of the parts were generic screws, bolts, springs, wheels, and so on. With a bit of ingenuity, I’m sure someone (not me!) could track down generic electric motors, mower blades, and other more specialized parts, and build their very own do-it-yourself lawn mower.

There are just a few problems with this idea.

  1. It would cost a lot more than the original mower
  2. It would take a lot of time that could possibly be used for better purposes
  3. It wouldn’t work and you’d end up with an expensive pile of junk to discard.

Why am I yammering on about exploded views of lawn mowers? Because the idea of assembling lawn mowers from generic parts is a lot like what all too many struggling schools do in the name of whole school reform.

In education, the equivalent do-it-yourself idea using generic parts is the idea that if you choose one program for reading and another for behavior and a third for parent involvement and a fourth for tutoring and a fifth for English learners and a sixth for formative assessment and a seventh for coaching, the school is bound to do better. It might, but this piecemeal approach is really hard to do well.

The alternative to assembling all of those generic parts is to adopt a comprehensive school improvement model. These are models that have coordinated, well worked-out, well-supported approaches to increasing student success. Our own Success for All program is one of them, but there are others for elementary and secondary schools. After years of encouraging schools receiving School Improvement Grants (SIG) to assemble their own comprehensive reforms (remember the lawn mower?), the U.S. Department of Education finally offered SIG schools the option of choosing a proven whole-school approach. In addition to our Success for All program, the U.S. Department of Education approved three other comprehensive programs based on their evidence of effectiveness: Positive Action, the Institute for Student Achievement, and New York City’s small high schools of choice approach. These all met the Department’s standards because they had at least one randomized experiment showing positive outcomes on achievement measures, but some had a lot more evidence than that.

Comprehensive approaches resemble the fully assembled lawn mower rather than the DIY exploded view. The parts of the comprehensive models may be like those of the do-it-yourself SIG models, but the difference is that the comprehensive models have a well-thought-out plan for coordinating all of those elements. Also, even if a school used proven elements to build its own model, those elements would not have been proven in combination, and each might compete for the energies, enthusiasm, and resources of the beleaguered school staff.

This is the last year of SIG under the old rules, but it will continue in a different form under ESSA. The ESSA School Improvement provisions require the use of programs that meet strong, moderate, or promising evidence standards. Assembling individual proven elements is not a terrible idea, and is a real improvement on the old SIG because it at least requires evidence for some of the parts. But perhaps broader use of comprehensive programs with strong evidence of effectiveness for the whole school-wide approach, not just the parts, will help finally achieve the bold goals of school improvement for some of the most challenging schools in our country.

Love and Evidence

Valentine’s Day is this Sunday. If you are spending it thinking about effect sizes or research designs or education policy, shame on you. Unless, of course, that sort of thing turns you on.

So what does love have to do with evidence? Everything, actually. Our field is education. Education is empty without love. Evidence helps teachers and principals give every child the best possible chance to achieve success in school and in life. An educator who loves children wants the best for them. The purpose of educational research, development, and evaluation is to provide educators with pragmatic means of showing their love for children. Love without effective teaching is not enough, of course, and technically proficient teaching means little without love. But the two together are the most powerful force in education.

Children, especially young ones, completely trust their teachers. They look up to them with hope and respect. They are easy to love, even if sometimes hard to teach. But how can we give them any less than what we know how to give? Evidence does not provide all the answers or solve all the problems, but how is it responsible and loving to ignore evidence that could help students succeed?

I recently heard a story that illustrates what I’m talking about. A mother in a poor, Appalachian school in Kentucky came to meet with her daughter’s middle school principal. The school was using our Success for All program, which was adopted to improve very low reading proficiency rates. Even though the staff voted to adopt the research proven approach, there was some grumbling about the instructional processes that were required by the program among some of the staff. After all, change is hard. The principal was considering letting some teachers opt out.

The mother told the principal that her daughter, now in eighth grade, had never been able to read. Because of the school’s new program, she was now learning, excitedly bringing home books to read aloud to her.

The mother burst into tears. She’d never heard her daughter read to her before. She urged the principal to hold her ground and keep the program. Ultimately she did so.

This incident, repeated thousands of times every year for many proven programs, is a direct product of decades of research, development, and dissemination. All that R&D might sound technical and boring. But the outcome is a concrete expression of our love for children.

Love comes in many forms. On Valentine’s Day, we celebrate one of them. But the rest of the year, let’s remember that as educators, our love for children has to drive everything we do, including our choice of programs and practices that work. How can we want anything less than the best for the children who depend on us?

Smart Philanthropy

Americans are very generous. We give more than $300 billion annually to every kind of charity, hoping to do good in the world. All charities have mission statements, stated right on the dozens of calendars they send us, and most claim to be making a difference in some valued outcome. Yet tough-minded, tender-hearted donors want to know more. Is their donation producing a concrete outcome?

Dean Karlan, a Yale economist, has just launched a new organization, called ImpactMatters, to do “impact audits” on nonprofits. These focus primarily on assessing impact of the charities’ services on the outcomes they claim. Karlan’s team examines the evidence, particularly randomized experiments, as well as financial and management issues, and picks out charities that are transparent, well-managed, and making a well-documented difference.

In a December 4th Wall Street Journal article, Karlan introduced his organization and its purpose, and the first four programs identified by ImpactMatters as meeting its standards were named on December 11th on the ImpactMatters website. One was our Success for All reading program, and the others were international charities focused on the ultra-poor in developing countries and healthcare in Nepal.

The appearance of ImpactMatters could make a difference in philanthropy, and that would be terrific in itself. However, its significance goes far beyond philanthropy.

ImpactMatters is one more indication that good intentions are no longer sufficient. Government, philanthropists, and leaders of all kind are increasingly demanding rigorous evidence of impact. We all know where the road paved with good intentions goes. The road paved with good evidence, acted upon with integrity, purpose, and caring, goes straight to heaven. Karlan’s stated purpose is to help people give with their minds, not just their hearts. I hope this will make the difference it is intended to make. Personally, I’d rather get a lot fewer calendars and a lot more impact for my donations. Doesn’t everyone feel the same way?

Show Me the Evidence

2015-01-22-Evidence01_22_2015.jpg

This year, evidence-based reform in education got off to a great start with an article by Ron Haskins in the New York Times on December 31 explaining why evidence of effectiveness must become an expected part of the process by which policy ideas are adopted (or not). More recently, I received a book Haskins wrote with colleague Greg Margolis on this topic, Show Me the Evidence. Both the article and the book mentioned our Success for All program as an example of what “proven” looks like in education, but they are a lot more important than that.

Haskins makes a powerful argument for putting all social programs to the test. Those that work should be expanded. Those that don’t should be replaced by other approaches that work better.

The need for evidence should be obvious, but very few federal programs have evidence of effectiveness. Few even have a process for finding out what works and encouraging grantees to use proven approaches, instead of approaches with the same desired outcomes that do not work or whose effects are unknown. Haskins estimates the 75 percent of programs and practices intended to help people do better at school or work have little or no impact. Such programs are well-meaning, but they need to be improved or replaced with equally well-meaning approaches known under well-defined circumstances to have positive impacts.

The importance of Haskins’ article and book lies especially in the importance of Haskins himself. He knows whereof he speaks. Advisor to House Republicans in the 1990s and then an advisor to President George W. Bush on social policy, he was a key architect of the 1996 welfare overhaul. Welfare programs that worked improved peoples’ lives and saved federal and state governments billions of dollars. Those that didn’t were replaced. To this day, those innovations represent the best example of evidence-driven policy.

Haskins is a proud Republican. He wants every dollar of federal expenditure to do what it is intended to do. Is there anyone, of any political persuasion, who does not want the same? This is not a question of ideology. It’s a question of sound governance.

When Ron Haskins and others were starting out, evidence was a pretty risky idea. Today, evidence is showing up throughout government — still not nearly as much as it should, but far more than it did. Sooner or later, government will become more competent and cost-effective at achieving goals we all share. Ron Haskins was there when it mattered. He still is, and it still does.

 

Teachers as Professionals in Evidence-Based Reform

2014-12-18-HP5512_18_2014.jpg

In a February 2012 op-ed in Education Week, Don Peurach wrote about a 14-year investigation he carried out as part of a large University of Michigan study of comprehensive school reform. In the overall study, our Success for All program and the America’s Choice program did very well in terms of both implementation and outcomes, while an approach in which teachers largely made up their own instructional approaches did not bring about much change in teachers’ behaviors or student learning. Because both Success for All and America’s Choice have well-specified training, teacher’s manuals, and student materials, the findings support the idea that it is important for school-wide reform models to have a well-structured approach.

Peurach’s focus was on Success for All as an organization. He wanted to know how our network of hundreds of schools in 40 states contributes to the development of the approach and to each other’s success. His key finding was that Success for All is not a top-down approach, but is constantly learning from its teachers and principals and then spreading good practices throughout the network.

In our way of thinking, this is the very essence of professionalism. A teacher who does wonderful, innovative things in one class is perhaps benefiting 25 children each year, but one whose ideas scale up to inform the practices of hundreds of thousands of schools is making a real difference. Yet in order for teachers’ ideas and impact to be broadly impactful, it helps a great deal for the teachers to be part of a national or regional network that speaks a common language and has common standards of practice.

Teachers need not be researchers to contribute to their profession. By participating in networks of like-minded educators – implementing, continuously improving and communicating about practical approaches intended to improve outcomes of proven approaches – they play an essential role in the improvement of their profession.