High-Reliability Organizations

I’m writing this blog from the inside of an airplane high above the Atlantic. I have total confidence that my plane will deliver me safely to Europe. It’s astonishing. The people who run every aspect of this plane are ordinary folk. I knew a guy in college who spent his entire career as a pilot for the very airline I’m flying today. He was competent, smart, and very, very careful. But he was not expected to make things up as he went along. He liked to repeat an old saying: “There are old pilots and there are bold pilots, but there are no old, bold pilots.”

When I was younger, I recall that airplane crashes were relatively common. These were always prominently reported in the news. But today, airplane disasters not caused by terrorists or crazy people are extremely rare. The reason is that air disasters are so catastrophic that airlines have adopted procedures in every aspect of their operation to ensure that planes arrive safely at their destinations. Every system important to safety is checked and rechecked, with technology and humans backing each other up. I happen to have a nephew who is studying to be an aircraft mechanic. His course is extremely rigorous. Most people don’t make it through. His final test, he says, will have 80 questions. The minimum acceptable score: 80. His brother is a nuclear engineer on a navy submarine. Same kind of training, same requirement for success. No room for error. The need for such care in airplanes and submarines is obvious. But why not in education?

My friend and colleague Sam Stringfield had this idea many years ago. Based on it, he and a Welsh colleague, David Reynolds, created what they called “high-reliability schools.” They evaluated them in Wales, and found substantially greater gains in schools using this approach than in control schools.

Despite its success, the high-reliability idea did not catch hold in education. Yet any student who is unnecessarily failing in school is a catastrophe waiting to happen. You don’t need a lot of data tables to be convinced that students not reading well by third grade are headed for big trouble. They are disproportionately likely to end up in special education, to repeat one or more grades, to drop out of high school, and to get into behavioral difficulties and problems with the law. Each of these outcomes is hugely damaging to the student and hugely expensive to the taxpayer.

Yet there is no problem in all of education that is better researched than early reading failure. There are many proven strategies known to greatly reduce reading failure: whole school methods, small group, individual tutoring, technology, and more. Our Evidence for ESSA web site lists dozens of proven approaches. It is probably already the case that any school could identify students at risk of reading failure in kindergarten or first grade and then apply proven, easily available methods conscientiously to ensure that virtually every child will succeed in reading.

The point here is that if we wanted to, we could treat early reading the way airlines and submarines treat safety, as a life or death issue.

If schools accepted the high-reliability challenge for early reading, here is what they would do. First, they’d adopt proven pre-reading programs for pre-kindergarten, and then proven beginning reading programs for grades K-3. Teachers of these grades would receive extensive professional development and then in-class coaching to help them use these proven strategies as well as they were used in the research that validated them, or better.

Starting in kindergarten, we’d start to assess students in early reading skills, so we’d know which students need assistance in which specific skills. We’d continue to assess all students over time to be sure that all are on a path to success. The assessments would include vision and hearing so that problems in these areas are solved.

Each school would have staff trained and equipped to provide an array of services for students who are in need of additional help. These would include small-group tutoring for students with mild problems, and one-to-one tutoring for more serious problems. Multiple proven programs, each focusing on distinct problems, would be ready to deploy for students who need them. Students who need eyeglasses, hearing accommodations, or other health assistance would be treated. Students who are English learners would receive assistance with language and reading.

The point is, each school would be committed to ensuring the success of every child, and would be prepared to do so. Like my high-reliability nephews, the goal of every person in every school would be zero failures. Not just fewer. Zero.

There is no question that this goal could be accomplished. The only issue is whether it could be accomplished at a cost that would be politically acceptable. My guess is that a full-scale, replicable schoolwide strategy to ensure zero reading failures in high-poverty schools could add about $200 per child per year, from grades pre-K to 3. A lot of money, you say? Recall from a previous blog that the average per-pupil cost in the U.S. is approximately $11,000. What if it were $11,200, just for a few years? The near-term savings in special education and retentions, much less longer-term costs of delinquency and dropout, would more than return this investment.

But more than cost-effectiveness, there is a moral imperative here. Failing children who could succeed is simply wrong. We could greatly reduce or eliminate this problem, just as the aircraft industry has done. Our society must come to see school failure as the catastrophe that it is, and to use whatever proven methods are needed to make reading failure a problem of the past.

This blog is sponsored by the Laura and John Arnold Foundation

Advertisements

The Maryland Challenge

As the Olympic Games earlier this summer showed, Americans love to compare ourselves with other countries. Within the U.S., we like to compare our states with other states. When Ohio State plays the University of Michigan, it’s not just a football game.

In education, we also like to compare, and we usually don’t like what we see. Comparisons can be useful in giving us a point of reference for what is possible, but a point of reference doesn’t help if it is not seen as a peer. For example, U. S. students are in the middle of the pack of developed nations on Program for International Student Assessment (PISA) tests for 15 year olds, but Americans expect to do a lot better than that. The National Assessment of Educational Progress (NAEP) allows us to compare scores within the U.S., and unless you’re in Massachusetts, which usually scores highest, you probably don’t like those comparisons either. When we don’t like our ranking, we explain it away as best as we can. Countries with higher PISA scores have fewer immigrants, or pay their teachers better, or have cultures that value education more. States that do better are richer, or have other unfair advantages. These explanations may or may not have an element of truth, but the bottom line is that comparisons on such a grand scale are just not that useful. There are far too many factors that are different between nations or states, some of which are changeable and some not, at least in the near term.

If comparisons among unequal places are not so useful, what point of reference would be better?

Kevan Collins, Director of the Education Endowment Foundation in England (England’s equivalent to our Investing in Innovation (i3) program), has an answer to this dilemma, which he explained at a recent conference I attended in Stockholm. His idea is based on a major, very successful initiative of Tony Blair’s government beginning in 2003, called the London Challenge. Secondary schools in the greater London area were put into clusters according to students’ achievement at the end of primary (elementary) school, levels of poverty, numbers of children speaking languages other than English at home, size, and other attributes. Examination of the results being achieved by schools within the same cluster showed remarkable variation in test scores. Even in the poorest clusters there were schools performing above the national average, and in the wealthiest clusters there were schools below the average. Schools low in their own clusters were given substantial resources to improve, with a particular emphasis on leadership. Over time, London went from being one of the lowest-achieving areas of England to scoring among the highest. Later versions of this plan in Manchester and in the Midlands did not work as well, but they did not have much time before the end of the Blair government meant the end of the experiment.

Fast forward to today, and think about states in the U. S. as the unit of reform. Imagine that Maryland, my state, categorized its Title I elementary, middle, and high schools according to percent free lunch, ethnic composition, percent English learners, urban/rural, school size, and so on. Each of Maryland’s Title I schools would be in a cluster of perhaps 50 very similar schools. As in England, there would be huge variation in achievement within clusters.

Just forming clusters to shame schools low in their own cluster would not be enough. The schools need help to greatly improve their outcomes.

This being 2016, we have many more proven programs than were available in the London Challenge. Schools scoring below the median of their cluster might have the opportunity to choose proven programs appropriate to their strengths and needs. The goal would be to assist every school below the median in its own cluster to at least reach the median. School staffs would have to vote by at least 80% in favor to adopt various programs. The school would also commit to use most of its federal Title I funds to match supplemental state or federal funding to pay for the programs. Schools above the median would also be encouraged to adopt proven programs, but might not receive matching funds.

Imagine what could happen. Principals and staffs could no longer argue that it is unfair for their schools to be compared to dissimilar schools. They might visit schools performing at the highest levels in their clusters, and perhaps even form coalitions across district lines to jointly select proven approaches and help each other implement them.

Not all schools would likely participate in the first years, but over time, larger numbers might join in. Because schools would be implementing programs already known to work in schools just like theirs, and would be held accountable within a fair group of peers, schools should see rapid growth toward and beyond their cluster median, and more importantly, the entire clusters should advance toward state goals.

A plan like this could make a substantial difference in performance among all Title I schools statewide. It would focus attention sharply where it is needed, on improved teaching and learning in the schools that need it most. Within a few years, Maryland, or any other state that did the same, might blow past Massachusetts, and a few years after that, we’d all be getting visits from Finnish educators!

Lighthouses

Visiting coastal lighthouses is one of the highlights of any beach vacation. Mostly relics of the 18th and 19th century, lighthouses played a vital role in helping harbor pilots and ship captains find channels and avoid hazards such as rocks and sandbars. They saved many ships and crews that would otherwise have been lost at night or in storms or fog.

In education, the idea of “lighthouse schools” comes up frequently. The notion here is to identify schools that are making impressive progress, especially in challenging circumstances, and then publicize their successes. The metaphor, of course, is that these “lighthouses” guide others to reach their goals.

Taken as a form of journalism, rather than a form of science, a policy of identifying and publicizing lighthouse schools in high-poverty neighborhoods makes a statement that success is actually possible. With stories of tough but loving principals, caring and hard-working staff and ringing praise from parents and politicians, lighthouse stories are irresistible feel-good press.

However, if what we want is evidence-based reform, identifying and describing lighthouse schools is less useful. The process is riddled with problems. First, you have to make sure that the lighthouse school is truly a shining example. In the 1980s, everyone was abuzz about District 2 in New York City, which was rapidly rising in academic performance using a variety of innovative methods and visionary leadership. I asked my New York friends at the time what was going on in District 2, and they fell on the floor laughing. It turned out that District 2 was quickly gentrifying. A former high-poverty area was now attracting many upper-middle class families eager for quality schools that did not charge tuition. Lighthouse schools often have such explanations.

Lighthouse schools are sometimes identified based on a single outstanding year, which may be in the past. This may mean that a temporarily-outstanding school is now rather ordinary.

Even if a given school has made major, sustained gains serving the same population it always served, it is rarely clear what caused the change. Was it the principal? New teachers? Additional resources? Innovative programs? In a single school, it is impossible to pick out what made the difference because everything is intertwined with everything else. A journalistic approach simply cannot account for this.

Further, even if we have strong suspicions about what makes the lighthouse shine, that factor may not be replicable. What if it is the principal? Another school might hire away that principal, but that hardly moves the system forward. What if a local foundation gave the school a pile of money? What if the school managed to attract volunteer tutors from across the city? Such advantages are possible in some schools or even districts, but not in others.

The whole lighthouse school idea is undercut by the observation that, whatever the “secret sauce” might be, it does not travel well. If it did, we would find not only lighthouse schools but lighthouse districts and states, in which great ideas spread outward using replicable methods. The very fact that one school stands out from its neighbors should give us pause about whether the neighbors have the capacity and the willingness to imitate success.

Lighthouse schools can certainly contribute ideas or inspiration to evidence-based reform, but before a program can be considered effective and replicable, it needs to be clearly defined and then evaluated by rigorous methods. Such evaluation would normally compare at least 20 schools assigned at random to implement the program to 20 similar schools that continue with their practice as usual. If such a study finds that the schools using the innovative program did better, then we really know something worthwhile. In this scenario, we could have confidence because factors other than the innovation balance out. The experimental and control schools are likely to have equal numbers of good principals, equal funding (on average), equally qualified teachers (on average) and so on. The only difference between the experimental and control schools is the “secret sauce” itself, which, if it can work in 20 schools or more, is probably replicable.

Lighthouses once guided ships to safe harbors, but in education, policies limited to finding and celebrating lighthouse schools are less likely to improve outcomes more broadly. They may lead policy in a good direction, but they may just as likely guide us onto the rocks.

The Message From Baltimore

Why Baltimore?

The tragic events of recent weeks could have happened in any big city in America. Rough treatment of minorities by police is hardly unique to Baltimore. Serious injuries and deaths are an all-too-common result. This is not to disparage individual law enforcement officers, who have to negotiate a complex and often hostile set of relationships that are not of their making. But again, this is not unique to Baltimore. After Ferguson, and New York, and North Charleston, a spark was bound to ignite somewhere, and that place happened to be Baltimore.

I have lived in Baltimore for more than 40 years. My wife and I have raised three children adopted from South America. One of them identifies himself as black. For years, he has told us stories of police harassment, being pulled over for “driving while black,” and being suspected of crimes he could not have committed. This is reality. I do not think there is any parent of a black child who does not understand this to be true, and who does not fear that an innocent act, a taunting remark, or simply being in the wrong place at the wrong time could end up with real harm at the hands of the police.

But this reality exists in every big city. Why was Baltimore the flashpoint?

One reason, ironically, is that Baltimore has been getting better. A down-at-the-heels port city in the 1970s, it is now improving economically, socially, and physically. Also, almost the entire power structure of the city is African-American. This includes the mayor, police chief, state’s attorney, and other leaders. An African-American middle class is growing rapidly but moving to the suburbs. Other middle-class people are moving into city neighborhoods and adding vitality and new businesses.

However, the people left in the inner city are not benefitting from all of this change. Many are unemployed or underemployed. Too many are dropouts or have such poor basic skills that they can only qualify for low-level jobs.

The anger and frustration of many Baltimoreans is understandable. They see people all around them, including people who look like them, entering the middle class. But from their point of view, the American dream remains unattainable.

My colleagues and I work in the schools that serve the very communities affected by the recent disturbances. The children who come to these schools start off bright-eyed and bushy-tailed, full of enthusiasm and confidence, like children everywhere. But then all too many of them experience failure. And all that motivation drains away. The majority of fourth graders read below the “basic” level on NAEP Reading. Poor reading skills make it almost impossible for children to grow up with confidence, graduate from high school, and go on to college or high-paying jobs.

None of this is surprising to any educator. But here is what should be infuriating: School failure is preventable. With every passing year we gain increasing evidence that virtually all children can learn to read and to succeed in other subjects. Using EDGAR standards, I count 28 elementary reading programs ready for use today that have at least moderate evidence of effectiveness. Yet none of these is used broadly enough to solve the reading problem in high-poverty schools. There is no reason that every Title I school in America should not be using proven programs.

The smoke rising over Baltimore in recent days is a signal to the whole country. Until we make opportunities available to all and treat everyone with respect and dignity, we can expect the frustrations of those who are shut out to boil over. Many things need to change, but one of the most fundamental is also among the easiest: Make sure that every child, whatever his or her background, learns to become a successful and confident reader. We know how to do it. Now let’s do it.

On Beyond Preschool: Alleviating Poverty Over a Lifespan

2014-09-17-HP44_9_17_2014.jpg

Preschool is good for disadvantaged children. But is preschool alone sufficient to significantly reduce poverty and inequality? Like many researchers, I’m delighted about the current enthusiasm for preschool at the policy level, and would not want to throw cold water on it. Yet almost all studies of garden-variety replicable preschool programs show that the clear positive effects of preschool in the early grades fade as children go through elementary school. This should not be a cause for despair, but rather for realism about how to help children succeed all the way through to adulthood. If preschool is seen as the first step in a multi-step societal strategy, it is worthy of all the attention and investment it is currently receiving. If we’re counting on preschool alone to solve all problems, we’re just not being serious.

Isabel Sawhill and Quentin Karpilow at the Brookings Institution recently issued a paper making exactly this point, and illustrating it with evidence. They argue for intervening early and often rather than once and done. To support their position, they take proven, replicable programs that are readily available and imagine that government provided disadvantaged children with all of them in sequence, from early childhood to adolescence.

Sawhill & Karpilow start with a gap in “success rates” at various ages between children born into families living above or below 200 percent of the poverty line. The gap in chances of entering the middle class is 20 percentage points at age 40.

According to the authors, preschool participation plus the HIPPY parenting program could reduce a 22 percent “success gap” at school entry by 14 percentage points. Yet this falls to 5 points by middle childhood, 3 points by adolescence, and 2 points by middle age. By this analysis, preschool is necessary but not sufficient to significantly reduce inequality.

Sawhill and Karpilow then (statistically) added in our Success for All program plus social-emotional learning interventions in elementary school. They also added in the Talent Development high school model.

The final result, by their estimates, was an elimination of the poverty gap in early and middle childhood. By adulthood, these interventions were estimated to reduce the 20-point success gap by 15 percentage points.

The exact size of the gap reduction is speculative, of course, but it’s the thinking that matters here, not the specifics. It never made sense that one big inoculation in preschool would set up a child for success throughout his or her life. Poverty isn’t like polio, which can be cured in one treatment. The factors that lead to a child being in a disadvantaged family at preschool are likely to persist afterwards, and top-quality education is needed at every age to help children overcome effects of poverty. Seen as a good start in a series of proven approaches appropriate to children’s needs from preschool through high school, preschool makes good sense. But if we fail to follow up with effective programs for the elementary and secondary grades, we will still have a lot of unnecessary inequality when today’s toddlers become tomorrow’s taxpayers.

Restoring Opportunity

2014-03-13-HP32March_13_14.jpg

I just read a fascinating book, Restoring Opportunity, by Greg Duncan and Richard Murnane.. It describes the now-familiar problems of growing inequality in America between the educational haves and have-nots, but then goes on to describe some outstanding preschool, elementary, and high school programs that may offer models of how to help disadvantaged children close the gap. Refreshingly, Duncan and Murnane do not stop with heartwarming tales of successful schools, but also present data from randomized experiments showing the impacts on children, especially for the small high school initiative in New York City and the University of Chicago Charter Network.

From these and other examples, Duncan and Murnane derive some factors common to outstanding schools: Accountability for outcomes within schools, extensive professional development and support for teachers, and experimentation and evaluation to identify effective models.

Readers of this blog won’t be surprised to learn that I support each of these recommendations. So let’s start there. How do we get more than 100,000 schools to become markedly more effective? Or to make the problem a little easier, how about just the 55,000 Title I schools?

Duncan and Murnane are forthright about many of the solutions that aren’t likely to make a widescale difference. They note that while there are a few promising charter management organizations, charters overall are not generally being found to improve learning outcomes, and some of the most celebrated charters achieve good results by burning out young, talented teachers, a strategy that is hard to sustain and harder to scale. Popular solutions such as ratcheting up accountability, providing vouchers, and changing governance have also been disappointing in evaluations.

There is a strategy that puts all of Duncan and Murnane’s principles into practice on a very large scale: Comprehensive school reform. They note the strong evaluation results and widespread impact of two CSR models, one of which is our Success for All programfor elementary and middle schools. CSR models exemplify the principles Duncan and Murnane arrive at, but they can do so at a substantial scale. That is, they invariably provide a great deal of professional development and support to teachers, accountability for outcomes within schools, links to parents, provisions for struggling students, and so on. Unlike charters, CSR models do not require radical changes in governance, which explains why they have been able to go to substantial scale far more rapidly.

The only problem with comprehensive school reform models is that at present, there are too few to choose from. Yet with support from government and foundations, this could rapidly change.

Imagine a situation in which Title I school staffs could choose among proven, whole-school reform models the one they thought best for their needs. The schools themselves and the leaders of their CSR reform networks would be responsible for the progress of children on state standards, but otherwise these schools would be free to implement their proven models with fidelity, without having to juggle district and CSR requirements.

Such a strategy could accomplish the goals Duncan and Murnane outline in thousands of schools, enough to make real inroads in the problems of inequity they so articulately identify.

Eliminating Achievement Gaps in England

2014-01-30-HPEnglandImage013014.jpg

Imagine that the U.S. government released a report showing that the achievement gap between White students and African American and Hispanic children had been eliminated. (In fact, the gap has hardly budged since 1980.) Such a report would cause dancing in the streets, champagne all around, and a genuine sense of a job well done.

Just such a report was released, but in England, not the U.S. In that country, longstanding achievement gaps have been greatly diminished or eliminated, according to Ofsted, an independent government agency that manages school inspectors. As recently as 2007, White English students far outscored those who were Black African, Afro-Carribean, Pakistani, or Bangladeshi. Today, all of these gaps have disappeared, or are now very small, in reading and math. The result is that instead of worrying about the achievement gap, the English are worrying about the perceived low performance of White students. However, in actuality, White students also made gains over the same period, just not as much as minorities.

How did the English accomplish this feat of closing the achievement gap, and what can we learn from their efforts in the U.S.? Such a large shift may have many causes, but the Ofsted report gives the main credit to a massive investment made by the previous Labor government called the London Challenge. This initiative provided substantial professional development and assistance to high-poverty schools throughout the London metropolitan area. A Manchester Challenge did the same in that city. Since most minority students in England live in London or Manchester, this appears to have been enough to affect reading and math gaps in the entire country.

England is very similar to the U.S. in most ways. It has similar income per capita, and its overall scores on international tests such as PISA and TIMSS are similar to ours. However, some aspects of the English context may not transfer to the U.S. For example, the proportion of students in England who are minorities is just 7%, compared to 30% in the U.S. A key structural difference is that although there are districts (local authorities) in England, they are very weak. Principals (head teachers) and boards of governors for each school have great autonomy, but the national government plays a much stronger role than in the U.S. What this means is that if the national government decides to intervene to improve schools, they can do so, and this is what happened in the London and Manchester Challenges. Also, basic per-pupil funding in the UK is equalized, as it is in all civilized countries, and in fact schools with many poor or minority students get extra funding. This contrasts with the situation in the U.S., where funding is largely dependent on local or state tax receipts, so poor areas receive much less than rich ones. A London or Manchester Challenge therefore builds on a level playing field, while in the U.S., just getting to equality would be a major accomplishment.

The most important lesson from the English example is this: achievement gaps are not inexorable. Substantial investment and concerted effort can make a meaningful difference. The English example does not provide a simple road map for the U.S., but it should remove a lot of excuses for the persistence of gaps in academic achievement in our country.