Evidence-Based Reform and the Multi-Academy Trust

Recently, I was in England to visit Success for All (SFA) schools there. I saw two of the best SFA schools I’ve ever seen anywhere, Applegarth Primary School in Croyden, south of London, and Houldsworth Primary School in Sussex, southeast of London. Both are very high-poverty schools with histories of poor achievement, violence, and high staff turnover. Applegarth mostly serves the children of African immigrants, and Houldsworth mostly serves White students from very poor homes. Yet I saw every class in each school and in each one, children were highly engaged, excited, and learning like crazy. Both schools were once in the lowest one percent of achievement in England, yet both are now performing at or above national norms.

In my travels, I often see outstanding Success for All schools. However, in this case I learned about an important set of policies that goes beyond Success for All, but could have implications for evidence-based reform more broadly.

Both Applegarth and Houldsworth are in multi-academy trusts (MATs), the STEP Trust and the Unity Trust, respectively. Academies are much like charter schools in the U.S., and multi-academy trusts are organizations that run more than one academy. Academies are far more common in the U.K. than the U.S., constituting 22% of primary (i.e., elementary) schools and 68% of secondary schools. There are 1,170 multi-academy trusts, managing more than 5,000 of Britain’s 32,000 schools, or 16%. Multi-academy trusts can operate within a single local authority (school district) (like Success Academies in New York City) or may operate in many local authorities. Quite commonly, poorly-performing schools in a local authority, or stand-alone academies, may be offered to a successful and capable multi-academy trust, and these hand-overs explain much of the growth in multi-academy trusts in recent years.

What I saw in the STEP and Unity Trusts was something extraordinary. In each case, the exceptional schools I saw were serving as lead schools for the dissemination of Success for All. Staff in these schools had an explicit responsibility to train and mentor future principals, facilitators, and teachers, who spend a year at the lead school learning about SFA and their role in it, and then taking on their roles in a new SFA school elsewhere in the multi-academy trust. Over time, there are multiple lead schools, each of which takes responsibility to mentor new SFA schools other than their own. This cascading dissemination strategy, carried out in close partnership with the national SFA-UK non-profit organization, is likely to produce exceptional implementations.

I’m sure there must be problems with multi-academy trusts that I don’t know about, and in the absence of data on MATs throughout Britain, I would not take a position on them in general. But based on my limited experience with the STEP and Unity Trusts, this policy has particular potential as a means of disseminating very effective forms of programs proven effective in rigorous research.

First, multi-academy trusts have the opportunity and motivation to establish themselves as effective. Ordinary U.S. districts want to do well, of course, but they do not grow (or shrink) because of their success (or lack of it). In contrast, a multi-academy trust in the U.K. is more likely to seek out proven programs and implement them with care and competence, both to increase student success and to establish a “brand” based on their effective use of proven programs. Both STEP and Unity Trusts are building a reputation for succeeding with difficult schools using methods known to be effective. Using cascading professional developing and mentoring from established schools to new ones, a multi-academy trust can build effectiveness and reputation.

Although the schools I saw were using Success for All, any multi-academy trust could use any proven program or programs to create positive outcomes and expand its reach and influence. As other multi-academy trusts see what the pioneers are accomplishing, they may decide to emulate them. One major advantage possessed by multi-academy trusts is that much in contrast to U.S. school districts, especially large, urban ones, multi-academy trusts are likely to remain under consistent leadership for many years. Leaders of multi-academy trusts, and their staff and supporters, are likely to have time to transform practices gradually over time, knowing that they have the stable leadership needed for long-term change.

There is no magic in school governance arrangements, and no guarantee that many multi-academy trusts will use the available opportunities to implement and perfect proven strategies. Yet by their nature, multi-academy trusts have the opportunity to make a substantial difference in the education provided to all students, especially those serving disadvantaged students. I look forward to watching plans unfold in the STEP and Unity Trusts, and to learn more about how the academy movement in the U.K. might provide a path toward widespread and thoughtful use of proven programs, benefiting very large numbers of students. And I’d love to see more U.S. charter networks and traditional school districts use cascading replication to scale up proven, whole-school approaches likely to improve outcomes in disadvantaged schools.

Photo credit: Kindermel [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)]

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.

The Sailor and the Sailboat: Leadership and Evidence

My one extravagance is that I live on the Chesapeake Bay and have a small sailboat. I love to sail, even if I’m not especially good at it, but sailing small boats teaches you a lot of important life lessons.

One of these lessons is that leadership is crucial, but leadership can only make a difference if leaders have the tools to translate leadership into outcomes.

Here’s what I mean from a sailing perspective. A sailboat is just a hull, sails, lines, a mast, a rudder, and a centerboard. When these are all in good working order, it still takes a good sailor to manage a small sailboat in heavy weather. However, when any one component is lacking, all hell breaks loose. For example, on my 11-foot sailboat, sometimes the rudder falls off in rough water. Without a rudder, it doesn’t matter how good a sailor you are. You aren’t going anywhere. Similarly, we once lost a mast in a heavy wind. Yikes!

Principals and superintendents in Title I schools are a lot like small-boat sailors in heavy weather, every single day. If all the structures and supports are in place, and if they have a great crew, capable school or district leaders can do wonders for their children.

Proven programs do not manage schools on their own. What they do is help provide the sails, mast, rudder, and lines known to work effectively with a good captain and crew.

Sometimes I hear educational leaders dismiss the importance of proven programs, saying that the only thing that matters is good leadership. But this is only half right. Great leadership is essential to make proven programs work, but proven, replicable programs and other infrastructure are equally essential to enable great leaders to have great results with kids.

So yes, recruit the best captains you can, and mentor them as much as possible. But give them, or enable them to acquire, sailboats known to work. Too many potentially great captains are given sailboats lacking a rudder or mast. When this happens, they’re sunk from the beginning.

Tailoring Evidence-Based Reform to Different Problems

Not long ago, I gave a speech at the American Psychological Association’s convention in Honolulu (all right, fighting for evidence-based reform does have its pleasures). Readers of this column will not be surprised at anything I said, but I got one question that provoked some thought. My questioner wanted to know why I kept referring to such easy-to-define-and-measure problems as ensuring that children can read or understand algebra, rather than much more complex problems such as how to lead schools.

I didn’t say it at the time, but I think this is both a silly and a profound question. The silly part is its implication that if evidence can’t solve all problems, it is of little value. In fact, is there anyone out there who thinks that it is not important to identify effective and replicable approaches to teaching reading, algebra, and all the other relatively easy-to-define, easy-to-measure problems of education?

Yet solving these does still leave some very important but less-well-defined problems that may take different approaches. These approaches should still be informed by evidence, but perhaps different types of evidence from the design-build-evaluate-disseminate model that usually leads to proven and replicable approaches to reading or algebra, if anything does.

Take leadership, which was my questioner’s example. I don’t think anyone will ever develop and evaluate a “principal protocol” for all schools, but it’s easy to imagine many innovations that could help principals be more effective, if they turned out to work in well-designed evaluations. If you break the principal’s role into its components, this is easy to see. For example, principals play a key role in teacher evaluation, and if anyone designs a teacher evaluation strategy that improves teacher performance overall, this should improve students’ performance, and that is easy to measure. Principals can play a key role in such schoolwide issues as attendance and behavior problems, and solutions for these exist and can readily be implemented and measured.

Principals play a leading role in managing resources, and the impact of each resource can have its own evidence base. Given a certain level of discretionary funding, should principals hire classroom aides, reduce class size, adopt particular programs, implement after-school programs, or purchase playground equipment? There is already evidence on most of these; the principal should be aware of this evidence and take it into account, and more evidence of this kind and better dissemination would be helpful.

Principals should be able to collaborate with staff to set goals, and then motivate and enable the staff to achieve the goals and monitor progress toward doing so. This is less cut-and-dried, but professional development for goal-setting and continuous progress toward targets is available, and the use of specific professional development models can be evaluated and replicated.

The kinds of leadership skills I suspect my questioner was thinking about are perhaps harder to measure and harder to influence. For example, positive relationships with staff, students, parents, and community leaders. Or the ability to make good decisions under pressure. Or the ability to communicate enthusiasm and high expectations for learning, or to serve as a positive role model and moral exemplar. Principals are probably more likely to learn these skills from observing their own principals and through other general life experience, but I would never rule out the possibility that professional development and coaching could build these skills as well. Research might also focus on identifying and selecting extraordinary leaders and keeping them on the job and growing in wisdom and capability over time.

Of course, if principals and school staffs chose and effectively implemented proven classroom programs, proven attendance and behavior programs, proven programs for struggling students, proven parent involvement programs, and so on, then the job of being principal would be a lot easier, and a lot more fulfilling. So rather than worrying about areas in which develop-evaluate-disseminate models don’t directly apply, it might be a good idea to use what we already know how to do, expand the range of proven approaches, and establish incentives and supports to create a school culture that respects and seeks proven solutions.

This may not solve every problem faced by every principal, but it would be a heck of a start, and then we could use different research and development methods to solve the remaining problems.

NOTE: You can obtain my APA address at http://www.youtube.com/watch?v=peiZVIJ0Gaw. I’m sorry you couldn’t be in Hawaii to hear it!

Follow Robert E. Slavin on Twitter: www.twitter.com/RobertSlavin

Lessons from Innovators: The National Forum to Accelerate Middle Grades Reform

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the Forum for Youth Investment, highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

This blog, based on an interview between the Forum for Youth Investment and Debby Kasak, director of the National Forum to Accelerate Middle Grades Reform, shares how school-to-school mentoring is both bringing about substantial improvements and itself serving as an important sustainability strategy.

Mentoring is Good for Mentors and Mentees
Coaches and mentors, whether at the individual or school level, can improve their own practice by helping others. That is what has begun to happen among schools involved in the National Forum’s i3 project. The National Forum is an alliance of educators, researchers, national associations, and officers of professional organizations and foundations committed to promoting the academic performance and healthy development of young adolescents. Through its Schools to Watch (STW) program, the National Forum has developed criteria for identifying high-performing middle-grades schools and created tools to help schools use them.

The National Forum’s i3 development grant is focused on improving 18 low-performing schools in three states using the STW framework and criteria. The goal is for those schools to learn from other STW schools that have been performing well. “We are inspiring schools to change their practice through whole school intervention,” says Kasak. “Each i3 school is matched up with demographically similar STW schools so they can see that it is possible to make change, even with a tough student population. It helps bring the theory to life for them. Given all the things that teachers get confronted with, they really respond when they see other teachers who are getting results.” But it isn’t just the low-performing schools and their teachers and administrators who are benefitting. “Successful schools can be powerful change agents in the lives of schools that need help, but interestingly, we’ve found that those mentor schools are improving their practice too,” reports Kasak. By helping others – coaching and sharing tools and strategies – schools and individuals within them are reminded to shore up their own promising practices.

Building Relationships is Key
“It sounds like a cliché, but one thing we have learned that can’t be underscored enough is that relationships matter,” Kasak shares. “The first six to seven months that we were involved in this project it was really important that we had coaches in the buildings who could form good relationships with teachers and principals. We needed to take the time to nurture those relationships. And as we did that, we saw the culture and climate of those schools changing.” Supportive relationships help schools weather the inevitable transitions that occur at the senior administration level. If teachers and coaches have a strong network and are committed to the work, it is less disruptive when a principal or superintendent leaves. A cadre of advocates for the initiative remains to educate new leaders. According to Kasak, that is exactly what happened in Chicago. “In Chicago, schools across the city are divided into networks. Originally, all of our schools were part of one network and we had a really supportive network leader. When the district administration changed, the networks were reorganized and our schools were no longer in the same network. One of our new network leaders wasn’t as supportive. But in one school, a teacher invited the Mayor to come visit the school; low and behold he did, and he brought the network leader with him. Seeing the school in action, hearing the teachers talk about their experiences, and building that relationship with the school staff made all the difference. He (the network leader) has been much more supportive since.”

Evaluation Can Be a Powerful Tool
Another way the National Forum has built relationships is through evaluation. Although it may sound counterintuitive, Kasak has found that working with the project evaluation team to look at what they are doing in a developmental way has helped them to share more information with schools than they might have otherwise and to build trust and commitment to the effort. “We are finding that in the second year we have gotten much better participation rates – almost 100% of the faculty in our 18 buildings – than we did the first year,” reports Kasak. In speculating why that might be, the National Forum came up with a couple of explanations. “In part, we know this has to do with being in our second year – teachers understand the process better. But we also credit our evaluation team. They regularly give data back to the schools which helps them better understand how all this work is impacting their school culture. Our evaluation team has really helped us to ask: Are doing what we said we would? Is it working? And how can we improve?”

Participating Schools are Part of the Sustainability Pipeline
The National Forum has an innovative approach to scaling their innovation and sustaining those schools where they are already working. Their two networks – low-performing schools supported through the i3 project and higher-performing Schools to Watch schools – create a natural pipeline toward STW status. The goal is to have all of their i3 schools eventually become STW schools who then mentor and support other low-performing schools that may receive funding through additional i3 funding or other sources down the road. Only three years into the i3 initiative, this pipeline is already in action in North Carolina. “We have one rural school in our i3 project that has just been terrific over the past several years. Recently, it applied to be a Schools to Watch school. They were evaluated and received a very high score, so were designated as a STW school. Now they are in a position to mentor other i3 schools in North Carolina. They benefit from the mentoring process itself, and then every three years will have to go through a re-designation process to maintain STW status, ensuring they are always on their game and thinking about how to get better. This school just went from being one site in a project to being part of a sustainable system of reform. We hope to do this with all of our i3 schools.”

Follow Robert E. Slavin on Twitter: www.twitter.com/RobertSlavin

Lessons from Innovators: Children’s Learning Initiative

Children’s Literacy Initiative

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the two foundations, will highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

Today’s post focuses on the Children’s Literacy Initiative (CLI). It is based on conversations between the Forum and CLI’s Executive Director, Kelly Hunter, on what it takes to maintain fidelity to a complex model in light of constant change in urban school districts. A summary of her comments is as follows.

Plan for change and stick to your core. School systems are in constant flux and developers must be prepared for instability. The Children’s Literacy Initiative (CLI) tries to do that by using training, coaching and other supports to promote quality teaching to ensure that students in low performing, urban districts are proficient readers, writers and thinkers by the end of third grade. They are currently attempting to scale their effort to four new districts, Camden, Chicago, Newark and Philadelphia. This is easier said than done. Such districts experience frequent teacher and administrator turnover, school closures and mergers, and charter formation. Hunter suggests that if you want to implement with fidelity you first have to take a long and hard look at your model, make decisions about what is core, and then message those core ingredients in a way that respects where schools are coming from. Hunter notes, “We realized that we were struggling with messaging our change model. Even though research shows quality teaching is the number one school factor, funders and others were focused on other reforms that are sexy today. We didn’t want to focus on being negative or bad mouthing other reforms. We just knew we had to be strong in our position, share the research, and stay clear about our message and core ingredients.”

Identify the right champions. Kelly and her partners at CLI have learned that regional superintendents are a critical ingredient for sustained change. These area leaders have considerable influence over principals. “At the beginning,” Hunter notes, “we would get central office and schools to sign off, but not the regional superintendents. Then we would be off and running but all of a sudden the regionals were messaging something different than what we were doing.” When regional leaders began to understand the importance of fidelity and appreciate the core ingredients, they were then able to share their enthusiasm with principals or set standards to reinforce values and practices consistent with the model.

Partner to multiply resources and minimize obstacles. As they push towards scale, leaders at CLI have also learned the importance of cultivating new and varied partnerships. In addition to district staff, especially important partners include local funders and other program providers. Local funders are essential from a sustainability standpoint. It is also critical to partner with other entities that provide related services or technical assistance within a building or district – even when they involve a different subject matter or grade. These partnerships can allow for more comprehensive and coherent supports across disciplines and grade levels and minimize confusion among and competing demands on district staff. “It’s about enhancing what we are doing, not changing it,” comments Hunter. For example, in one i3 school in West Philadelphia, Drexel University was providing coaching services in math while CLI was providing literacy coaching. By working together, they were able to make coaching across these topics more consistent and communication more streamlined.

Scale back to scale up. Implementing innovative practices is complicated and labor intensive. Regional knowledge is necessary to help align external needs and resources with your own organizations’ demands and capacities. Networking locally is a great way to learn about a school, community or district, and to identify key stakeholders, funders, and advocates. But building this knowledge and these relationships takes staff, time, and energy. To address this challenge, CLI revisited their initial plan and decided to concentrate energy and resources on implementing the model deeply in four cities rather than spread themselves thinly across ten. According to Hunter, “we knew that in some communities, we didn’t have enough local influence, networking and outreach to raise the dollars and implement the model with fidelity. We were chasing dollars and our model was being compromised. Ultimately that compromises student achievement.” Instead, she says, “over time we hope to build our presence in and around our four hubs and eventually serve as a model for other communities as they scale to surrounding schools and districts.”

Follow Robert E. Slavin on Twitter: www.twitter.com/RobertSlavin

Lessons from Innovators: Children’s Learning Initiative

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the two foundations, will highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

Follow Robert E. Slavin on Twitter: www.twitter.com/RobertSlavin