Little Sleepers: Long-Term Effects of Preschool

In education research, a “sleeper effect” is not a way to get all of your preschoolers to take naps. Instead, it is an outcome of a program that appears not immediately after the end of the program, but some time afterwards, usually a year or more. For example, the mother of all sleeper effects was the Perry Preschool study, which found positive outcomes at the end of preschool but no differences throughout elementary school. Then positive follow-up outcomes began to show up on a variety of important measures in high school and beyond.

Sleeper effects are very rare in education research. To see why, consider a study of a math program for third graders that found no differences between program and control students at the end of third grade, but then a large and significant difference popped up in fourth grade or later. Long-term effects of effective programs are often seen, but how can there be long-term effects if there are no short-term effects on the way? Sleeper effects are so rare that many early childhood researchers have serious doubts about the validity of the long-term Perry Preschool findings.

I was thinking about sleeper effects recently because we have recently added preschool studies to our Evidence for ESSA website. In reviewing the key studies, I was once again reading an extraordinary 2009 study by Mark Lipsey and Dale Farran.

The study randomly assigned Head Start classes in rural Tennessee to one of three conditions. Some were assigned to use a program called Bright Beginnings, which had a strong pre-literacy focus. Some were assigned to use Creative Curriculum, a popular constructive/developmental curriculum with little emphasis on literacy. The remainder were assigned to a control group, in which teachers used whatever methods they ordinarily used.

Note that this design is different from the usual preschool studies frequently reported in the newspaper, which compare preschool to no preschool. In this study, all students were in preschool. What differed is only how they were taught.

The results immediately after the preschool program were not astonishing. Bright Beginnings students scored best on literacy and language measures (average effect size = +0.21 for literacy, +0.11 for language), though the differences were not significant at the school level. There were no differences at all between Creative Curriculum and control schools.

Where the outcomes became interesting was in the later years. Ordinarily in education research, outcomes measured after the treatments have finished diminish over time. In the Bright Beginnings/Creative Curriculum study the outcomes were measured again when students were in third grade, four years after they left school. Most students could be located because the test was the Tennessee standardized test, so scores could be found as long as students were still in Tennessee schools.

On third grade reading, former Bright Beginnings students now scored significantly better than former controls, and the difference was statistically significant and substantial (effect size = +0.27).

In a review of early childhood programs at www.bestevidence.org, our team found that across 16 programs emphasizing literacy as well as language, effect sizes did not diminish in literacy at the end of kindergarten, and they actually doubled on language measures (from +0.08 in preschool to +0.15 in kindergarten).

If sleeper effects (or at least maintenance on follow-up) are so rare in education research, why did they appear in these studies of preschool? There are several possibilities.

The most likely explanation is that it is difficult to measure outcomes among four year-olds. They can be squirrely and inconsistent. If a pre-kindergarten program had a true and substantial impact on children’s literacy or language, measures at the end of preschool may not detect it as well as measures a year later, because kindergartners and kindergarten skills are easier to measure.

Whatever the reason, the evidence suggests that effects of particular preschool approaches may show up later than the end of preschool. This observation, and specifically the Bright Beginnings evaluation, may indicate that in the long run it matters a great deal how students are taught in preschool. Until we find replicable models of preschool, or pre-k to 3 interventions, that have long-term effects on reading and other outcomes, we cannot sleep. Our little sleepers are counting on us to ensure them a positive future.

This blog is sponsored by the Laura and John Arnold Foundation

Advertisements

High-Reliability Organizations

I’m writing this blog from the inside of an airplane high above the Atlantic. I have total confidence that my plane will deliver me safely to Europe. It’s astonishing. The people who run every aspect of this plane are ordinary folk. I knew a guy in college who spent his entire career as a pilot for the very airline I’m flying today. He was competent, smart, and very, very careful. But he was not expected to make things up as he went along. He liked to repeat an old saying: “There are old pilots and there are bold pilots, but there are no old, bold pilots.”

When I was younger, I recall that airplane crashes were relatively common. These were always prominently reported in the news. But today, airplane disasters not caused by terrorists or crazy people are extremely rare. The reason is that air disasters are so catastrophic that airlines have adopted procedures in every aspect of their operation to ensure that planes arrive safely at their destinations. Every system important to safety is checked and rechecked, with technology and humans backing each other up. I happen to have a nephew who is studying to be an aircraft mechanic. His course is extremely rigorous. Most people don’t make it through. His final test, he says, will have 80 questions. The minimum acceptable score: 80. His brother is a nuclear engineer on a navy submarine. Same kind of training, same requirement for success. No room for error. The need for such care in airplanes and submarines is obvious. But why not in education?

My friend and colleague Sam Stringfield had this idea many years ago. Based on it, he and a Welsh colleague, David Reynolds, created what they called “high-reliability schools.” They evaluated them in Wales, and found substantially greater gains in schools using this approach than in control schools.

Despite its success, the high-reliability idea did not catch hold in education. Yet any student who is unnecessarily failing in school is a catastrophe waiting to happen. You don’t need a lot of data tables to be convinced that students not reading well by third grade are headed for big trouble. They are disproportionately likely to end up in special education, to repeat one or more grades, to drop out of high school, and to get into behavioral difficulties and problems with the law. Each of these outcomes is hugely damaging to the student and hugely expensive to the taxpayer.

Yet there is no problem in all of education that is better researched than early reading failure. There are many proven strategies known to greatly reduce reading failure: whole school methods, small group, individual tutoring, technology, and more. Our Evidence for ESSA web site lists dozens of proven approaches. It is probably already the case that any school could identify students at risk of reading failure in kindergarten or first grade and then apply proven, easily available methods conscientiously to ensure that virtually every child will succeed in reading.

The point here is that if we wanted to, we could treat early reading the way airlines and submarines treat safety, as a life or death issue.

If schools accepted the high-reliability challenge for early reading, here is what they would do. First, they’d adopt proven pre-reading programs for pre-kindergarten, and then proven beginning reading programs for grades K-3. Teachers of these grades would receive extensive professional development and then in-class coaching to help them use these proven strategies as well as they were used in the research that validated them, or better.

Starting in kindergarten, we’d start to assess students in early reading skills, so we’d know which students need assistance in which specific skills. We’d continue to assess all students over time to be sure that all are on a path to success. The assessments would include vision and hearing so that problems in these areas are solved.

Each school would have staff trained and equipped to provide an array of services for students who are in need of additional help. These would include small-group tutoring for students with mild problems, and one-to-one tutoring for more serious problems. Multiple proven programs, each focusing on distinct problems, would be ready to deploy for students who need them. Students who need eyeglasses, hearing accommodations, or other health assistance would be treated. Students who are English learners would receive assistance with language and reading.

The point is, each school would be committed to ensuring the success of every child, and would be prepared to do so. Like my high-reliability nephews, the goal of every person in every school would be zero failures. Not just fewer. Zero.

There is no question that this goal could be accomplished. The only issue is whether it could be accomplished at a cost that would be politically acceptable. My guess is that a full-scale, replicable schoolwide strategy to ensure zero reading failures in high-poverty schools could add about $200 per child per year, from grades pre-K to 3. A lot of money, you say? Recall from a previous blog that the average per-pupil cost in the U.S. is approximately $11,000. What if it were $11,200, just for a few years? The near-term savings in special education and retentions, much less longer-term costs of delinquency and dropout, would more than return this investment.

But more than cost-effectiveness, there is a moral imperative here. Failing children who could succeed is simply wrong. We could greatly reduce or eliminate this problem, just as the aircraft industry has done. Our society must come to see school failure as the catastrophe that it is, and to use whatever proven methods are needed to make reading failure a problem of the past.

This blog is sponsored by the Laura and John Arnold Foundation

Whadja Do In School Today?

Every parent of a four or five year old knows the drill. Your child comes home after pre-kindergarten or kindergarten. “Whadja do today?” you  say with eager anticipation, thinking of all the friends your child must have made, the stories your child heard, the songs your child sang, the projects or dress-up or phonics or math, or…well, anything.

“Nuffin,” your child says, wandering out of range to avoid further interrogation.

You know your child did a whole lot more than “nuffin.” But how can you find out so that you can build on what the teacher did each day?

One answer is something our group at the Success for All Foundation created utilizing Investing in Innovation (i3) funding with partners at Sesame Workshop, Sirius Thinking, and Johns Hopkins University. We call it Home Links. Home Links are 10-15 minute videos, akin to short television shows, that parents and children watch together, 4 evenings a week. Each show uses content from Sesame Street and animations we have made with Sirius Thinking, so they are a bit like Sesame Street shows themselves, with one huge difference: the content of the shows reflects the activities that children and teachers were doing that day in school.

The Home Links give kids reinforcement and extension of vocabulary and skills they learned that day, and that’s important. But more important, they tell parents what’s happening in school. When a show contains skits about fall, the letter V, counting to five, and singing traditional songs, the parents know that all of these things are happening in school. Our surveys found that 96% of the time, a parent, grandparent, or other relative watches with the child. At the end of each show there is music and movement, and parents tell us they dance with their children, and they love the closeness and fun. But parents also now know how to support their children’s learning. If the topic is markets, they know to point out interesting things when they next are at the market with their child. If the letter is T, they know to point out things that begin with T. If the math segment is on shapes, parents know to ask children about shapes they see in daily life. Home should not be another classroom, but it’s the ideal place for a child to learn that the things he or she is learning in school are important to his or her parents and exist in his or her community. It also helps children understand that knowing about and learning about those things brings pride and builds curiosity.

Home Links are sent home on DVDs each day. We are now looking for funding to make an online version so families can download Home Links to digital devices such as phones and tablets.

Right now, Home Links are being used in approximately 300 preschool and kindergarten classes already using our proven Success for All whole-school approach. In the future, we hope to disseminate Home Links to preschools and kindergartens whether or not they use Success for All.

When this happens, more and more parents won’t have to ask, “Whadja do in school today?” They’ll know. And they’ll know how to build on what they find out.

And that ain’t nuffin’.

 

The Investing in Innovation (i3) program is a federal competitive grant program at the U.S. Department of Education, within the Office of Innovation and Improvement (OII). It provides funding to support local education agencies or nonprofit organizations in partnership with LEAs and/or schools to expand and develop innovative practices that can serve as models of best practices and to identify and document best practices that can be shared and taken to scale in the areas of improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

More information on the i3 program can be found here.

More information on Success for All Foundation’s grant Around the Corner: A Technology-Enhanced Approach to Early Literacy can be found here.

Those Flying Finns: Is it Saunas or Reading That Make the Difference?

I recently attended a conference in Stockholm, at which there were several Finns and a lot of discussion about the “Finnish Miracle,” in which Finland was found to score at the top on PISA (Program for International Student Assessment). PISA periodically tests representative samples of fifteen year olds in math, science, and reading.

The Finnish Miracle became apparent in 2001, and has been talked to death since then. Much about what I heard at the conference was familiar. Finland is a small, homogeneous country in which teaching is an honored profession. I heard that Helsinki, the capital, hires 80 teachers a year, and gets thousands of applicants. Maybe these factors are all we need to know.

However, I heard something else that I knew but had forgotten.

Ten years before the Finnish PISA Miracle, there was an international test of reading, called the IEA Reading Literacy Study, which tested the reading skills of students ages 9 to 10 in 30 countries. The U.S. scored second on this test, behind- you guessed it – Finland. I looked it up, and discovered that the difference was huge. Finnish children scored 31% of a standard deviation ahead of the U. S.

A (Swedish) speaker at my conference, Jan-Eric Gustafsson, brought up the earlier IEA Reading Literacy study. He explained that throughout the 1980s, Finland had a relentless policy of ensuring that every child learned to read in the early grades. If they needed it, struggling students were given one-to-one tutoring focused on phonics as long as necessary to ensure success in this crucial subject. In light of their focus on early reading success, the outcomes on the IEA reading tests are more comprehensible.

Now space forward to the PISA tests reported 2001. The fifteen year olds who took the test were, of course, subject to the Finnish reading policy throughout their elementary years. Not only reading, but also math and science, are surely influenced by success in elementary reading.

It’s possible that Finland’s success in reading in the 1990s was equally a product of outstanding and honored teachers, a homogeneous society, and other factors (though these were also true in other Nordic countries that did not score nearly as well). Perhaps Finns eat a lot more smoked fish or spend a lot more time in saunas than other people, and these explain academic success. But it must be at least a partial explanation of Finland’s reading success that they focused substantial resources over a long time period on reading for all. In turn, their students’ success on PISA must be at least partially a result of their earlier success in reading.

One reason this all matters to the U.S. and other non-Finnish countries is that while we cannot all become Finns, we can ensure that virtually every child learns to read confidently and capably by third grade.

Many states have “Reading by Third Grade” laws that threaten to hold back third graders if they are not reading at grade level, and usually provide a last-chance summer school course to avert retention. Neither of these strategies (retention and last-chance summer school) have evidence of effectiveness. In contrast, I noted in a recent blog that there were 24 elementary programs for struggling readers that have strong, moderate, or promising evidence of effectiveness according to ESSA evidence standards.

The 24 programs were proven in our own country, and most have been widely and successfully applied. There is plenty of rationale for using these programs no matter what the Finns are doing or have done in the past. But if one of our goals is to keep up with or surpass our economic competitors in terms of education, to produce a capable workforce able to deal with complex problems of all kinds, then we need to provide our children with top-quality reading programs in the first place and effective support for struggling readers. It would be expensive to do this, perhaps, but certainly much cheaper than providing smoked fish and saunas to every U. S. family!

Early Childhood Education in the Balance

Back in the day, a kindergarten was a garden for children, a place where children could play, sing, paint, and pretend. Letters, numbers, and anything that smacked of formal schooling was minimized. Instead, kindergarten was intended to facilitate the transition from home to school, in a home-like setting.

Today, of course, kindergarten is less of a garden and more of a hothouse. At least in public schools, it’s a rare kindergarten that does not have a strong focus on letters and numbers. A child exposed only to the play-oriented children’s garden of old would arrive in first grade at a serious disadvantage. In most kindergarten classes there is still plenty of play, singing, and make-believe, but also a lot of literacy and numeracy.

Debate in early childhood education has largely shifted from the kindergarten to the pre-kindergarten. For a long time, programs for four-year-olds have resembled kindergartens of the past. Children are painting, playing with blocks, dressing up for make-believe, using sand and water tables, singing, and listening to stories.

In most states, pre-K is not available to all, and many children who attend pre-K do so as part of the federal Head Start program. A lot of attention has been paid to the question, “Does Head Start work?” For decades, the evidence that it does has depended on longitudinal studies of the Perry Preschool, the Abecedarian Project, and other small, colossally funded experimental approaches. However, evaluations of run-of-the-mill Head Start programs find a consistent and depressing pattern. Immediately after their Head Start experience, young children perform somewhat better on cognitive measures than do similar children who did not receive pre-K services at all, but within a year or two these differences fade away.

Seeing these outcomes, early childhood researchers began in the 1990s to experiment with ways to make Head Start and other early childhood approaches more effective. Numerous studies compared outcomes for children who were all in preschool, but who received different programs. In the mid-2000s, a large federal project called the Preschool Curriculum Evaluation Research (PCER) initiative evaluated a large number of programs using consistent, rigorous methods. This study added substantially to the number and quality of studies of preschool models of all kinds.

My colleagues Bette Chambers, Alan Cheung, and I have just completed a review of research on studies that compared alternative approaches to pre-K. We found 32 studies of 22 programs that met our standards. These studies were of exceptional quality; 30 of the studies involved random assignment to conditions. We mainly compared programs with elements focused on literacy (which we called “balanced” approaches) to those that did not have such elements (“developmental” approaches). The outcomes were striking. At the end of pre-K, children in the balanced programs performed better, on average, on both literacy and language measures. The literacy outcomes were not too surprising, because the balanced programs had a stronger emphasis on literacy. However, at the end of kindergarten, the children who had been in the balanced groups still performed at a higher level on both literacy and language measures.

Our review supports the idea that young children can benefit from literacy experiences, to learn letters and sounds, while they continue to play, pretend, draw, and sing. Keeping literacy out of the mix does not benefit children immediately or one year later.

I’d be the last person to want to take the garden out of kindergarten or preschool. Pre-K can still be fun, social, and interactive. But adding in a focus on literacy helps children arrive in first grade ready to succeed in reading. How can that be a bad thing?

Happy 50th Birthday, Head Start!

This week marks the 50th anniversary of Head Start! Children, please look up from your sand tables and dress-up corners and finger-painting tables and sing “Happy Birthday”! Gloria, I’m watching you. Do not even think of putting finger paint in Michael’s hair! Is everyone ready?

Of all the Great Society programs, Head Start is perhaps the most popular. It provides center-based services to millions of very cute 3- and 4-year-olds, mostly children from disadvantaged families. If members of the public, educators, and policy makers know a single conclusion from educational research, it is that early-childhood programs have substantial and long-term positive impacts. As one consequence of this understanding, President Obama and his administration have pushed hard to expand Head Start and other early-childhood programs to serve many more children.

Does Head Start work? Well, it all depends on what you mean by “Head Start” and what you mean by “work.” Some highly enriched early-childhood models, such as the famous Perry Preschool and the Abecedarian program, do have evidence of long-term gains in important life outcomes not enjoyed by similar children who did not attend preschool at all. However, longitudinal evaluations of run-of-the-mill Head Start programs find limited impacts into kindergarten and few detectable differences afterwards. As a result of these evaluations, early-childhood experts are now careful to say that “high-quality” early-childhood or Head Start programs work, leaving undefined exactly what they mean by “high-quality.”

As more children enroll in Head Start and other early-childhood programs, the question has moved from “Does it help disadvantaged children attend early-childhood programs?” to “What kinds of early-childhood programs should be provided to disadvantaged children?” Here there is ferocious debate. On one side are traditionalists who insist that early-childhood programs emphasize play, imagination, listening to stories, singing, and crafts, for example, but not phonemic awareness or other pre-reading skills. Such “developmental” programs are designed to ease children into the school in a home-like setting and build children’s language, school skills, and general orientation toward learning.

On the other side there are some educators who believe that disadvantaged children in pre-kindergarten need to be taught like first graders to ingrain the school readiness, literacy, and math skills necessary to succeed in school. In the middle are “balanced” programs, advocated by those who believe that in addition to play, exploration, imagination, and language, it is beneficial to expose children to phonemic awareness, phonics, and other pre-reading as well as early math skills, on the principle that it is important to give disadvantaged preschoolers a, well, “head start” on the skills that will soon determine their success in school. Preschool should not look like third grade, with kids in rows answering questions and doing worksheets. (In fact, third grade should not look like this, either.) Phonemic awareness and phonics in balanced programs are typically introduced in preschool using rhymes, games, songs, and exploratory activities to learn the sounds and shapes of letters. As phonics and math have definitively pushed their way into kindergarten, traditionalists hold onto preschool as the last bastion of child-centered education, while advocates of balanced programs argue that children need to be prepared for the settings in which they will soon find themselves.

My colleagues and I are carrying out a review of research on the literacy and language outcomes of different approaches to early-childhood education. (Watch for it in the Best Evidence Encyclopedia in the next month or so.) Our findings are very interesting. We focused on studies that compared children in “developmental” programs with those in “balanced” programs, which are ones that include most elements of developmental programs but also include direct teaching of phonemic awareness and phonics. Not surprisingly, both in preschool and on kindergarten follow-up measures, children who participated in balanced programs performed much better on assessments of early reading skills. Perhaps more surprisingly, these children also performed better than those in the developmental programs on measures of language, in preschool and on kindergarten follow-up.

What our findings suggest is that teaching phonics and phonemic awareness in preschool is beneficial for reading and, far from undermining children’s language development, also enhances performance in this arena.

It is probably unrealistic to expect that one year of quality early childhood will turn around a child’s life forever, but seen as the beginning of a progression from high-quality preschool into high-quality kindergarten into high-quality elementary and secondary school, preschool is very important. Our review supports the idea that a portion of the precious time preschool teachers have with young children can be devoted to building pre-reading skills without harming language development, and in fact contributing to overall performance.

And that’s a head start worth celebrating!

Lessons from Innovators: Children’s Learning Initiative

 

Children’s Literacy Initiative2013-02-19-HP4Image21813.jpg

The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The W. T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the two foundations, will highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform.

Today’s post focuses on the Children’s Literacy Initiative (CLI). It is based on conversations between the Forum and CLI’s Executive Director, Kelly Hunter, on what it takes to maintain fidelity to a complex model in light of constant change in urban school districts. A summary of her comments is as follows.

Plan for change and stick to your core. School systems are in constant flux and developers must be prepared for instability. The Children’s Literacy Initiative (CLI) tries to do that by using training, coaching and other supports to promote quality teaching to ensure that students in low performing, urban districts are proficient readers, writers and thinkers by the end of third grade. They are currently attempting to scale their effort to four new districts, Camden, Chicago, Newark and Philadelphia. This is easier said than done. Such districts experience frequent teacher and administrator turnover, school closures and mergers, and charter formation. Hunter suggests that if you want to implement with fidelity you first have to take a long and hard look at your model, make decisions about what is core, and then message those core ingredients in a way that respects where schools are coming from. Hunter notes, “We realized that we were struggling with messaging our change model. Even though research shows quality teaching is the number one school factor, funders and others were focused on other reforms that are sexy today. We didn’t want to focus on being negative or bad mouthing other reforms. We just knew we had to be strong in our position, share the research, and stay clear about our message and core ingredients.”

Identify the right champions. Kelly and her partners at CLI have learned that regional superintendents are a critical ingredient for sustained change. These area leaders have considerable influence over principals. “At the beginning,” Hunter notes, “we would get central office and schools to sign off, but not the regional superintendents. Then we would be off and running but all of a sudden the regionals were messaging something different than what we were doing.” When regional leaders began to understand the importance of fidelity and appreciate the core ingredients, they were then able to share their enthusiasm with principals or set standards to reinforce values and practices consistent with the model.

Partner to multiply resources and minimize obstacles. As they push towards scale, leaders at CLI have also learned the importance of cultivating new and varied partnerships. In addition to district staff, especially important partners include local funders and other program providers. Local funders are essential from a sustainability standpoint. It is also critical to partner with other entities that provide related services or technical assistance within a building or district – even when they involve a different subject matter or grade. These partnerships can allow for more comprehensive and coherent supports across disciplines and grade levels and minimize confusion among and competing demands on district staff. “It’s about enhancing what we are doing, not changing it,” comments Hunter. For example, in one i3 school in West Philadelphia, Drexel University was providing coaching services in math while CLI was providing literacy coaching. By working together, they were able to make coaching across these topics more consistent and communication more streamlined.

Scale back to scale up. Implementing innovative practices is complicated and labor intensive. Regional knowledge is necessary to help align external needs and resources with your own organizations’ demands and capacities. Networking locally is a great way to learn about a school, community or district, and to identify key stakeholders, funders, and advocates. But building this knowledge and these relationships takes staff, time, and energy. To address this challenge, CLI revisited their initial plan and decided to concentrate energy and resources on implementing the model deeply in four cities rather than spread themselves thinly across ten. According to Hunter, “we knew that in some communities, we didn’t have enough local influence, networking and outreach to raise the dollars and implement the model with fidelity. We were chasing dollars and our model was being compromised. Ultimately that compromises student achievement.” Instead, she says, “over time we hope to build our presence in and around our four hubs and eventually serve as a model for other communities as they scale to surrounding schools and districts.”