Nevada Places Its Bets on Evidence

In Nevada, known as the land of big bets, taking risks is what they do. The Nevada State Department of Education (NDE) is showing this in its approach to ESSA evidence standards .  Of course, many states are planning policies to encourage use of programs that meet the ESSA evidence standards, but to my knowledge, no state department of education has taken as proactive a stance in this direction as Nevada.

 

Under the leadership of their state superintendent, Steve Canavero, Deputy Superintendent Brett Barley, and Director of the Office of Student and School Supports Seng-Dao Keo, Nevada has taken a strong stand: Evidence is essential for our schools, they maintain, because our kids deserve the best programs we can give them.

All states are asked by ESSA to require strong, moderate, or promising programs (defined in the law) for low-achieving schools seeking school improvement funding. Nevada has made it clear to its local districts that it will enforce the federal definitions rigorously, and only approve school improvement funding for schools proposing to implement proven programs appropriate to their needs. The federal ESSA law also provides bonus points on various other applications for federal funding, and Nevada will support these provisions as well.

However, Nevada will go beyond these policies, reasoning that if evidence from rigorous evaluations is good for federal funding, why shouldn’t it be good for state funding too? For example, Nevada will require ESSA-type evidence for its own funding program for very high-poverty schools, and for schools serving many English learners. The state has a reading-by-third-grade initiative that will also require use of programs proven to be effective under the ESSA regulations. For all of the discretionary programs offered by the state, NDE will create lists of ESSA-proven supplementary programs in each area in which evidence exists.

Nevada has even taken on the holy grail: Textbook adoption. It is not politically possible for the state to require that textbooks have rigorous evidence of effectiveness to be considered state approved. As in the past, texts will be state adopted if they align with state standards. However, on the state list of aligned programs, two key pieces of information will be added: the ESSA evidence level and the average effect size. Districts will not be required to take this information into account, but by listing it on the state adoption lists the state leaders hope to alert district leaders to pay attention to the evidence in making their selections of textbooks.

The Nevada focus on evidence takes courage. NDE has been deluged with concern from districts, from vendors, and from providers of professional development services. To each, NDE has made the same response: we need to move our state toward use of programs known to work. This is worth undergoing the difficult changes to new partnerships and new materials, if it provides Nevada’s children better programs, which will translate into better achievement and a chance at a better life. Seng-Dao Keo describes the evidence movement in Nevada as a moral imperative, delivering proven programs to Nevada’s children and then working to see that they are well implemented and actually produce the outcomes Nevada expects.

Perhaps other states are making similar plans. I certainly hope so, but it is heartening to see one state, at least, willing to use the ESSA standards as they were intended to be used, as a rationale for state and local educators not just to meet federal mandates, but to move toward use of proven programs. If other states also do this, it could drive publishers, software producers, and providers of professional development to invest in innovation and rigorous evaluation of promising approaches, as it increases use of approaches known to be effective now.

NDE is not just rolling the dice and hoping for the best. It is actively educating its district and school leaders on the benefits of evidence-based reform, and helping them make wise choices. With a proper focus on assessments of needs, facilitating access to information, and assistance with ensuring high quality implementation, really promoting use of proven programs should be more like Nevada’s Hoover Dam: A sure thing.

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.

Photo by: Michael Karavanov [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

Lessons from China

Recently I gave a series of speeches in China, organized by the Chinese University of Hong Kong and Nanjing Normal University. I had many wonderful and informative experiences, but one evening stood out.

I was in Nanjing, the ancient capital, and it was celebrating the weeks after the Chinese New Year. The center of the celebration was the Temple of Confucius. In and around it were lighted displays exhorting Chinese youth to excel on their exams. Children stood in front of these displays to have their pictures taken next to characters saying “first in class,” never second. A woman with a microphone recited blessings and hopes that students would do well on exams. After each one, students hit a huge drum with a long stick, as an indication of accepting the blessing. Inside the temple were thousands of small silk messages, bright red, expressing the wishes of parents and students that students will do well on their exams. Chinese friends explained what was going on, and told me how pervasive this spirit was. Children all know a saying to the effect that the path to riches and a beautiful wife was through books. I heard that perhaps 70% of urban Chinese students go to after-school cram schools to ensure their performance on exams.

The reason Chinese parents and students take test scores so seriously is obvious in every aspect of Chines culture. On an earlier trip to China I toured a beautiful house, from hundreds of years ago, in a big city. The only purpose of the house was to provide a place for young men of a large clan to stay while they prepared for their exams, which determined their place in the Confucian hierarchy.

As everyone knows, Chinese students do, in fact, do very well on their exams. I would note that these data come in particular from urban Eastern China, such as Shanghai. I’d heard about but did not fully understand policies that contribute to these outcomes. In all big cities in China, students can only attend schools in their city neighborhoods, where the best schools in the country are, if they were born there or own apartments. In a country where a small apartment in a big city can easily cost a half million dollars (U.S.), this is no small selection factor. If parents work in the city but do not own an apartment, their children may have to remain in the village or small city they came from, living with grandparents and attending non-elite schools. Chinese cities are growing so fast that the majority of their inhabitants come from the rest of China. This matters because admirers of Chinese education often cite the amazing statistics from the rich and growing Eastern Chinese cities, not the whole country. It’s as though the U.S. only reported test scores on international comparisons from suburbs in the Northeastern states from Maryland to New England, the wealthiest and highest-achieving part of our country.

I do not want to detract in any way from the educational achievements of the Chinese, but just to put it in context. First, the Chinese themselves have doubts about test scores as the only important indicators, and admire Western education for its broader focus. But just sticking to test scores, China and other Confucian cultures such as Japan, South Korea, and Singapore have been creating a culture valuing test scores since Confucius, about 2500 years ago. It would be a central focus of Chinese culture even if PISA and TIMSS did not exist to show it off to the world.

My only point is that when American or European observers hold up East Asian achievements as a goal to aspire to, these achievements do not exist in a cultural vacuum. Other countries can potentially achieve what China has achieved, in terms of test scores and other indicators, but they cannot achieve it in the same way. Western culture is just not going to spend the next 2500 years raising its children the way the Chinese do. What we can do, however, is to use our own strengths, in research, development, and dissemination, to progressively enhance educational outcomes. The Chinese can and will do this, too; that’s what I was doing traveling around China speaking about evidence-based reform. We need not be in competition with any nation or society, as expanding educational opportunity and success throughout the world is in the interests of everyone on Earth. But engaging in fantasies about how we can move ahead by emulating parts of Chinese culture that they have been refining since Confucius is not sensible.

Precisely because of their deep respect for scholarship and learning and their eagerness to continue to improve their educational achievements, the Chinese are ideal collaborators in the worldwide movement toward evidence-based reform in education. Colleagues at the Chinese University of Hong Kong and the Nanjing Normal University are launching Chinese-language and Asian-focused versions of our newsletter on evidence in education, Best Evidence in Brief (BEiB). We and our U.K. colleagues have been distributing BEIB for several years. We welcome the opportunity to share ideas and resources with our Chinese colleagues to enrich the evidence base for education for children everywhere.

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.

What if a Sears Catalogue Married Consumer Reports?

When I was in high school, I had a summer job delivering Sears catalogues. I borrowed my mother’s old Chevy station wagon and headed out fully laden into the wilds of the Maryland suburbs of Washington.

I immediately learned something surprising. I thought of a Sears catalogue as a big book of advertisements. But the people to whom I was delivering them often saw it as a book of dreams. They were excited to get their catalogues. When a neighborhood saw me coming, I became a minor celebrity.

Thinking back on those days, I was thinking about our Evidence for ESSA website (www.evidenceforessa.org). I realized that what I wanted it to be was a way to communicate to educators the wonderful array of programs they could use to improve outcomes for their children. Sort of like a Sears catalogue for education. However, it provides something that a Sears catalogue does not: Evidence about the effectiveness of each catalogue entry. Imagine a Sears catalogue that was married to Consumer Reports. Where a traditional Sears catalogue describes a kitchen gadget, “It slices and dices, with no muss, no fuss!”, the marriage with Consumer Reports would instead say, “Effective at slicing and dicing, but lots of muss. Also fuss.”

If this marriage took place, it might take some of the fun out of the Sears catalogue (making it a book of realities rather than a book of dreams), but it would give confidence to buyers, and help them make wise choices. And with proper wordsmithing, it could still communicate both enthusiasm, when warranted, and truth. But even more, it could have a huge impact on the producers of consumer goods, because they would know that their products would need to be rigorously tested and found to be able to back up their claims.

In enhancing the impact of research on the practice of education, we have two problems that have to be solved. Just like the “Book of Dreams,” we have to help educators know the wonderful array of programs available to them, programs they may never had heard of. And beyond the particular programs, we need to build excitement about the opportunity to select among proven programs.

In education, we make choices not for ourselves, but on behalf of our children. Responsible educators want to choose programs and practices that improve the achievement of their students. Something like a marriage of the Sears catalogue and Consumer Reports is necessary to address educators’ dreams and their need for information on program outcomes. Users should be both excited and informed. Information usually does not excite. Excitement usually does not inform. We need a way to do both.

In Evidence for ESSA, we have tried to give educators a sense that there are many solutions to enduring instructional problems (excitement), and descriptions of programs, outcomes, costs, staffing requirements, professional development, and effects for particular subgroups, for example (information).

In contrast to Sears catalogues, Evidence for ESSA is light (Sears catalogues were huge, and ultimately broke the springs on my mother’s station wagon). In contrast to Consumer Reports, Evidence for ESSA is free.  Every marriage has its problems, but our hope is that we can capture the excitement and the information from the marriage of these two approaches.

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.

Picture source: Nationaal Archief, the Netherlands

 

How Classroom-Invented Innovations Can Have Broad Impacts

When I was in high school, I had an after-school job at a small electronics company that made and sold equipment, mostly to the U.S. Navy. My job was to work with another high school student and our foreman to pack and unpack boxes, do inventories, basically whatever needed doing.

One of our regular tasks was very time-consuming. We had to test solder extractors to be sure they were working. We’d have to heat up each one for several minutes, touch a bit of solder to it, and wipe off any residue.

One day, my fellow high school student and I came up with an idea. We took 20 solder extractors and lined them up on a work table with 20 electrical outlets. We then plugged them in. By the time we’d plugged in #20, #1 was hot, so we could go back and test it, then #2, and so on. An hour-long job was reduced to 10 minutes. We were being paid the princely sum of $1.40 an hour, so we were saving the company big bucks. Our foreman immediately saw the advantages, and he told the main office about our idea.

Up in the main office, far from the warehouse, was a mean, mean man. He wore a permanent scowl. He had a car with mean, mean bumper stickers. I’ll call him Mr. Meanie.

Mr. Meanie hated everyone, but he especially hated the goofy, college-bound high school students in the warehouse. So he had to come see what we were doing, probably to prove that it was dumb idea.

Mr. Meanie came and asked me to show him the solder extractors. I laid them out, same as always, and everything worked, same as always, but due to my anxiety under Mr. Meanie’s scowl, I let one of the cords touch its neighboring solder extractor. It was ruined.

Mr. Meanie looked satisfied (probably thinking, “I knew it was a dumb idea”), and left without a word. But as long as I worked at the company, we never again tested solder extractors one at a time (and never scorched another cord). My guess is that long after we were gone, our method remained in use despite Mr. Meanie. We’d overcome him with evidence that no one could dispute.

In education, we employ some of the smartest and most capable people anywhere as teachers. Teachers innovate, and many of their innovations undoubtedly improve their own students’ outcomes. Yet because most teachers work alone, their innovations rarely spread or stick even within their own schools. When I was a special education teacher long ago, I made up and tested out many innovations for my very diverse, very disabled students. Before heading off for graduate school, I wrote them out in detail for whoever was going to receive my students the following year. Perhaps their next teachers received and paid attention to my notes, but probably not, and they could not have had much impact for very long. More broadly, there is just no mechanism for identifying and testing out teachers’ innovations and then disseminating them to others, so they have little impact beyond the teacher and perhaps his or her colleagues and student teachers, at best.

One place in the education firmament where teacher-level innovation is encouraged, noted, and routinely disseminated is in comprehensive schoolwide approaches, such as our own Success for All (SFA). Because SFA has its own definite structure and materials, promising innovations in any school or classroom may immediately apply to the roughly 1000 schools we work with across the U.S. Because SFA schools have facilitators within each school and coaches from the Success for All Foundation who regularly visit in teachers’ classes, there are many opportunities for teachers to propose innovations and show them off. Those that seem most promising may be incorporated in the national SFA program, or at least mentioned as alternatives in ongoing coaching.

As one small example, SFA constantly has students take turns reading to each other. There used to be arguments and confusion about who goes first. A teacher in Washington, DC noticed this and invented a solution. She appointed one student in each dyad to be a “peanut butter” and the other to be a “jelly.” Then she’d say, “Today, let’s start with the jellies,” and the students started right away without confusion or argument. Now, 1000 schools use this method.

A University of Michigan professor, Don Peurach, studied this very aspect of Success for All and wrote a book about it, called Seeing Complexity in Public Education (Oxford University Press, 2011). He visited dozens of SFA schools, SFA conferences, and professional development sessions, and interviewed hundreds of participants. What he described is an enterprise engaged in sharing evidence-proven practices with schools and at the same time learning from innovations and problem solutions devised in schools and communicating best practices back out to the whole network.

I’m sure that other school improvement networks do the same, because it just makes sense. If you have a school network with common values, goals, approaches, and techniques, how does it keep getting better over time if it does not learn from those who are on the front lines? I’d expect that such very diverse networks as Montessori and Waldorf schools, KIPP and Success Academy, and School Development Program and Expeditionary Learning schools, must do the same. Each of the improvements and innovations contributed by teachers or principals may not be big enough to move the needle on achievement outcomes by themselves, but collectively they keep programs moving forward as learning organizations, solving problems and improving outcomes.

In education, we have to overcome our share of Mr. Meanies trying to keep us from innovating or evaluating promising approaches. Yet we can overcome blockers and doubters if we work together to progressively improve proven programs. We can overwhelm the Mr. Meanies with evidence that no one can dispute.

This blog was developed with support from the Laura and John Arnold Foundation. The views expressed here do not necessarily reflect those of the Foundation.