What Makes Educational Technology Programs Work?

2015-01-29-HP59_01_29_15.jpg

While everyone else is having a lot more fun, my colleagues and I sit up late at night writing a free website, the Best Evidence Encyclopedia (www.bestevidence.org), which reviews evaluations of educational programs in reading, math, and science.

The recent reports reinforce an observation I’ve made previously. When programs are found to have little or no impact on student learning, it is often the case that they provide very little professional development to teachers. Giving teachers lots of professional development does not guarantee positive effects, but failing to do so seems to virtually guarantee disappointing impacts.

This observation takes on new importance as technology comes to play an increasing role in educational innovation. Numerous high-quality studies of traditional computer-assisted instruction programs, in which students walk down the hall or to the back of the classroom to work on technology largely disconnected from teachers’ instruction, find few positive effects on learning. Many technology applications appearing in schools today have learned nothing from this sad history and are offering free or low-cost apps that students work on individually, with little professional development for teachers or even any connection to their (non-technology) lessons. In light of the prior research, it would be astonishing if these apps made any difference in student learning, no matter how appealing or well-designed they are.

Alongside the thousands of free apps going into schools, there has also developed an entirely different approach to technology, one that integrates technology with teacher lessons and provides teachers with extensive professional development and coaching. Studies of such programs do find significant positive effects. As one example, I recently saw an evaluation of a reading and math program called Time to Know. In Time to Know, teachers use computers and their own non-computer lessons to start a lesson. Students then do activities on their individual devices, personalized to their needs and learning histories. Student learning is continuously assessed and fed back to the teacher to use in informing further lessons and guiding interventions with individual students.

Time to Know provides teachers with significant professional development and coaching, so they can use it flexibly and effectively. Perhaps as a result, the program showed very good outcomes in a small but high-quality study, with an effect size of +0.32 in reading and +0.29 in math.

There are many other studies of classroom programs that improve student learning, in particular studies of forms of cooperative learning in many subjects and grade levels. As a group, the outcomes reported in these studies are always far higher than those seen in studies of traditional technology applications, in all subjects and grade levels. What is interesting about the study of Time to Know is that here is an unusually positive outcome for a technology application in a rigorous experiment. What is unique about the intervention is that it embeds technology in the classroom and provides teachers with extensive PD. Perhaps classroom-embedded technology with adequate professional development is the wave of the future, and perhaps it will finally achieve the long-awaited breakthroughs that technology has been promising for the past 40 years.

Advertisements

How Universal Access to Technology Could Advance Evidence-Based Reform. Or Not.

2014-11-20-HP4211_20_2014.jpg

Since the early 1960s (at least), breakthroughs in education caused by advances in technology have been confidently predicted. First it was teaching machines, then mainframes, then laptops, then video disks, then interactive whiteboards, and now blended and flipped learning. Sadly, however, each innovation of the past has ended up making little if any difference in student achievement. I remain hopeful that this time, technology could produce breakthroughs, if the new capabilities of technology are used to create systematically enhanced environments for learning, new approaches to teaching based on new technologies are rigorously evaluated, and approaches found to be successful are broadly disseminated.

My reason for hope, this time around, lies in the fact that schools are rapidly moving toward providing universal access to tablets or other relatively low-cost digital devices. This is a potential game-changer, as universal access makes it possible for teachers to give digital assignments to all students. It also makes it possible for developers to create replicable strategies that make optimal use of personalized instruction, simulations, visual media, games, sophisticated real-time assessments, links to other students within and beyond the classroom, links to prescreened and curated information, and so on. If students also have compatible technology at home, this adds the possibility of integration of homework and classwork (which is essential for blended and flipped learning, for example, but also for simpler means of making homework engaging, game-like, and useful for learning).

All of these possibilities are only potentials, not actualities, and given the long, sad history of technology in schools, they may well not take place. A lot of the applications of universal access to digital devices common today are merely reinventions of computer-assisted instruction (CAI), which has a particularly poor research record. Other uses are for poorly designed project-based learning or for applications that do little more than make traditional teaching a little easier for teachers. There are hundreds of applications available for every possible classroom use, but the quality of these applications varies widely, and they do not readily integrate with each other or with other instruction or standards. Hardworking, tech-savvy teachers can in principle assemble fabulous lessons, but this is difficult to do on a large scale.

Before universal access to technology can transform education, a great deal of creative work needs to be done to make and evaluate courses or major portions of courses using the new technology opportunities. Imagine developers, researchers, nonprofits, and for-profits partnering with experienced teachers to create astonishing, integrated, and complete approaches to, say, beginning reading, elementary math and science, secondary algebra, or high school physics. In each case, programs would be rigorously evaluated, and then disseminated if found to be effective.

There are many applications of universal access technologies that make me optimistic. For example, current or on-the-horizon technologies could enhance teachers’ abilities to teach traditional lessons. Prepared lessons might incorporate visual media, games, or simulations in initial teaching. They might use computer-facilitated cooperative learning with embedded assessments and feedback to replace worksheets. They might embed assessments in games, simulations, and cooperative activities to replace formative and summative assessments. Simulations of lab experiments could make inquiry-oriented instruction in science and math far more common. Access to curated, age-appropriate libraries of information could transform social studies and science. Computerized assessments of writing, including creative writing as well as grammar, punctuation, and spelling could help students working with peers to become more effective writers.

In each of these cases, extensive development, piloting, and evaluation will be necessary, but once created and found to be effective, digitally enhanced models will be extremely popular, and their costs will decline with scale.

Even as technology’s past should make us wary of unsupported claims and premature enthusiasm, the future can be different. In all areas of technology other than education, someone creates a new product, finds it to be effective, and then makes it available for widespread adoption. A time of tinkering yields to a time of solid accomplishment. This can happen in education, too. With adequate support for R&D, breakthroughs are likely, and when they happen in any area, they increase the possibilities of breakthroughs in other areas.

Sooner or later, technology will help students learn far more than they do today. The technology models ready to go today do not yet have the evidence base to justify a lot of optimism, but in the age of universal access, we’ve only just begun.