A big reason that many educators don’t like to participate in experiments is that they don’t want to take a 50-50 chance of being assigned to a control group. That is, in randomized experiments, schools or teachers or students are often assigned at random to receive the innovative treatment (the experimental group) or to keep doing whatever they were doing (the control group). Educators often object, complaining that they do not think it is fair that they might be assigned to the control group.
This objection always comes up, and I’m always surprised by it. What the educators who are so concerned about being in the control group don’t seem to realize is that they are already – for all intents and purposes – in the control group, but we’re giving them the potential opportunity to receive the new services and move into the experimental group. If the coin comes up heads, they move to the experimental group; if the coin comes up tails, they simply continue doing what they’ve already been comfortably doing, perhaps for many years. Usually, schools can purchase materials and training to adopt an innovative program outside of the experiment; all the experiment typically offers is a 50-50 chance to get the treatment for free, or at a deep discount. So if schools want the treatment, and are willing to pay for it, they can get it. Ending up in the control group is not so bad, either. Incentives are usually offered to schools in the control group, so a school might receive several thousand dollars to do anything they want other than the experimental treatment. Further, many studies use a “delayed treatment” design in which the control group gets training and materials to implement the experimental program after the study is over, a year or two after the schools in the experimental group received the program. In this way, having been in the control group in the short term serves to improve the school in the long run.
But isn’t it unethical to deprive schools or children of effective methods? If the methods were so proven and so widely used that not to use them would truly deprive students, then this would be unethical. But every experiment has to be passed by an Institutional Review Board (IRB), usually located in a university. IRB regulations require that the control group receive a treatment that is at least “state of the art,” so that no one gets less than the current standard of best practice. The experiment is designed to test out innovations whose effectiveness has not yet been established and that are not yet standard best practice.
In fields that respect evidence, yesterday’s experimental treatment becomes the standard of practice, and thereby becomes the minimum acceptable offering for control groups. This cycle continues indefinitely, with experimental treatments being progressively compared to harder-to-beat control treatments as evidence-based practice advances. In other words, doing experiments using control groups to improve education based on evidence would put education into a virtuous cycle of innovation, evaluation, and progressive improvement like that which has transformed fields such as medicine, agriculture, and technology to the benefit of all.
Most educators would prefer not to be in the control group, but they should at least be consoled by the knowledge that control groups play an essential role in evidence-based reform. In fact, the importance of knowing whether or not new methods add to student outcomes is so great that one could argue that it is unethical not to agree to participate in experiments in which one might be assigned to the control group. In an education system offering many more opportunities to participate in research, individual schools or educators may be in control groups in some studies and experimental groups in others.
As teachers, principals, and superintendents get used to participating in experiments, they are losing some of their earlier reluctance. In particular, when educators are asked to play an active role in developing and evaluating new approaches and in choosing which experiments to volunteer for, they become more comfortable with the concept. And this comfort will enable education to join medicine, agriculture, technology, and other fields in which a respect for evidence and innovation drives rapid progress in meeting human needs.
If you like evidence-based reform in education, then be sure to tip your hat to the little-appreciated control group, without which most experiments would not be possible.