Educational Research – I don’t trust much of it. Here is why.

Recently, I had an opportunity to listen to a respected superintendent from Canada say something I have thought for a long time, “Much of the research in education is not replicable at scale.”  There are just too many variables to control when looking to replicate results across schools and communities that are vastly different from one another. When I hear about something that “works,” it often prompts me to dig deeper and try to assess similarities to our community of learners.  No program works equally well for every student – it is the creative work of skilled educators seeking to understand why and where an application or initiative fails or succeeds that is most valuable. Simply put, it is a process.

The other aspect of some educational research that is problematic is that the research often takes two correlated trends and presents it as one situation causing the other. The real explanation is usually much less exciting.  For example, consider the claim that students who take Advanced Placement courses in high school perform better in college. When looking at the research related to this topic, it is clear that students who take AP courses DO perform better in college.  However, taking AP courses in high school alone does not establish causality. Studies that merely find that students who are involved with the AP program in high school and subsequently perform better in college do not necessarily provide proof that the AP program caused the students to be successful in college. It is no surprise that the same motivated, hardworking, and academically advanced students who take AP classes in high school are still motivated, hardworking, successful students when they get to the university. So how can we know if it was the AP program that caused these students to do better in college?  To be clear, I am supportive of students taking a challenging course load aligned with their interests. If this means AP courses, great! However, many students that have not taken AP coursework in high school do very well in college, too.

Many of these research statistics that come from what is known as observational studies. In some cases, researchers have tried to use other factors like socioeconomic status to determine if they can see an impact.  This demographic approach is a good start to try to ascertain aspects of the student profile that could help inform us of issues related to student performance.

Moving educational research from correlation to causation would go a long way to eliminating hidden effects that prevent us from fully understanding the impact of our instructional decisions.  In the end, I feel that as educators – we have played a role in this push for “proof.” Learning and education is a human function. Trying to control it to the point of highly replicable results – while a noble goal – is just not realistic.  Focusing on the whole child and understanding each student’s unique needs is critical.