Paul Stapleton (Paul Stapleton is an independent researcher and editor. See more about Paul at his website academicproofreadingplus.com)
A few years ago, I read a newspaper article about a scientific study that found school pupils’ test scores were correlated with their breakfast-eating habits. Specifically, the researchers observed that pupils who ate breakfast had higher scores than those who had not.
Before I read any further, I stopped to think about this result and what the cause could be. Such thoughts about causation, of course, are natural to humans. If we get an upset stomach, we immediately wonder about the cause. Last night’s deep-fried dinner at a dai pai dong quickly comes to mind. Thinking about causation is almost as natural as breathing.
Regarding the case of the breakfast research, I continued reading the article about the study, and the cause for the correlation between the breakfast-eating habits and test scores quickly became apparent to me, and as I continued reading, my initial thoughts were confirmed. According to the researchers, pupils who ate breakfast had a full stomach which provided energy enabling them to concentrate on their learning, while those pupils who did not eat breakfast were hungry which distracted them from their learning, which in turn was reflected in their test scores. No brainer, right?
Wrong. Or at least not very satisfying.
This is the problem with noting a pattern and jumping to its most obvious cause, what Tversky and Kahneman (1974) termed System 1 thinking (work which partly contributed to Daniel Kahneman’s Nobel prize).
Instead, consider the following more satisfying answer. Let’s first consider the low test scores of a pupil we will call Bobby. He’s 10 years old and lives in a sub-divided flat in Mong Kok. His father disappeared when he was five, and his mother works two jobs starting at six in the morning…
You can fill in the rest of the story about Bobby and his breakfast. With a bit more System 2 thinking, i.e., deep thinking, you realize that the breakfast-test-scores correlation has much more to do with social conditions than it has to do with biology, i.e., not enough energy and low blood sugar levels. Poor Bobby not only doesn’t eat a proper breakfast, but he also doesn’t attend tutorial schools like the kids who do eat breakfast, nor do his parents help him with his homework, again unlike the kids who do eat breakfast. These differences between Bobby and his high scoring classmates provide a much fuller picture related to the social conditions that explain test scores. And this causative explanation would make the “breakfast angle” look pretty weak. In other words, there’s a much deeper story to tell, which is why I wrote to the newspaper after reading the article.
So what’s the connection between this story and conducting educational research, including at the doctoral level?
In short, as doctoral students begin preparing a thesis, they will probably have pre-conceived notions about the results and their causes. Then as data are collected, researchers notice patterns emerging which sometimes align with expectations, and sometimes do not. It is at this stage that researchers need to make concerted efforts to avoid simply using System 1 which gives easy answers and instead use System 2 to uncover the deeper causes related to the patterns that appear. The data that support pre-conceived beliefs are often seductive. This is called “confirmation bias” which requires careful reflection, although that’s not to imply that System 1 is always incorrect.
Inevitably, there will be data that do not fit into any of the emerging patterns, or worse, contradicts them. It may be tempting to dislike this data, ignore it, or even make it disappear. However, instead of viewing this nasty data as troublesome, it could be viewed as gold. Inconvenient data can sometimes trigger counterintuitive ideas and be a window into new ways of understanding while providing a compelling approach to tell a story. In essence, revealing deeper meanings is what characterizes good quality educational research.
Accordingly, let the data do the talking…and remember to always eat your breakfast.
References
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science, 1124-1131.
Further Reading
Stapleton, P. (2019). Avoiding cognitive biases: Promoting good decision making in research methods courses. Teaching in Higher Education 24(4), 578-586.
