Ka Ya Lee, Harvard University
Which metrics should we use to assess institutions, such as schools or a national education system? I argue that this is ultimately a question of values, and thus, the deeper question is this: What values should guide our assessment endeavors? Once we reconfigure the question of metrics as a question of values, then we can understand what is lacking in the discourse about educational measurement: voices from philosophers of education.
C. Thi Nguyen (2024) introduces the concept of value capture, a phenomenon in which a stakeholder of an activity internalizes (often quantified) metrics as a guiding value of their activity. A runner, who aims to improve her lung capacity, would use metrics, such as VO2 max (measuring one’s maximal oxygen consumption), which come to dictate her activity of running by prompting her to organize the training plan around the goal of maximizing VO2 max.
Value capture takes place at any scale. Individuals can be swayed by certain metrics related to their goals, whereas medium-scale organizations, such as schools, are also susceptible to value capture. In a culture that sets assessment scores as the sole metrics of educational success, student assessment shapes students’ identity itself (Nieminen, 2024). At the school-level, teachers and school administrators treat test scores as the central metric around which the majority of their teaching practices revolve, which Daniel Koretz (2017) calls the “Testing Charade.” When it comes to a large-scale example, education policymakers set PISA scores as the metrics that guide the nation’s education policy (e.g. Tamir, 2023).
As these cases demonstrate, value capture permeates the education sector. The obsession with PISA scores and other kinds of standardized test scores (e.g., the National Assessment of Educational Progress in the United States) and its dominant effects on education policy and practices are well-documented. The core feature of value capture is that metrics function not just as metrics but as action-guiding values as Nguyen (2024) writes, “Value capture is when an externally sourced value, like a metric, comes to dominate your practical reasoning.”
In response to using standardized test scores as the metric and hence the guiding value in education, some international and national organizations started proposing alternative metrics. For instance, UNESCO (2018) has proposed measuring equity in education. One can appreciate these efforts to replace or supplement the existing standardized testing paradigm as a quest for better metrics. The concept of value capture, however, reveals that the issue lies deeper. That is, whether we measure educational equity or educational outcomes is not merely a matter of metrics, but it is about values: what values should we internalize and allow to guide our understandings of the success of educational institutions? In other words, “what should we measure?” is ultimately a question of “what should we value?”
Translating the issue of metrics in educational measurement into values also highlights something that is missing in these developing conversations: insights from philosophers of education, who are trained to bring precision and clarity to normative issues that cannot be resolved by empirical evidence alone. In describing the inception of OECD’s well-being measure, for instance, Rappleye et al. (2020) note, “Comprised of about 20 scholars from North American and Europe, mostly statisticians, assessment specialists, and psychologists, the group sought to identify the normative assumptions about society and individuals that would underpin the OECD’s key competencies.” Rappleye et al. (2020) criticize the lack of representation from non-Western countries, but what is also salient is the absence of philosophers, whose very job is to think about values in a clear and precise manner. Thus, philosophy of education should matter in dialogues about educational measurement.
If I am correct in arguing that philosophers of education should be included in the process of reconfiguring educational measurement, what are the areas in which they can provide clarity and precision? I conclude by suggesting two lines of inquiry that philosophers of education are well-qualified to undertake. The first group of questions is what I call the “learning of what?” question. When measuring equity in learning (UNESCO, 2018), what areas of learning should we talk about? The standardized testing paradigm limits the measurement of learning to mostly reading and mathematics — and sometimes science and history — but is that sufficient? The second line of inquiry is the “opportunity for what?” question. When measuring equal opportunity (UNESCO, 2018), what should we set as the normative goal of opportunity distributions? Should it be something broad, such as opportunity for flourishing (e.g. Schouten, 2023), or something narrower, such as opportunity for educational achievement? I dream of the day when philosophers of education join the table with other decision-makers in educational measurement.
