by Michael HartneyThe 74
This story appeared first 74A non-profit news site covering education. Sign up for the free newsletter from The 74 Get more like this in your inbox.
America’s pastime is back, but a new twist. This season, Major League Baseball introduced an automated ball-strike challenge system that allows players to challenge umpire decisions in real time and overturn clearly erroneous calls.
The change has been highly controversial. Purists see umpiring as more art than science and worry that technology robs the game of its human element, making umpires almost-impossible to retain. value. Proponents counter that ABS encourages data-driven decisions that justify and improve Accountability.
If this debate sounds familiar to those in K-12 education, it should.
More than a decade ago, the reformer That tried to fix a broken teacher evaluation system failed A rarely used measure of actual effectiveness to distinguish between high and low performers and to deploy, reward, and retain teacher talent. Most controversially, reformers adopted new technologies known as “value-added” measures of teacher performance that were ultimately abandoned after political resistance. suspicious use
In many ways, the ABS system of grading umpires provides a useful lens for reconsidering what teacher evaluation reforms got right, where they went wrong, and what reformers and critics missed.
Five lessons are different.
First, start with where value-added useful information is provided—and where it is not. These measures were not equally informative for all teachers. Instead, they were strongest at the extremes, where the signal is largest. They were only available to a subset of teachers, since only certain subjects and grade levels are tested. As Corey Koedel explainsValue-added measures are most useful for identifying the highest- and lowest-performing teachers, while distinguishing the middle is much more difficult.
Baseball developed an evaluation system based on that insight. ABS is not trying to make every call correct or perfectly umpires from first to worst. Rather, it is designed to catch the most obvious mistakes, consistently identifying poor umpiring.
Consider the controversial umpire CB Buckner D Information show him Repeatedly Miss clear callThe stakes are too high to ignore. ABS is designed to recognize this and the same reasoning applies to teacher evaluation. Even if value-added measures cannot rank teachers perfectly, they can identify clear cases where students are being consistently shortchanged with ineffective instructors. Reformers sometimes pushed these measures too far, but critics were quick to dismiss data that clearly meant something in some contexts.
A second lesson comes from how unions responded to moments of reform. There are only two basic ways to evaluate performance: subjective judgment or objective measures. Value-adding was an incomplete attempt to introduce more objective information into a system long dominated by subjective evaluations. It was not a defense of good evaluation to reject it without offering a meaningful alternative with tangible consequences for poor performance. It was a refusal to fully evaluate.
This shift is particularly striking in contrast to how unions approached the issue in baseball. MLB umpires are unitedAnd their association did not completely block ABS. Instead, the union agreed to a hybrid system that preserves their roles while using new data to ensure umpiring excellence.
There was a time when teachers-union leaders said the same thing. as Albert Shankar Once said, unions should be “willing to identify excellence and not just be concerned with protecting jobs and protecting due process.” But this mindset often fails to take root. After the National Education Association briefly showed some openness to assessment reform in 2011, it opposite Of course, this holds that “standardized tests, even if deemed valid and reliable, cannot be used to support an employment action against a teacher.” Then NEA-President Lily Eskelsen Garcia went so far accused Value-added as “mark of Satan”.
Some local unions have adopted peer review as an alternative: teachers evaluating other teachers. In principle, this makes sense. A Study Cincinnati’s evaluation system, which relies heavily on peer observation, found that teachers became more effective after the evaluation. But that’s a different question from whether systems meaningfully differentiate performance or weed out persistent underperformers at scale. There, the evidence is less clear and appraisal systems designed to support improvement are not always appropriate for making high-stakes personnel decisions.
Third, measurement is only important if it carries consequences. In baseball, it does. Umpires are graded using performance data and those evaluations directly affect playoff game assignments (bonuses). In other words, measurement is not symbolic, it is directly tied to results. In education, that link is often missing. Many districts still rely on last-in, first-out rules that remove new teachers first, regardless of their classroom performance. Here again baseball provides a clear example of why it misses the mark. When the analyst comparison With retired veteran umpires replaced by younger teams, the younger team was significantly more accurate.
Fourth, these debates remind us of the lengths to which self-interested actors will go to protect the status quo when reform threatens their career interests. consider complaint Veteran pitcher Walker Buehler said veteran players should get a more generous strike zone. The logic is familiar. In education, similar arguments are made to distance experienced teachers from performance-based decisions. In both cases, the argument is less about being right than about protecting incumbents.
Ultimately, this system is about getting it right for the people. In baseball, the league operates primarily under a fiduciary obligation to the integrity of the game. That is, there is a duty to act in the best interests of the game, which includes striving for fairness and accuracy in how the game is played. As a recent Analysis Put that way, MLB’s governing rules and “best interest of baseball” authority create a fiduciary-like duty to minimize preventable errors that could undermine confidence in the outcome. That obligation has practical implications. It doesn’t disappear because new technology makes it uncomfortable for workers or changes how work is done. If better tools can mitigate the obvious flaws, ignoring them risks undermining the game itself
Education rarely works this way. Student education is often considered one goal among many, balanced against it Adult interest. Courts have even refused to recognize the fundamental right to effective education as recently as 2014 Vergara v. California But if schools exist for anything, they are exist To educate students. That should be the north star. Assessment systems are not about satisfying adults. They ensure that students are not consistently shortchanged.
Baseball has a better way of showing that. Imperfect tools can still improve decision making when used where they are most powerful. The question is not whether one measure captures everything. Whether it helps us avoid the most obvious mistakes.
–
This is the story is produced by 74A nonprofit, independent news organization focused on education in America.
***
If you believe in the work we’re doing at The Good Men Project, please join us as a Premium Member today.
All premium members can watch The Good Men Project without any ads.
Need more information? A full list of benefits is here.
Photo credit: splash
