The Robo-Ump Test Run

7.1 Percentage Points of Correctness

MLB's Automated Ball-Strike system (ABS) has been tested in minor league games since 2022 and in select spring training and exhibition contexts since 2024. Across 184,000 tracked pitches in ABS-equipped games, the system called 99.4% of pitches correctly based on the Statcast-defined strike zone. Human umpires, tracked over the same period in non-ABS games, called 92.3% correctly.

The 7.1-point gap sounds small. It is not. At 300 pitches per game, 7.1% translates to roughly 21 incorrect calls per game under human umpires. In ABS games, the number drops to 2. Those 19 additional correct calls per game affect at-bat outcomes, pitch counts, baserunners, and scoring. Over a 162-game season, the accumulated accuracy difference changes hundreds of plate appearance outcomes.

The errors are not random. Human umpires have documented biases: they expand the zone with two strikes, shrink it on 3-0 counts, favor veteran pitchers, and call more strikes when the home team is pitching. ABS eliminates all of these. The zone is the zone, every pitch, every count, every inning.

Pitchers Stopped Painting Corners

The first unexpected consequence of ABS appeared within weeks of its minor league deployment. Pitchers began throwing fewer pitches on the edges of the zone. In human-umpired games, 34.2% of pitches targeted the corners (within 1 inch of the zone boundary). In ABS games, that number dropped to 27.8%. Pitchers moved their targets inward by an average of 1.3 inches.

The logic is straightforward. With a human umpire, a pitch 0.5 inches off the plate has a 40-60% chance of being called a strike. That uncertainty makes the corner a profitable target. With ABS, the same pitch is a ball 100% of the time. The expected value of painting the corner collapsed. Pitchers adapted by moving their targets to locations where they could be confident of getting strike calls.

Walk rates increased by 0.8 percentage points in ABS games despite the system being more accurate. Pitchers who relied on borderline calls to pad their strike counts found themselves behind in more at-bats. They threw more pitches in the zone to compensate, and hitters punished those pitches. The strikeout rate dropped by 1.2 percentage points. Batting average rose by 8 points.

Pitch Design Under a Perfect Umpire

When the zone is enforced perfectly, pitch design changes. The value of a pitch that crosses the corner at 99 mph drops if it misses the zone by half an inch 40% of the time. Under human umpires, those misses were often called strikes. Under ABS, they are always balls. The premium shifts toward pitches with high in-zone effectiveness rather than pitches that work on the edges.

The runs-per-game increase (4.52 to 4.81) is the number that gets attention. An extra 0.29 runs per game means roughly 47 more runs across a full season. That shifts the competitive balance toward offenses and away from the framing catchers and corner-painting pitchers who benefited most from human umpire variability.

A $20 Million Skill Becomes Worthless

Pitch framing is the art of a catcher receiving a borderline pitch in a way that makes the umpire more likely to call it a strike. The best framers in baseball are worth 15-25 runs per season above average. At roughly $9 million per win, elite framing is worth $15-25 million annually. Teams pay catchers largely based on this skill.

ABS eliminates framing value entirely. The system tracks the ball through the zone using Doppler radar and optical tracking. How the catcher receives the pitch is irrelevant. In ABS games, the correlation between framing metrics and called strike rate drops to zero. A catcher who was worth 20 runs for his framing in a human-umpired game is worth 0 framing runs in an ABS game.

This has enormous implications for the catcher market. Teams that invested in elite framers (the Dodgers, Astros, and Braves have been the most aggressive framing buyers) would see the return on that investment vanish overnight. Catchers whose primary value was framing would need to justify their salaries through hitting, game-calling, and blocking alone. Several current starting catchers would become replacement-level players without their framing value.

The Compromise Nobody Loves

MLB has tested two ABS formats. The full ABS model uses the system for every pitch with no human override. The challenge model keeps a human umpire calling balls and strikes but gives each team 3 challenges per game to appeal to the ABS system. The challenge model has received more institutional support because it preserves the umpire's visible role while adding a correction mechanism.

The challenge model produces mixed results. Teams use their challenges strategically, saving them for high-leverage situations. In tested games, 78% of challenges overturned the human umpire's call. That means teams were overwhelmingly correct in identifying missed calls. But the 3-challenge limit means roughly 15 incorrect calls per game still stand because teams run out of challenges or choose not to use them on lower-leverage pitches.

The full ABS model eliminates the problem entirely but removes the human element that baseball has marketed as part of the game's character. The debate between accuracy and tradition is the central tension holding back ABS adoption. The data strongly favors the full model. The culture strongly favors the compromise.

When the Robot Arrives

Full ABS deployment at the MLB level is widely expected by 2027 or 2028. The technology is ready. The testing data supports it. The remaining barriers are institutional: the umpires' union, broadcast partners who value the drama of contested calls, and a faction of team owners who worry about alienating traditionalist fans.

The umpires' union has the strongest leverage. ABS does not eliminate umpire jobs (someone still needs to manage the game, call plays at the plate, and handle ejections), but it reduces the umpire's most visible role to a ceremonial function. Negotiations over this transition will likely be the final bottleneck.

When ABS arrives, the adjustments will be immediate. Pitchers will retrain their targeting. Catchers will be re-valued. Walk rates will spike for the first season as pitchers adjust. Then the game will settle into a new equilibrium where the zone is the zone, every pitch, every time. The question was never whether ABS would come to baseball. It was always whether baseball would be ready for what comes next.

Sources & Data

Jesse Walker

Jesse Walker writes about baseball through data. He played outfield in high school, found his real position behind a spreadsheet, and hasn't stopped building models since.