The race to build computer-using AI agents has created a quieter problem: deciding whether an agent actually finished the job. A research preprint posted July 30 introduces OSReward, a benchmark designed to test the vision-language models increasingly used to grade agent trajectories. Its central finding is uncomfortable for a field that depends on automated evaluation: many judges are biased toward declaring success, even when the recorded run ended in failure.
A trajectory can include screenshots, mouse and keyboard actions, intermediate states, and the agent’s own reasoning. Reviewing that record by hand is slow, while custom verification scripts cover only tasks anticipated by their authors. Model-based judges promise scale, but OSReward asks whether substituting one AI for the human evaluator produces a trustworthy score.
A benchmark for the evaluator
The research team assembled 1,019 human-gold trajectories spanning web, desktop, mobile, and operating-system environments. Tasks ranged from routine chores to longer professional workflows and ran for as many as 100 steps. The labels went through multiple reviewers, with disputed cases escalated and low-quality examples removed. The paper says the annotation and re-verification work required roughly 800 human hours.
That full set is accompanied by OSReward-Hard, a 284-trajectory challenge subset concentrated on cases that divided human annotators, and OSReward-Multi, which adds efficiency and instruction-alignment ratings to successful runs. The distinction matters because a judge can appear reliable on obvious cases while failing precisely where evaluation is most valuable.
Across the main benchmark, some leading judges approached 90 percent binary accuracy. Performance fell sharply on the hard subset, where the best judge dropped below 70 percent. The dominant error was not excessive skepticism. It was leniency: failed executions were accepted as successes. The paper links part of that bias to judges trusting an agent’s confident narrative instead of grounding the verdict in what the screenshots and actions actually show.
Analysis: the scoreboard can exaggerate progress
TENS analysis: The consequential result is not that one evaluator ranks above another. It is that an agent benchmark can become optimistic when its judge shares the agent’s tendency to produce a plausible account of success. If a failed run is scored as complete, the error can flow into leaderboards, training data, reinforcement signals, and product claims. A measurement shortcut can therefore create the appearance of capability growth without any improvement in task completion.
The earlier OSWorld benchmark illustrates why this problem is difficult. It introduced 369 tasks across real web and desktop applications, each paired with setup instructions and a custom execution-based verifier. That design made evaluation reproducible, but creating a specialized verifier for every open-ended workflow does not scale easily. OSReward targets the gap between the reliability of those task-specific checks and the flexibility of a general model judge.
TENS analysis: OSReward also separates two questions that agent demos often collapse: did the task finish, and was it completed well? A trajectory can reach the requested state through unnecessary, risky, or misaligned actions. Binary success is necessary, but efficiency and instruction alignment become separate quality dimensions once agents operate across files, accounts, and applications.
An open judge, with limits
The researchers also released OS-Shepherd-100K, a corpus of nearly 100,000 reasoning-annotated judgments distilled from a larger pool using agreement filters. They trained 9-billion-parameter and 35-billion-parameter reward models on that data. According to the paper, the smaller model catches 57.6 percent of false successes in OSReward-Hard and can reduce evaluation cost by roughly 30 to 60 times compared with the frontier-model labeling setup used in the study.
Those numbers do not establish a universal evaluator. OSReward is labeled a work in progress, and its results come from the selected tasks, agent backbones, model versions, prompting protocol, and cost assumptions. Agreement filtering can also preserve errors shared by several teachers. The authors test transfer against human-written verifiers from OSWorld, WebArena, and AndroidWorld, but a learned judge should not be treated as ground truth merely because it is cheaper and more balanced than its peers.
TENS analysis: The practical path is a verification hierarchy rather than a single automatic referee. Deterministic checks should remain the standard where task completion can be measured directly. Model judges can cover open-ended cases, while difficult, high-impact, or disputed runs should escalate to human review. The key metric is not only average judge accuracy, but false-success recall: how often the evaluator catches an agent that looks convincing yet did not finish.
Computer-use agents will become easier to trust only if their scorekeepers become harder to impress. OSReward provides evidence that the evaluation layer is now a capability bottleneck of its own. The next advance may come not from an agent that clicks faster, but from a verifier that can reliably say when those clicks accomplished nothing.
Sources: OSReward research preprint; OSWorld research paper and benchmark.
Featured photograph: SpaceX mission control in Hawthorne, California. Credit: SpaceX via Wikimedia Commons. License: CC0 1.0. Modification: none.
