The Art of Building Verifiers for Computer Use Agents
- Day
- Day 4 — Session Day 3 · Thu, Jul 2
- Time
- 11:40am-12:00pm
- Room
- Expo Stage 1 NE
- Track
- —
Sponsor talks on harness engineering as a core skill, agent memory vs. learning, verifiers, and relational context engines.
Accessible with the Expo Explorer pass and above.
About this session
Every team building browser agents has the same problem: you can't trust your own evals. Browser tasks are too open-ended for deterministic checks, so teams use LLM verifiers as judges, and the judges are wrong constantly. WebVoyager misses 45% of failures. WebJudge misses 22%. Used as RL reward, you're not training a better agent, you're training a more confident liar. This talk walks through the Universal Verifier, open-sourced with Microsoft Research: false positive rate near zero, Cohen's κ matching human-human agreement. Four design principles, one open benchmark, and an honest account of where auto-research worked and where it plateaued.
