Real software engineering tasks are not asking a model to write a function, but long tasks (o(hours) reading unfamiliar code, running tools, recovering from wrong turns, and knowing when the job is actually done. In this keynote, Melvin Johnson (VP, Google DeepMind) covers how we post-trained for long-horizon agentic coding:
- Hard tasks. Popular benchmarks saturate, so we built internal realistic, multi-step software tasks.
- Graders that can't be gamed. When agents learn to satisfy unit tests without solving the problem, you need graders that check the work the way a reviewer would.
- Model behavior. An agent that says "fixed it" when it didn't is worse than no agent. We reward models for reporting accurately what they did and didn't do.
- Efficiency over long horizons. Finishing more of the task with fewer tokens and stable behavior across very long contexts.
- Shipping weekly. Modular improvements and dogfooding with Google engineers let us release in weeks instead of quarters.