Catching Reward Hacking From the Inside

Congrats! Your agent passed every test. But did it fix the bug in your code, or did it just learn how to cheat its way through the tests?
The July incident where OpenAI’s agents escaped their sandbox and breached Hugging Face made unaligned agentic behavior a mainstream headline. Reward hacking, eval awareness and cyber misuse are difficult to spot with black box methods, like output filters and LLM judge monitors. As a result, unaligned behavior can go undetected.
At Goodfire, we build monitors that read model’s internal activations. Interpretability lets us find representations tied to behaviors like reward hacking, awareness of being evaluated, and offensive cyber activity, then build into probes that fire while the model is still generating a response. That means we can sometimes catch a model heading toward bad behavior before it even happens. In this talk, I’ll walk through how we deploy activation monitors production and share lessons from real deployments with frontier labs and inference providers. I’ll share a recent example of how our monitors caught a coding agent building a seemingly innocuous test program well before the agent ultimately used it to cheat, and how our platform saves on compute by monitoring activations the model is already generating. I’ll also walk through how activation monitoring fits into a larger, multi-layer defense system and how to set the right thresholds for intervention.