Curiosity is bliss    Archive    TIL    Tools    Search    RSS    About

Julien Couvreur's programming blog and more

Adversarial evaluation for long-running agents

 

In Build Agents That Run for Hours (Without Losing the Plot), Ash Prabaker and Andrew Wilson described a practical pattern for long-running coding agents: split planning, building, and evaluation into separate roles, and make the evaluator adversarial.

Takeaways:

  • Self-evaluation is a trap. Use an adversarial evaluator.
  • Compaction doesn’t cure coherence drift. Structured handoffs do.
  • Make subjective quality gradable with rubrics the model can apply.
  • Read the traces. They’re your primary debugging loop.
  • Delete scaffolding when the model catches up. The frontier moves.