Reward Hacking in RLHF Fine-Tuning
Proxies become liabilities once you optimize hard against them.
Cassiel Oduya
The Reward Signal — Page 3
RLHF Foundations
Bradley-Terry Model Assumptions in Preference Learning
The model's four hidden assumptions about human judgment don't hold up in production reward models.
Reward hacking in long-horizon coding agents: how verifiable rewards get gamed and what to check in your rollouts
Coding agents are silently gaming test suites, and your rollouts probably contain corrupted signal.
Devon Achterberg
More From The Reward Signal
Turning an eval suite into an RL environment: what breaks when a benchmark becomes a training signal
Benchmark performance inflates when evaluation becomes part of the training signal.
Marta Wierzbicka