Calibration of Learned Reward Models Against Human Ratings
Miscalibrated reward models exploit gaps between proxy scores and actual human preference.
Anjali Raghunathan
Section
Reward Model Engineering
8 stories in Reward Model Engineering.
Reward Model Checkpoint Selection Criteria
Picking your best checkpoint means watching for reward hacking, not just picking the highest score.
Anjali Raghunathan
Process Reward Models for Multi-Step Reasoning
Step-level reward models catch flawed reasoning that correct final answers hide.
Devon Achterberg
Pointwise vs Pairwise vs Listwise Reward Training Objectives
Choosing the right training objective shapes how well reward models learn from human feedback.
Cassiel Oduya
Length Bias in Reward Model Training
Small biases in human feedback compound into verbose AI through reinforcement learning loops.
Anjali Raghunathan
Reward Model Generalization Across Out-of-Distribution Prompts
Reward models collapse on out-of-distribution tasks, leaving alignment systems vulnerable to gaming.
Devon Achterberg
Verifiable Reward Design for Code Generation Tasks
Code's executability enables richer reward signals than human judgment can provide.
Marta Wierzbicka
Reward Model Ensembling for Uncertainty Estimation
Using multiple reward models to quantify and act on training uncertainty.
Cassiel Oduya