GRPO vs PPO for Large Language Model Fine-Tuning
GRPO eliminates the critic model, cutting memory costs without sacrificing performance.
Devon Achterberg
Section
Post-Training Pipelines
6 stories in Post-Training Pipelines.
Catastrophic Forgetting in RL Fine-Tuned Language Models
Reward optimization in RL fine-tuning systematically erases previously learned capabilities.
Cassiel Oduya
Iterative Online RLHF With Continuously Updated Preference Data
Continuously updating preference data beats training reward models once on static data.
Marta Wierzbicka
Rejection Sampling Fine-Tuning as an RL Alternative
Rejection sampling matches reinforcement learning on math tasks without the complexity.
Devon Achterberg
Compute Budget Allocation Across RLHF Pipeline Stages
How to budget each RLHF stage separately before costs spiral.
Anjali Raghunathan
Multi-Stage Post-Training Order for Instruction-Following Models
The field is discovering that post-training stages must run in strict sequence, not interchangeably.
Devon Achterberg