Skip to content
ai

PPO

Proximal Policy Optimization

Definition

PPO is a reinforcement learning algorithm used in RLHF to fine-tune language models based on reward signals from a reward model. It constrains policy updates to stay close to the previous policy (via a clipping objective), ensuring stable training.

PPO-based RLHF powered InstructGPT and ChatGPT, though its complexity has led to the adoption of simpler alternatives like DPO.

Can this ship? Are we exposed? Can you prove it?

Every CISO is asking these three questions. We answer them with facts, not guesses, from repository to runtime.