Glossary
Policy model — Glossary
In RL, the component that selects actions; in RLHF, the language model being optimized.
Full definition
The policy maps states (or prompts) to actions (or responses). RLHF updates the policy to score well under a reward model while limiting drift from a reference policy.