Pretraining — Glossary

Pretraining objectives such as next-token prediction or masked language modeling teach structure from massive corpora. Most organizations consume pretrained bases rather than training from scratch.

Later stages—instruction tuning, preference optimization, domain adaptation—specialize behavior for products.