Fine-tuning — Glossary

Fine-tuning starts from a model that already learned general representations during pretraining and updates parameters (or a subset of them) on a targeted dataset. Goals include domain adaptation, instruction following, style control, or improved accuracy on a benchmarked task.

Parameter-efficient methods such as adapters or low-rank updates can reduce cost. Fine-tuning can also degrade broader capabilities or introduce new failure modes if data is narrow or poorly labeled. Evaluation should check both the target task and regressions on important general behaviors.