Short answer
Fine-tuning updates a pretrained model using your dataset so outputs better match desired behavior. It complements—not always replaces—prompting and RAG.
Why it matters
Teams choose fine-tuning when they need consistent formats, domain tone, or specialized skills that prompting alone cannot stabilize.
Article
Decision guide
Fine-tune for style, tool patterns, and stable skills. Retrieve for changing facts. Measure regressions on safety and general quality, not only the specialty task.
Examples
JSON clerk
Fine-tune an open-weight model on 5k validated prompt→JSON pairs to stabilize extractions for invoices.
FAQ
Does fine-tuning update knowledge cutoffs?
It can inject patterns from your data, but it is not a complete substitute for retrieval of changing facts.
Is LoRA “real” fine-tuning?
Yes. It is parameter-efficient fine-tuning that often matches full updates for many specialization tasks.
Sources
-
LoRA: Low-Rank Adaptation of Large Language Models
ICLR / arXivParameter-efficient fine-tuning method (Hu et al.).
-
Training language models to follow instructions with human feedback
NeurIPS / arXivInstructGPT / RLHF paper (Ouyang et al.).
-
OpenAI API Documentation
OpenAIOfficial OpenAI platform documentation.
-
Hugging Face Documentation
Hugging FaceOpen model hub and Transformers library docs.