Short answer
Fine-tuning updates a pretrained model using your dataset so outputs better match desired behavior. It complements—not always replaces—prompting and RAG.
Why it matters
Teams choose fine-tuning when they need consistent formats, domain tone, or specialized skills that prompting alone cannot stabilize.
Article
When it helps
If prompting and retrieval are not enough for consistent format or domain language, fine-tuning can specialize behavior—at the cost of building and maintaining a dataset.
Examples
JSON clerk
Fine-tune an open-weight model on 5k validated prompt→JSON pairs to stabilize extractions for invoices.
FAQ
Does fine-tuning update knowledge cutoffs?
It can inject patterns from your data, but it is not a complete substitute for retrieval of changing facts.
Is LoRA “real” fine-tuning?
Yes. It is parameter-efficient fine-tuning that often matches full updates for many specialization tasks.
Sources
-
LoRA: Low-Rank Adaptation of Large Language Models
ICLR / arXivParameter-efficient fine-tuning method (Hu et al.).
-
Training language models to follow instructions with human feedback
NeurIPS / arXivInstructGPT / RLHF paper (Ouyang et al.).
-
OpenAI API Documentation
OpenAIOfficial OpenAI platform documentation.
-
Hugging Face Documentation
Hugging FaceOpen model hub and Transformers library docs.