Short answer
Fine-tuning updates a pretrained model using your dataset so outputs better match desired behavior. It complements—not always replaces—prompting and RAG.
Why it matters
Teams choose fine-tuning when they need consistent formats, domain tone, or specialized skills that prompting alone cannot stabilize.
Article
Pipeline
Curate gold examples, freeze a validation set, tune adapters, run regression evals, and deploy behind a model ID. Prefer PEFT when iterating quickly.
Examples
JSON clerk
Fine-tune an open-weight model on 5k validated prompt→JSON pairs to stabilize extractions for invoices.
FAQ
Does fine-tuning update knowledge cutoffs?
It can inject patterns from your data, but it is not a complete substitute for retrieval of changing facts.
Is LoRA “real” fine-tuning?
Yes. It is parameter-efficient fine-tuning that often matches full updates for many specialization tasks.
Sources
-
LoRA: Low-Rank Adaptation of Large Language Models
ICLR / arXivParameter-efficient fine-tuning method (Hu et al.).
-
Training language models to follow instructions with human feedback
NeurIPS / arXivInstructGPT / RLHF paper (Ouyang et al.).
-
OpenAI API Documentation
OpenAIOfficial OpenAI platform documentation.
-
Hugging Face Documentation
Hugging FaceOpen model hub and Transformers library docs.