Short answer
Fine-tuning updates a pretrained model using your dataset so outputs better match desired behavior. It complements—not always replaces—prompting and RAG.
Why it matters
Teams choose fine-tuning when they need consistent formats, domain tone, or specialized skills that prompting alone cannot stabilize.
Article
Methods
Instruction tuning uses prompt–response datasets. LoRA injects low-rank adapters (Hu et al.). Preference methods further shape responses after supervised fine-tuning.
Examples
JSON clerk
Fine-tune an open-weight model on 5k validated prompt→JSON pairs to stabilize extractions for invoices.
FAQ
Does fine-tuning update knowledge cutoffs?
It can inject patterns from your data, but it is not a complete substitute for retrieval of changing facts.
Is LoRA “real” fine-tuning?
Yes. It is parameter-efficient fine-tuning that often matches full updates for many specialization tasks.
Sources
-
LoRA: Low-Rank Adaptation of Large Language Models
ICLR / arXivParameter-efficient fine-tuning method (Hu et al.).
-
Training language models to follow instructions with human feedback
NeurIPS / arXivInstructGPT / RLHF paper (Ouyang et al.).
-
OpenAI API Documentation
OpenAIOfficial OpenAI platform documentation.
-
Hugging Face Documentation
Hugging FaceOpen model hub and Transformers library docs.