Fine-tuning

Fine-tuning is the process of further training a pre-trained model on a smaller, task-specific dataset so it becomes better at a particular job. The base model already knows grammar, facts, and reasoning from pre-training; fine-tuning teaches it the style, format, or domain you care about. Common reasons to fine-tune include matching a brand voice, classifying support tickets, extracting structured fields from documents, and improving performance on low-resource languages. You typically need hundreds to thousands of high-quality examples. Each example pairs an input with the desired output, and the model's weights are updated to reduce the error on those examples. Fine-tuning is not always the right first step. Prompt engineering, retrieval augmentation, and few-shot examples are faster and cheaper to iterate. Fine-tuning becomes worthwhile when the behavior you want is hard to describe in a prompt, must be consistent at scale, or needs to run without sending long examples every request. Techniques like LoRA and QLoRA make fine-tuning feasible on consumer hardware by updating only a small subset of weights.

Published 2026-06-12

Related terms