Fine-tuning is additional training on a pre-trained foundation model using your data to change how it behaves — shaping it for a specific domain, task, or output style.

It's also the most over-prescribed technique in the machine intelligence toolkit. Before you invest in data curation, training runs, and ongoing maintenance, prove that RAG plus solid prompt engineering can't get you close enough. In most cases, they can.

Fine-tuning makes sense when the behavior you need can't be reliably stuffed into a context window: a consistent output style across thousands of generations, stable classification that doesn't drift with prompt wording, or domain-specific patterns that retrieval alone doesn't solve. These are real use cases. They're also the minority.

The operational reality is often undersold. Fine-tuning requires clean, representative training data and rigorous evals — otherwise you're optimizing toward noise. It also creates a maintenance commitment: when the base model improves, you have to decide whether to retrain, evaluate the new base against your fine-tuned version, and absorb that cost. It's not a one-time project.

Use it when you've exhausted cheaper options and have the data quality to do it right.

Further reading:

• LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
• Fine-tuning — OpenAI Documentation