Phase 3 — Core AI Engineering · Lesson 38 · 20 XP
Fine-tuning: when and how
Prompting and RAG change nothing about the model itself — only what goes into the input. Fine-tuning actually updates the model's weights. That makes it more powerful but also more expensive, slower to iterate on, and easy to reach for before you actually need it.
Fine-tuning tends to beat prompting only for narrow, high-volume, consistent-format tasks where good prompting has already hit a real ceiling — which is rare early in a project. LoRA fine-tunes a small set of added adapter weights instead of the entire model, which is dramatically cheaper and faster while still capturing most of the benefit for many tasks.
Exercise
Before reaching for fine-tuning, try to solve a task with better prompting and a few few-shot examples first. Only if that genuinely plateaus, try a small LoRA fine-tune on a small open model and compare the two approaches directly.
Check yourself
1. Why does the roadmap say to try fine-tuning rarely, and only after prompting has genuinely been exhausted?
2. What does LoRA fine-tune that a full fine-tune doesn't, and why does that make it so much cheaper?
Guardrails and prompt injection
Answer the check-yourself questions to unlock this