Engineering
RAG or fine-tuning? How we choose for client projects
Most companies do not need to train a model. Here is the decision grid we use on every assistant project.
Samuel Okafor
4 Nov 2026 · 7 min read
Every week a client asks us whether they should fine-tune a model on their data. In nine cases out of ten, the answer is no, and the reason is simple: what they need is for the model to know their facts, not to change how it writes.
Start with retrieval
Retrieval augmented generation keeps your documents outside the model. The assistant searches them at question time and quotes what it finds. It is cheaper, it updates the moment a document changes, and every answer can cite its source.
When fine-tuning makes sense
- A very specific output format repeated thousands of times
- A tone of voice that prompting alone cannot hold
- Latency or cost constraints that call for a smaller model
Our rule of thumb
Ship retrieval first, measure answer quality on 200 real questions, and only then decide whether fine-tuning closes a gap that matters for the business.