All insights

Engineering

RAG or fine-tuning? How we choose for client projects

Most companies do not need to train a model. Here is the decision grid we use on every assistant project.

Samuel Okafor

Samuel Okafor

4 Nov 2026 · 7 min read

RAG or fine-tuning? How we choose for client projects

Every week a client asks us whether they should fine-tune a model on their data. In nine cases out of ten, the answer is no, and the reason is simple: what they need is for the model to know their facts, not to change how it writes.

Start with retrieval

Retrieval augmented generation keeps your documents outside the model. The assistant searches them at question time and quotes what it finds. It is cheaper, it updates the moment a document changes, and every answer can cite its source.

When fine-tuning makes sense

  • A very specific output format repeated thousands of times
  • A tone of voice that prompting alone cannot hold
  • Latency or cost constraints that call for a smaller model

Our rule of thumb

Ship retrieval first, measure answer quality on 200 real questions, and only then decide whether fine-tuning closes a gap that matters for the business.

Share

Keep reading

Have a process AI could take off your hands?

Tell us about it. You get a written assessment and a first estimate within 48 hours, free.

Get my free assessment
WebsiteAdmin panel