Research
LLM engineering
What actually separates a language model demo from something you can put in front of a customer.
Retrieval is most of the problem
The single most consistent finding in our work is that retrieval quality dominates model choice. A smaller model reading the correct paragraph outperforms a larger one working from a poor one, at a fraction of the cost per call.
That makes chunking, indexing, and ranking the highest-leverage work in most projects — and it is the part that gets least attention, because it is not the interesting part.
Positions we hold
- Fine-tuning is for format and tone, rarely for knowledge. Using it to teach facts is the most expensive way to discover that retrieval was the answer.
- An evaluation set built from real cases is the difference between engineering and guessing. Without one, "it seems better" is the only available measure.
- Structured outputs beat parsing prose. If the next step is code, the model should be returning data.
- Model access belongs behind an interface. The model you start on is rarely the model you finish on.
- Open-weight models on your own hardware are a genuine option with a genuine quality cost, and clients deserve that stated plainly rather than sold either way.
Open questions
How to keep an evaluation set meaningful as a product changes. A test set built at launch measures a problem the system may no longer have.
Where the cost curve makes self-hosting rational for a mid-sized business, given that the answer moves every few months.
Start here
Tell us what is slow, manual, or breaking.
Answer a few questions and get a written brief back — scope, proposed architecture, and what it would take to build.