Data & accuracy
How recommendations are made
The ranking method, the competence bar, and why there's no LLM involved.
3 min read
There is no language model in the loop
Answers are computed directly from the catalogue by a deterministic analyst. No model provider is called, nothing is generated. That has three consequences worth knowing:
- The same question always produces the same answer.
- No figure can be invented — every number traces to a row in the catalogue.
- The consultant cannot reason about anything outside the catalogue, which is why it declines off-topic questions instead of improvising.
How a recommendation is built
- Your question is parsed into an intent, an objective, a task and a set of constraints.
- Constraints filter the catalogue — capability flags, price ceilings, context minimums, licensing.
- Remaining models are scored on the benchmarks relevant to your task.
- Models without enough benchmark coverage to score are excluded, and the answer says how many were.
- The shortlist is ranked, and the runner-up and cheapest options are checked for the trade-off section.
The competence bar
When you ask for the cheapest or fastest model that does something, optimising on price alone would return a model that technically has the capability flag but is far too weak for the work. So candidates must first clear a fit score of 55 on the task before the price sort runs.
If fewer than three models clear the bar, it is dropped and the answer tells you so — a stated assumption is more useful than a silently narrowed one.
What we don't do
We don't run our own evaluations, don't take payment for placement, and don't weight any maker preferentially. The ranking is arithmetic over third-party benchmark data and published prices.
Still stuck? Contact us