Data & accuracy

Data & limits

Where the catalogue comes from, what it covers, and where it will let you down.

4 min read

What the catalogue holds

384 entries covering list pricing, context and output limits, capability flags, measured throughput and latency, serving providers, licensing and knowledge cutoffs, plus third-party benchmark results.

Benchmark sources

  • Artificial Analysis — intelligence, coding, agentic and maths indices, plus GPQA and throughput measurements.
  • Design Arena — Elo ratings and win rates for UI and design generation, split by category.

Coverage is uneven — this matters

Only 213 of 384 entries report an intelligence index. 117 report a coding index. 134 carry a Design Arena Elo. Newly released and smaller open-weights models are the most likely to be missing scores.

Where a benchmark is missing it is excluded from scoring, never treated as zero, and the answer tells you how many models qualified. If a recommendation rests on a thin field, that is stated rather than hidden.

Batch and free-tier endpoints

74 entries are alternative serving modes of a model that is also listed separately — the same weights on a different endpoint at a different price. Batch endpoints are typically half price but asynchronous.

These are excluded from recommendations by default, because a 24-hour batch endpoint should not win an interactive comparison on cost. When a batch endpoint exists for a recommended model, it is mentioned in the trade-off section. Ask about batch or async explicitly to include them.

Known limits

  • Prices are list prices. Committed-use discounts, enterprise agreements and promotional rates are not reflected.
  • Cache-read pricing is noted where published but is not applied to cost projections.
  • Benchmarks measure benchmark performance. They are a proxy for how a model behaves on your workload, not a substitute for evaluating it.
  • Throughput and latency are point measurements and vary by provider, region and load.
  • The catalogue is a snapshot. A model released after it was compiled will not appear.