Live race
The same question goes out through different setups at the same instant. Each bar is grey while it waits for the first real token from the model, then blue while text streams in.
Benchmark results
Loading…
How to trust these numbers
- One change at a time. Every comparison differs in exactly one setting, on the same question, in random order, one request at a time.
- First token means a real token. It is timed at the client when the model's first word arrives, not when a spinner or an acknowledgement appears.
- Nothing is hidden. The model client makes one attempt, so throttling shows up as a failure instead of extra delay, and failures are counted and listed.
- Speedups carry an interval. "Unclear" means the 95% interval includes 1x, so the data does not show a difference.
- One race is one sample. It is a picture, not a measurement. The statistics come from the benchmark runs above.