Which GenAI Is the Fastest to Respond?

2026-08-29

Key Takeaways

Two different questions hide inside “fastest”: how quickly the first word appears, and how quickly the whole answer finishes. Different models win each.
On raw output speed, Celeris-1 leads at roughly 1,590 tokens per second, with Inception’s Mercury 2 second at about 1,017 tokens per second. Both are diffusion models rather than conventional autoregressive ones.
On time to first token, Gemini 2.5 Flash-Lite and Cohere’s Command A+ tie near 0.38 seconds, ahead of Granite 4.2 3B at 0.46 seconds.
Among frontier models, GPT-5.5 Mini reaches about 168 tokens per second and Claude Sonnet 4.6 about 104, while GPT-5.5 Standard runs near 92 and Claude Opus 4.7 near 78.
Specialist inference hardware changes the picture entirely: Cerebras and Groq deliver 480 to 525 tokens per second on large open models with first-token latency as low as 0.16 seconds.
Reasoning modes destroy interactive speed. GPT-5.5 Pro at high reasoning shows a median first-token time of 67 seconds, Gemini 3 Pro Deep Think 52 seconds, Claude Opus 4.7 at maximum thinking 28 seconds.
Geography adds real delay: median first-token time rises about 98 milliseconds in EU-Central and 207 milliseconds in APAC-Tokyo against a US-East baseline.

Software development, vibe coding – artistic impression. Image credit: Alius Noreika / AI

The fastest generative AI to respond in 2026 is Celeris-1, which produces roughly 1,590 tokens per second and reports a median response time of 158 milliseconds. If you care instead about how quickly text starts appearing, Gemini 2.5 Flash-Lite and Cohere’s Command A+ answer fastest, both around 0.38 seconds to first token.

Those are two different crowns, and picking the wrong one leads to the wrong model. A short chat reply feels fast when the first word arrives quickly. A long document draft feels fast when tokens stream out at volume. A model can lead on one measure and sit mid-table on the other, which is why a single “fastest AI” answer is usually misleading.

The Two Metrics That Matter

Time to first token (TTFT) measures the gap between sending a request and receiving the first piece of the reply. It governs perceived responsiveness. Below roughly 300 milliseconds an interface feels instant; above a second it feels like waiting.

Output speed, measured in tokens per second, governs how long a full answer takes. For a 1,000-token response, the difference between 80 and 400 tokens per second is 12.5 seconds versus 2.5 seconds. If you are unclear on what a token represents, our explainer on tokens as the currency of generative AI covers the unit itself.

Fastest Output Speed: The Diffusion Models Took the Lead

Model	Output speed	Architecture
Celeris-1	About 1,590 tokens per second	Diffusion
Mercury 2 (Inception)	About 1,017 tokens per second	Diffusion
Gemini 2.5 Flash-Lite	About 395 tokens per second	Autoregressive
LFM2.5-VL-1.6B (Liquid AI)	About 391 tokens per second	Autoregressive

The top two entries share an architecture, and that is the story of 2026. Conventional language models generate one token at a time, each conditioned on the last, which makes generation sequential and bound by memory bandwidth. Diffusion language models start with a rough draft of the entire response and refine it over a handful of parallel passes.

Celeris CTO Jesse Clark described the mechanism plainly: the model “starts with a rough version of the entire response and improves it over a few rapid passes.” The consequence is architectural rather than incremental — the workload shifts from memory-bandwidth-bound to compute-bound, which lets modern accelerators run closer to saturation.

Celeris-1 launched on 27 July 2026, claiming 1,664 tokens per second in its own testing, 24 times faster than GPT-5, a 158-millisecond median response and 75.9% on MMLU-Pro. CEO Tom Hamer framed the goal as “intelligence delivered per unit of time” rather than benchmark position.

Mercury 2 arrived earlier, on 24 February 2026, from Inception. It reports around 1,000 tokens per second against roughly 89 for Claude 4.5 Haiku Reasoning and 71 for GPT-5 Mini, while scoring 91.1 on AIME 2025, 73.6 on GPQA and 67.3 on LiveCodeBench. CEO Stefano Ermon put the case for speed as a production requirement: “Reasoning models are only as useful as their ability to run in production.”

Fastest to First Token: Small Models and Big Caches

Model	Time to first token
Gemini 2.5 Flash-Lite	About 0.38 seconds
Command A+ (Cohere)	About 0.38 seconds
LFM2-24B-A2B (Liquid AI)	About 0.42 seconds
Granite 4.2 3B (IBM)	About 0.46 seconds
Grok 4.1 Fast (xAI)	About 0.54 seconds
Ministral 3 3B (Mistral)	About 0.60 seconds

Small models dominate this table for an obvious reason: fewer parameters mean less work before the first token can be emitted. Grok’s placement here is consistent with the speed-first positioning we examined in our review of Grok 4.5 benchmarks and speed.

How the Frontier Models Compare

Most production work still runs on large general-purpose models, where speed differences are narrower but still meaningful.

Model	Median TTFT	95th percentile TTFT	Output speed
GPT-5.5 Mini	0.61s	1.32s	168 tokens/s
Claude Sonnet 4.6	0.74s	1.61s	104 tokens/s
Claude Opus 4.7	0.85s	1.83s	78 tokens/s
Gemini 3 Pro	0.93s	2.04s	84 tokens/s
GPT-5.5 Standard	1.12s	2.41s	92 tokens/s
Gemini 3 Flash	About 0.42s	—	About 200 tokens/s

The 95th percentile column is the one to design around. As the benchmark authors put it, “P50 is the marketing number. P95 is the reality” — one user in twenty waits nearly twice as long as the median suggests, and that is the experience your support inbox hears about.

At the provider level, OpenAI averages about 126 tokens per second across 27 tracked models, while Anthropic averages roughly 58. Anthropic’s speed-optimised line closes that gap considerably, as our look at Claude Haiku’s speed and cost profile describes.

Specialist Hardware Beats Software Optimisation

The fastest way to serve a conventional open model is not a better model — it is different silicon. Cerebras runs wafer-scale processors and Groq built language processing units designed around deterministic execution rather than general-purpose parallelism.

Configuration	Output speed	Median TTFT
Cerebras, Qwen 3 235B	525 tokens/s	0.21s
Cerebras, Llama 4 70B	520 tokens/s	0.16s
Groq, Llama 4 405B	480 tokens/s	0.18s

A 405-billion-parameter model answering in 0.18 seconds and streaming at 480 tokens per second outruns most small models on cloud GPUs. Nvidia noticed: it agreed in December 2025 to acquire Groq’s assets for about $20 billion, its largest deal on record, and said in August 2026 that Groq racks would be online within the year. The hardware layer is where the remaining speed lives, a point our guide to AI infrastructure and model performance develops further.

The Reasoning Tax

Extended thinking does not slow a model down slightly. It moves it into a different category altogether.

Configuration	Median time to first token
GPT-5.5 Pro, high reasoning	67 seconds
Gemini 3 Pro, Deep Think high	52 seconds
Claude Opus 4.7, maximum thinking	28 seconds

That is a 5x to 30x inflation over standard mode, which rules these configurations out of anything interactive. They belong in background jobs, batch analysis and agentic pipelines where a minute of thinking buys a better answer and nobody is watching a cursor blink.

Where You Are Sitting Changes the Answer

Network distance is a fixed tax that no model choice removes. Against a US-East baseline for GPT-5.5 standard, median first-token time rises 38 milliseconds in US-West, 98 milliseconds in EU-Central and 207 milliseconds in APAC-Tokyo. For a European or Asian user, a nominally slower model served from a nearby region can feel faster than a quicker model served from Virginia.

Choosing the Right Kind of Fast

For voice interfaces and real-time agents, optimise time to first token and accept a smaller model — anything under 0.5 seconds keeps a conversation feeling natural. For long-form generation, summarisation and code output, optimise tokens per second, where the diffusion models and specialist hardware pull far ahead. For high-volume open-model serving, put the model on Cerebras or Groq rather than searching for a faster model.

And for anything where the answer must be right more than it must be quick, ignore all of these numbers. A benchmark measuring latency tells you nothing about whether the output is correct, which is a separate comparison entirely — one we work through in our guide to which LLM answers user queries best.

One methodological caution: published speed figures are screening evidence, not service-level guarantees. They vary by prompt length, output length, region, time of day and provider load. Measure on your own traffic before committing.

If you are interested in this topic, we suggest you check our articles:

Sources: Artificial Analysis, PR Newswire (Celeris), Business Wire (Inception), Digital Applied, BenchLM, CNBC, CNBC (Groq racks)

Written by Alius Noreika