What Tokens Per Second Actually Feels Like
Runs on your numbers
Nothing is seeded. This tool computes from what you type in, so it cannot be more accurate than the figure you give it.
Preset rates are round numbers chosen to span the range, not measurements. Presets naming a specific machine arrive with the first published run, and not before.
This simulator needs JavaScript. Without it, the arithmetic is still the whole story: divide 138 tokens by the rate you care about to get the number of seconds.
Runs in your browser. Nothing you type here is uploaded, logged or sent anywhere. The page is a static file and the arithmetic happens on your machine. Same principle as the rest of the site.
How this is calculated
There are only two operations in this tool, and neither is hidden.
seconds to finish = tokens ÷ (tokens per second)
tokens shown at time t = t × (tokens per second)
The constants it leans on:
| Constant | Value | Where it comes from |
|---|---|---|
| Words per token, English prose | 0.75 | OpenAI's published rule of thumb |
| Adult silent reading rate, non-fiction | 238 words/min | Brysbaert 2019, meta-analysis of 190 studies |
| Reading rate in tokens/sec | 5.3 | 238 ÷ 60 ÷ 0.75 |
Worked example, the one printed above: the sample answer is 109 words, which this page splits into 138 tokens. At 8 tokens per second that is 138 ÷ 8, or 17 s. At 3 tokens per second it is 46 s. The panes are driven off one clock so the two rates start together and you are comparing them, not watching them in turn.
What this can't know
- It is not a tokenizer. The sample is split by a rule of thumb, splitting words longer than five characters in half. A real model uses a learned sub-word vocabulary, so its token count for this same passage would differ, and differ more for code, for numbers, and for languages other than English.
- It ignores time to first token. The panes start printing immediately. A real request pauses first while the prompt is processed, and that pause grows with prompt length.
- Real generation is not perfectly even. This streams at a constant rate. On a real machine the rate drifts down as the context fills, and can drop sharply once a model spills out of memory or the hardware throttles.
- Your reading speed is not the average. The 238 words a minute above is a mean across studies, with a wide spread around it. The rate that feels fast enough to you is yours to find, which is what the input box is for.
- It says nothing about output quality. A fast small model and a slow large one are not the same answer arriving at different speeds.
Sources
- Brysbaert, M. (2019). How many words do we read per minute? A review and meta-analysis of reading rate. Journal of Memory and Language, 109. doi.org/10.1016/j.jml.2019.104047
- OpenAI. What are tokens and how to count them? help.openai.com
- No figure on this page came off the bench. When runs are published, machine presets are seeded from them and this tool is re-badged Measured. Until then it computes only from what you type in. The method those runs will be held to is on the methodology page.
FAQ
What is a good tokens per second for a local LLM?
Judge it against your own reading speed rather than a leaderboard. Average adult reading is around 5.3 tokens per second, so anything below that means waiting for words you have already caught up with, and anything above it means the text is there when you look for it. For back-and-forth chat where you read every word, the gap between 15 and 60 tokens per second matters much less than the gap between 3 and 15. For a model writing code or long output you are going to skim, faster keeps paying off.
Is tokens per second the same as words per second?
No. A token is roughly three quarters of an English word, so a rate in tokens per second is about 0.75 times as many words. The ratio shifts with the model, and it gets much worse for languages that the tokenizer was not mostly trained on, where a single word can cost several tokens.
Does time to first token matter more than tokens per second?
They are different waits and both are worth knowing. Time to first token is how long you stare at nothing after pressing enter, and it grows with the length of your prompt. Tokens per second is how fast the answer arrives once it starts. A short question with a long answer is dominated by tokens per second; a long document pasted in with a one-line answer is dominated by time to first token. This simulator models only the second one.
Why is my model slower than the tokens per second I saw quoted?
Quoted rates usually come from a short context, a small batch, and a machine that had just been idle. Filling the context window slows generation as the attention cache grows, a heavier quantization level trades speed for quality in either direction, background load steals memory bandwidth, and a laptop that has been generating for a few minutes may be thermally throttled. A rate without its conditions attached cannot be compared to anything.
Where to go next
- Never run a model locally before? Run your first local LLM with Ollama gets you to a working model in about fifteen minutes.
- Measured rates for specific hardware and models go in the benchmark database as runs are published.
- The rest of the tools, including what is queued and what is waiting on the bench.