What the AI Token Counter does
This token counter tells you how many tokens a piece of text or a chat conversation uses, exactly for OpenAI's two published tokenizers (o200k_base and cl100k_base) and as clearly labelled estimates for Claude and Gemini, whose current tokenizers are not public. It also shows what share of a chosen model's context window the text would take.
Everything runs in your browser. The OpenAI tokenizer data - the same rank files OpenAI publishes with its open-source tiktoken library - is downloaded once from this site and nothing you paste is uploaded. There is no single "token count": the same sentence can be 9 tokens for one model and 33 for another, which is why every row here names its tokenizer and says whether it is exact.
How to use it
- Paste your text, or load one of the examples. Counting starts as soon as there is text; the first count downloads the two OpenAI tokenizer files (about 5 MB, cached by your browser afterwards).
- If you are counting a conversation, switch to Chat messages and paste a JSON messages array, or lines that begin with
system:,user:andassistant:. The OpenAI rows then add the per-message formatting overhead. - Pick the model you plan to use. Its context window is filled in from the provider's model page where one was confirmed; otherwise type the figure you use.
- Read the table: rows marked Exact are real tokenizer output; rows marked Estimate state the method used. Download the table as CSV if you need to keep it.
Reading the results
The big number is the o200k_base count, the encoding used by GPT-4o, GPT-4.1, the GPT-5 family and the o-series reasoning models. cl100k_base is the older encoding behind GPT-4 and GPT-3.5 Turbo and is still used by the text-embedding-3 models.
Claude and Gemini rows are estimates of characters divided by four, the rule of thumb both providers publish for English. Anthropic states that Claude Opus 4.7 and later use a newer tokenizer producing about 30% more tokens for the same text, so that row multiplies by 1.3. For an exact Claude figure use Anthropic's token-counting endpoint; for Gemini use its countTokens method.
Characters per token is a quick health check. English prose usually sits around four; code, numbers, non-Latin scripts and emoji push it down, which means more tokens and more cost for the same visible length.
Worked example: one sentence, three scripts
"The invoice is due on 30 September." is 35 characters. o200k_base and cl100k_base both split it into 9 tokens, and the characters-divided-by-four estimate also gives 9 - for plain English the rule works.
The Hindi version, "हिन्दी: चालान 30 सितंबर को देय है।", is 34 characters. o200k_base needs 14 tokens, cl100k_base needs 33, and the four-characters rule still says 9. The newer OpenAI encoding was trained on far more non-English text, so it packs Devanagari much better; and the estimate undercounts badly for scripts where one character is several bytes.
The Japanese line "日本語：請求書の支払期日は9月30日です。" is 21 characters but 17 tokens in o200k_base and 24 in cl100k_base, against an estimate of 6. The lesson: treat the Claude and Gemini estimates as a floor for non-English text and confirm with the provider's counting endpoint before relying on them.
What is exact here and what is not
The OpenAI counts use byte-pair encoding exactly as tiktoken does it: the text is split with the encoding's published pattern, each piece is turned into UTF-8 bytes, and adjacent byte pairs are merged in rank order until no listed merge is left. The rank files are OpenAI's own, served from this site under their MIT licence, and the implementation is checked against an independent tokenizer on English, Arabic, Hindi, Japanese, emoji, code and edge cases.
What is not included: special tokens such as <|endoftext|> are counted as ordinary text, which is how an API treats them inside a message; images, audio and tool definitions add tokens this page cannot see; and the chat overhead of three tokens per message plus three for the reply comes from OpenAI's cookbook and may differ slightly on newer models. Providers bill from their own usage figures, so treat the response's usage field as the final word.
Limitations: what the result does not prove
- Claude and Gemini figures are estimates. Their current tokenizers are not published, and the four-characters rule is least reliable for code, numbers and non-Latin scripts.
- Counts cover text only. Images, files, audio, tool or function definitions and system scaffolding added by a provider are billed too and are not counted here.
- A model's tokenizer can change between versions. The table names the encoding used so you can check it against the model you actually call.
- Context-window figures come from the providers' pages on the reviewed date; where no official figure was confirmed the field is left blank for you to fill in.
Privacy: where your data goes
Everything you paste, type or drop is processed in this browser tab. It is not uploaded, logged, stored or sent to analytics. Session recording and tag-manager scripts are switched off on this page.
Standards and sources
- OpenAI tiktoken encodings (o200k_base, cl100k_base) - checked 19 Sep 2026
- Anthropic token counting - checked 19 Sep 2026
- OpenAI cookbook - How to count tokens with tiktoken
- Anthropic - Pricing (tokenizer note for Claude 4.7 and later)
- Google - Understand and count tokens (Gemini API)
- gpt-tokenizer (MIT) - source of the vendored rank files
Frequently asked questions
Why does the same text give different token counts for different models?
Each model family has its own tokenizer - its own vocabulary of text pieces learned from its training data. A newer vocabulary with more non-English pieces can cover the same Hindi sentence in less than half the tokens of an older one, so a count is only meaningful next to the tokenizer that produced it.
Is the Claude token count exact?
No. Anthropic has not published the tokenizer used by current Claude models, so this page shows an estimate of characters divided by four, raised by 30% for Claude Opus 4.7 and later as Anthropic's pricing page describes. For an exact count, call Anthropic's token-counting endpoint with your real request.
Which tokenizer does GPT-5 use?
The GPT-5 family, GPT-4.1, GPT-4o and the o-series reasoning models use the o200k_base encoding. GPT-4 and GPT-3.5 Turbo used cl100k_base. The o200k_base row is the one to read for current OpenAI chat models.
How many words is 1,000 tokens?
For ordinary English prose, roughly 750 words - Anthropic and OpenAI both quote about four characters or three-quarters of a word per token. Code, tables, numbers and most non-English text use more tokens per word, so paste a real sample rather than relying on the ratio.
Do chat messages cost more tokens than the same text pasted as one block?
Slightly. Each message carries role markers and separators; OpenAI's cookbook puts this at about three tokens per message plus three to start the reply. In Chat messages mode the OpenAI rows add that overhead so you can see it separately from the text itself.
Is my text uploaded when I count tokens?
No. The tokenizer files are downloaded from this site to your browser and the counting happens locally; the text you paste never leaves the page. You can confirm this in your browser's network panel - the only requests are for the tokenizer and price files.
Last reviewed by the A2Z.Tools team against the sources listed above.
