LLM Token Counter
Paste any text or prompt to count tokens across GPT, Claude, Llama, and Gemini tokenizers, and estimate API cost before you send the request.
0 tokens
Est. cost: $0.0000 input / $0.0000 output per 1K tokens at current published pricing
What is LLM Token Counter?
An LLM token counter breaks input text into the same sub-word units ('tokens') that a language model's tokenizer would produce, then reports the total count. This matters because every LLM API charges and limits context by tokens, not characters or words — and tokenization rules differ by model family, so the same sentence can be 40 tokens on GPT-4 and 52 on Llama 3. This tool is for developers, prompt engineers, and anyone integrating an LLM API who needs to plan prompt length against a model's context window (e.g. 128k, 200k tokens) or forecast cost before making a call. Paste or type text into the box and select a model family; the tool runs the actual tokenizer encoding for that family (BPE-based tiktoken for OpenAI models, SentencePiece-based encodings for Llama, and equivalent public tokenizers for Claude and Gemini where available) and displays the token count alongside an estimated cost using current published per-1K/per-1M token pricing for that model. If a model's exact tokenizer isn't public, the tool shows a clearly labeled close approximation rather than a false exact number.
Token count is computed by running the input string through the actual published tokenizer for the selected model (tiktoken encodings such as cl100k_base/o200k_base for OpenAI models, SentencePiece/BPE vocabularies for Llama-family models). Where a vendor has not published an exact public tokenizer (e.g. some Claude and Gemini versions), the tool uses the vendor's official approximation guidance (commonly ~4 characters per token in English) and labels the result as an estimate. Cost estimates multiply the resulting token count by the model's current published input/output price per 1K or 1M tokens.
| Model family | Tokenizer type | Typical tokens per English word | Exact public tokenizer? | |
|---|---|---|---|---|
| GPT-4 / GPT-3.5 (OpenAI) | tiktoken (BPE) | ~0.75 | Yes | |
| Claude (Anthropic) | Proprietary BPE | ~0.8 (approx.) | No — estimate only | |
| Llama 3 (Meta) | SentencePiece/BPE | ~0.75 | Approximated here (~4 chars/token) | |
| Gemini (Google) | SentencePiece-based | ~0.8 (approx.) | No — estimate only |
- Input: 'The quick brown fox jumps over the lazy dog.' → GPT-4 (cl100k_base): 10 tokens. Llama 3: 11 tokens.
- Input: a 500-word product description → GPT-4: ~650 tokens (~$0.0065 at $0.01/1K input tokens). Claude estimate: ~670 tokens.
- Input: a 20,000-character code file → GPT-4: ~5,100 tokens, comfortably inside a 128k context window; Llama 3 8B (8k context): would need chunking.
Reference: OpenAI tiktoken (official tokenizer library), OpenAI: Managing tokens documentation, Anthropic: Token counting guidance, Meta Llama model tokenizer (SentencePiece)
How to use LLM Token Counter
- Paste or type your prompt, document, or code into the input box.
- Select the model family (GPT-4, GPT-3.5, Claude, Llama, or Gemini) whose tokenizer you want to use.
- Read the token count and estimated cost shown below the input.
- If your text exceeds the selected model's context window, trim or split it and re-check the count before sending it to the API.
Common errors
- Token count looks 'wrong' compared to a character or word count — this is expected; tokens are sub-word units, not characters or whole words, so counts will never match either directly.
- 'Estimate only' label appears for Claude or Gemini — this happens because those vendors haven't published an exact public tokenizer, so the tool falls back to their official approximation formula instead of a false exact count.
- Cost estimate seems off after a price change — API pricing changes periodically; if a number looks stale, check the model vendor's current pricing page, since this tool uses a fixed reference price that is updated periodically, not live.
- Very large pastes (100k+ characters) slow down or freeze the input box — browser-based tokenization on very long text is CPU-intensive; split the text into smaller chunks and sum the counts.
FAQ
How do I count tokens for an LLM prompt before calling the API?
Paste the exact text you plan to send — including system prompt, few-shot examples, and user message — into the counter, select the target model, and use the displayed token total to check it fits within that model's context window before making the request.
Why does the same text have a different token count on GPT-4 versus Llama?
Each model family uses its own tokenizer trained on its own vocabulary, so the same sentence gets split into different sub-word chunks. GPT-4 and GPT-3.5 use OpenAI's tiktoken encodings, while Llama models use a separate SentencePiece-based vocabulary, so counts for identical text will differ, sometimes by 10-20%.
Is token count the same as word count or character count?
No. A token is roughly 3/4 of an English word on average, but punctuation, numbers, whitespace, and non-English text can all tokenize differently — a single word can be one token or split into several, so token count should always be checked directly rather than estimated from word or character count.
How accurate is the token count for Claude and Gemini models?
Because Anthropic and Google haven't published exact public tokenizers for all their models, counts for Claude and Gemini are close estimates based on each vendor's official approximation guidance, not exact byte-for-byte tokenizer output. For billing-critical decisions, treat these as a reliable estimate rather than an exact figure.
Can I use this tool to estimate API cost before sending a request?
Yes — after the tool counts tokens for your selected model, it multiplies that count by the model's current published input (and, where relevant, output) price per token to show an estimated cost, which is useful for budgeting before you make the actual API call.