BPE Token Counter & Context Visualizer
Exact token counts for the cl100k_base encoding (GPT-3.5/4-class), sub-word fragmentation view, and context-window usage — computed by the real BPE rank table in your browser.
Total Tokens (cl100k_base)
…
Characters
…
Est. KV-cache RAM (rough)
—
8k Context Usage
—
Sub-word token fragments (cl100k_base):
📦 Tokenizer Engine
Ranks fetched once from this site, then cached by your browser
Packages & Libraries: real Byte-Pair Encoding via js-tiktoken with the cl100k_base rank table (the encoding used by GPT-3.5/GPT-4-class models). Counts are exact for this encoding — other model families (Llama-3, Mistral) use different BPE vocabularies and will tokenize differently.
⚖️ Legal Notice: Provided "as is", without warranty. Token counts are exact for cl100k_base only; KV-cache VRAM figures are rough uniform-model approximations. All tokenization runs locally in your browser.