Tokens and the context window.
You will know what tokens are, why they make spelling hard, what the context window holds, and why long chats slow down and lose early details.
Type How many letters are in the word strawberry? into a chatbot and you will usually get the right answer, ten. Ask how many times the letter r appears and, for a long while, many models confidently said two. (It is three.) A system that can draft a contract tripping over a spelling question looks absurd until you know one thing about how it reads.
That one thing is the token, and it explains a surprising amount: the spelling slips, why long chats slow down, and why starting fresh often helps.
The model does not read words
Before any text reaches a language model, it is chopped into pieces called tokens. A token is a chunk of text the model treats as a single unit. Common short words are usually one token each. Longer or rarer words are split into several: a word like "unbelievable" might become something like "un", "believ" and "able". Punctuation marks are often tokens of their own. Spaces usually get attached to the start of the word that follows them.
In English, a token works out to about three quarters of a word on average, so a 1,000-word article is roughly 1,300 tokens. Other languages often cost more. Text in a language that was less common in the training data, or that uses a different script, can take two or three times as many tokens to say the same thing. Code, numbers and unusual formatting also tend to be token-heavy.
Why chop text this way instead of using whole words or single letters? Whole words would need a vocabulary of every word in every language, and the model would still be stuck on a new one. Single letters would make every sentence very long and push the useful patterns far apart. Pieces of words are the compromise: a vocabulary of tens of thousands of chunks or more that can spell out anything, with common things kept short.
See a sentence as tokens
Type anything. The pieces below are an approximation of how a model would cut it up. Real tokenizers differ in the details, but the shape is right: common words stay whole, rare and long words break into parts.
Why spelling and wordplay are hard
Now the strawberry puzzle makes sense. The model never sees the letters s, t, r, a, w. It sees one or two tokens that stand for the whole word, much as you see the word rather than sounding out each letter. To count the r's, it has to recall how that token is spelled, and spelling is not something it was trained to care about. It was trained to predict the next token, and you can predict the next token very well without knowing how many r's are in the current one.
The same gap shows up elsewhere. Reversing a word, counting syllables, solving an anagram: these tasks live at the letter level, below the layer the model works at. It often manages them anyway, from having seen enough examples, but this is one of the places where fluency and reliability come apart. Recent models have improved here, partly by reasoning step by step before answering. The point is not that they always fail. It is that when they do fail at something a child could do, tokens are usually why.
Arithmetic has a related problem. A number like 48,213 may be split into chunks that do not line up with place value, so the model is doing math on awkward pieces. This is one reason it is wise to have a model use a calculator or write code for any numbers that matter.
The context window: everything it can see
The context window is the total amount of text the model can take in at one time, measured in tokens. Think of it as the model's desk. Everything on the desk is visible. Anything not on the desk does not exist, as far as the model is concerned.
What is on the desk during a chat? More than you might think:
- Every message you have typed in this conversation.
- Every reply the model has given.
- Any files or documents you pasted or attached, in full.
- Instructions the app itself adds before the conversation starts, which you do not see.
All of it counts toward the limit, and a long PDF can fill a large share of the desk on its own.
How big is the desk? It varies by model and keeps growing. Early chatbots could hold a few thousand tokens, a few pages of text. Recent models hold hundreds of thousands of tokens, enough for a long book. The exact figures change too fast to be worth memorizing. What does not change is that the window is finite, and that everything in the conversation competes for the same space.
It re-reads everything, every turn
Here is the part that surprises most people. The model has no memory of the conversation between turns. Each time you send a message, the app gathers the entire conversation so far, your messages and its replies and the files, and hands the whole thing to the model as a single block of tokens. The model reads all of it from the beginning and produces the next reply. Then it is done. On your next message, the whole block is handed over again, one turn longer.
This is why the model seems to remember what you said twenty messages ago. It is not remembering; it is re-reading. It is also why the conversation feels continuous when, from the model's side, every turn is a fresh start with a long transcript attached.
Two practical consequences follow. First, each turn costs more than the last, because there is more to read: more computing time and, for anyone paying per token, more money. Second, when the transcript grows past what the window can hold, something has to give. Depending on the app, the oldest messages may be dropped or summarized, and the model may be working from less than you think it has been told.
What this means in practice
Knowing about tokens and the window changes a few habits.
Long chats degrade. Even well inside the limit, a detail you mentioned early can get crowded out by everything that came after. The model is reading it, but it is one line among thousands, and its pull on the reply can fade. If something matters, restate it near your current question. Asking for shorter replies helps too, since the model's own answers fill the desk alongside yours.
Start fresh for a new topic. If you spent an hour planning a trip and now want help with a work email, open a new chat. The travel transcript is making every turn slower and giving the model irrelevant material to be distracted by. A clean window with just the email and your request will do better.
Know when you are near the edge. Signs include replies that forget constraints you set earlier, repeat things already covered, or get vaguer about details from the start of the chat. That is the window filling up. Summarize what matters in a few lines, start a new conversation, and paste the summary in as your first message.
None of this requires counting tokens yourself. It requires remembering that the model works from a desk of fixed size, that everything on it is read again every turn, and that you control what goes on it.
Try it
First, see tokens in action. Type: Split the sentence "The unbelievably quick fox jumped over 13 fences" into the tokens you think a language model would use, and explain why some words become more than one token. The exact split will vary, and the model may not know its own tokenizer precisely, but the explanation should match this lesson: common words stay whole, long and rare words get chopped, numbers and punctuation are their own pieces.
Then test the window. In a chat where you have already exchanged a dozen or more messages, type: Quote the very first message I sent in this conversation, word for word. If it gets it exactly, the transcript is still comfortably inside the window. If it paraphrases or guesses, you are watching the early part of the chat lose its grip.
Key terms
- Token
- A chunk of text, often part of a word, that a language model treats as a single unit; in English about three quarters of a word on average.
- Tokenizer
- The step that chops text into tokens before the model sees it, using a fixed vocabulary of pieces.
- Context window
- The maximum amount of text, measured in tokens, that a model can take in at one time, including your messages, its replies and any files.
- Turn
- One exchange in a chat, your message and the model's reply, after which the whole conversation is handed back to the model again.
Check yourself
Models read text as tokens inside a fixed-size window that is re-read every turn, so keep what matters close and start fresh when the topic changes.