Parameters, Tokens, and Context Windows

Build the mental model

You have probably seen model names like '7B' or '70B' and wondered what the number means. It is short for billions of parameters - the individual numeric values inside the model that were adjusted during training. A 7B model has about 7 billion of these values; a 70B model has about 70 billion.

It is tempting to assume bigger number always means better model, the way a bigger engine usually means a faster car, but that assumption breaks down quickly with AI. A model's quality actually depends on several things together:

• Architecture - how the pieces are wired
• The data it was trained on
• How well it was fine-tuned for the task you care about
• How it was quantized for size (covered in a later lesson)

A smaller model trained carefully on excellent data can outperform a larger model trained carelessly, so parameter count alone is only a rough hint, never a final answer.

Separately from parameters, every model reads and writes text in chunks called tokens - not quite whole words, not quite letters, usually somewhere in between. A tokenizer is the piece of software that splits your text into these chunks before the model can process it. Every model has a context window: a maximum number of tokens it can pay attention to at once. That budget is shared by everything you feed it:

• The system prompt that sets its behavior
• The entire conversation history so far
• Your newest message

So a long conversation can eventually push older messages out of the model's view entirely, even though nothing was ever deleted from your screen.

Token

A small chunk of text a model processes at a time - not quite a whole word, not quite a single character.

Tokenizer

The piece of software that splits text into tokens before a model can process it.

Context Window

The maximum number of tokens a model can pay attention to at once.

WHAT FILLS A CONTEXT WINDOW
---------------------------
CONTEXT WINDOW (fixed token budget, e.g. 8000 tokens)
-------------------------------------------------------
| system prompt   | conversation history | new message |
-------------------------------------------------------
  sets behavior      everything said so far   your latest
                                                  input

  all three share the SAME budget - if the total goes
  over the limit, the oldest history drops out of view

Connect it to a real scenario

Suppose you are comparing two local models before downloading one: a 3B model and a 13B model. It is easy to just grab the 13B model assuming bigger is automatically better, but if your task is simple - say, classifying short customer messages into categories - a well-tuned 3B model might answer just as accurately while using far less memory and running noticeably faster on your hardware.

Understanding tokens and context windows matters just as practically: if you are building a tool that feeds a model long documents plus a running conversation, you need to know roughly how many tokens your input costs, or you will hit the context limit mid-conversation and watch the model 'forget' things you said earlier.

The heuristic estimator below is not a real tokenizer - real ones use trained vocabularies - but it gives you a fast, rough sense of how much of your budget a piece of text will consume.

• Parameter count alone never determines model quality
• Tokens are the small chunks a model reads and writes text in
• System prompt, history, and new message all share one context window

Try the working example

def estimate_tokens(text):
    """Very rough heuristic only: about 1 token per 4 characters.
    Real tokenizers use trained subword vocabularies and do NOT
    work this way - this is just a fast mental-model estimate."""
    return max(1, round(len(text) / 4))


sentences = [
    "Hello, how are you today?",
    "Local AI keeps your data on your own device.",
    "ဒီနေ့ မိုးရွာနေတယ်။",
]

for s in sentences:
    tokens = estimate_tokens(s)
    print(f"'{s}' -> ~{tokens} tokens (chars={len(s)})")

'Hello, how are you today?' -> ~6 tokens (chars=25)
'Local AI keeps your data on your own device.' -> ~11 tokens (chars=44)
'ဒီနေ့ မိုးရွာနေတယ်။' -> ~5 tokens (chars=19)

Hugging Face docs: Tokenizers — Local AI / Local LLM