Thuta Learning
Search
IntermediateAIbeginner

Tokens and Context Windows

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

  • Understand Tokens and Context Windows without the intimidation factor
  • Get hands-on practice trying it yourself
  • Spot the common pitfalls and sidestep them with a smile

Understand tokens, context windows, input/output budgets, and how to manage information as a conversation gets longer.

Let's think about it this way for a moment

A model doesn't read text word by word — it reads small chunks called tokens. The context window is the limit on everything the model can consider in a single request: instructions, history, documents, and output all included. Longer text doesn't automatically mean more useful context.

Tokens and Context Windows lesson illustration
Prompting and LLMs — Tokens and Context Windows

Let's connect this to everyday life

Instead of sending the entire chat history every time, summarize the older parts, pick out only the documents relevant to the current task, and cap the output length. This keeps cost and latency down, and helps make sure important instructions don't get lost.

Let's try it hands-on together

text
Context budget
├── System instructions
├── User request
├── Relevant chat history
├── Retrieved documents
└── Reserved output space
You should see
You'll be able to manage a chat request so it doesn't overflow the context.

5-minute try-it

Imagine you have a very long chat history. Sort it into what must be kept, what should be summarized, and what can be dropped.

A quick word of caution

Don't treat AI output as the final, guaranteed-correct answer. For anything important — critical facts, code, or user data — have a human review it before you use it.

OpenAI — Text generation — OpenAI

Easy traps

  • Assuming the context window is the model's permanent memory
  • Sending an entire document in full

Exercise

Imagine you have a very long chat history. Sort it into what must be kept, what should be summarized, and what can be dropped.

You'll know it worked when: You'll be able to manage a chat request so it doesn't overflow the context.

Tokens and Context Windows | Thuta Learning