Context window vs Training data

Why they differ

Text in the context window is only read for the current answer; training data shaped the model beforehand. A supplier may still keep chats and reuse them as training data later.

Context window

Language models

The most text, counted in tokens, that a language model can take in and keep in view at one time, including its own answer.

Formal

The fixed limit on how many tokens a language model can handle in one go; the prompt, earlier turns of the chat, added documents and the answer must all fit inside it.

In plain English

Like the size of a desk - everything the model is working on has to fit on it, and when it is full, something has to be cleared away to make room.

In practice

A clerk in a region works with an AI assistant through a long afternoon; it starts ignoring the format rules given at the start, because the chat app has cut the oldest messages to keep the rest inside the context window.

Why it matters

It limits how much a model can weigh in one answer, and everything inside it - including text from a file nobody checked - shapes that answer.

Training data

Machine learning basics

The examples a model learns from; its behaviour, its blind spots and its mistakes all come from what is in them.

Formal

The collection of examples, often with the correct answers attached, used during model training to set a model's internal numbers; kept apart from the data later used to test it.

In plain English

Like the textbooks and past exams a student studies from. If they are wrong, one-sided or out of date, so is what the student learns.

In practice

A pension fund wants to train a model on ten years of member emails and must first check which of them hold personal data and on what legal basis they may be used.

Why it matters

Training data is an asset to protect and a source of risk. It can leak personal data, carry unfair patterns, or be quietly changed by an attacker.

Shared connections

Knowledge cutoff