Decoder vs Encoder

Why they differ

An encoder reads the whole input in both directions to understand it; a decoder only looks back and writes new tokens one by one.

Decoder

Model architectures

The half of a model that writes output one token at a time, each new token based only on what came before it.

Formal

A stack of neural network layers in which each token may look only at earlier tokens, so the stack can guess the next token, add it to the text and repeat; decoder-only designs drop the encoder entirely.

In plain English

Like a storyteller by a campfire who can hear everything said so far but never peeks ahead, adding one word, then the next.

In practice

A product owner at a webshop sees that product texts twice as long take about twice as long to appear and cost twice as much, because the decoder must produce every token in turn.

Why it matters

Almost every large language model is decoder-only, which is why replies arrive gradually and why the length of an answer, not just the question, drives its cost.

Encoder

Model architectures

The half of a model that reads the whole input at once and turns it into embeddings that capture its meaning.

Formal

A stack of neural network layers that takes a full sequence of tokens and, letting every token look both forwards and backwards, produces one embedding per token for later steps to use.

In plain English

Like an interpreter who hears the whole sentence before saying anything, because a word at the very end can change what the first one meant.

In practice

At a district heating company, an IT developer sorts incoming customer emails with an encoder-only model such as BERT, which reads each message whole and labels it “bill”, “fault” or “moving house”.

Why it matters

Encoders handle search and sorting cheaply but cannot write new text, so knowing the difference lets you pick a smaller, faster tool when the job is only to understand text.

Shared connections

Attention mechanism Token Transformer