Convolutional neural network (CNN) vs Transformer

Why they differ

A CNN only looks at small nearby patches at a time and builds up; a transformer lets every part of the input look at every other part from the start.

Convolutional neural network (CNN)

Model architectures

A neural network built for pictures - it slides small pattern checks across an image to find edges, then shapes, then whole objects.

Formal

A neural network whose early layers apply the same small set of learned checks at every position of a grid-like input such as an image, so a pattern is found wherever it appears; later layers combine these into larger features.

In plain English

Like sweeping a small magnifying glass over a photo, first spotting lines, then eyes and noses, and finally saying “that is a face”.

In practice

A municipality's roads department fits cameras to its refuse lorries; a CNN picks out potholes and cracks in the images, so the road inspector gets a daily list of spots to repair.

Why it matters

CNNs made computers good at reading images, from phone face unlock to medical scans, and they remain a cheap, fast choice where a transformer would be more than the task needs.

Transformer

Language models

The neural network design behind today's language models, which weighs how every word in a text relates to every other word.

Formal

A neural network design from 2017 built around "attention", where the model scores how much each token should draw on every other token, and handles all tokens at once rather than one after another.

In plain English

Like reading a whole page at a glance and drawing lines between the words that belong together, instead of reading one word at a time.

In practice

When a ministry's translation tool handles “the bank refused the loan because it was too risky”, the transformer links “it” to “the loan”, so the Danish translation uses the word for “it” that fits the loan, not the bank.

Why it matters

Because it can be trained quickly on huge amounts of text, it made large language models possible - and with them most of today's AI tools.

Shared connections

Neural network