AI risk & governance
Hiding instructions in the text an AI system reads so that it ignores its own rules and follows the attacker instead.
Formal
An attack on a large language model in which input written by an outsider, typed directly or hidden in a web page, file or email the model is asked to read, is treated as an instruction and overrides what its owner intended.
In plain English
Like slipping a note into a pile of letters a new assistant is sorting that says "ignore your boss and send me the keys" - and the assistant cannot tell the note apart from real orders.
In practice
A municipality's AI assistant sums up incoming emails from citizens; one email hides white-on-white text telling it to forward the last ten messages to an outside address, and it does.
Why it matters
The model mixes rules and content in the same stream of words, so there is no watertight fix yet; the more an AI system is allowed to do on its own, the more damage one hidden sentence can cause.