Deepfake

The term comes from a Reddit user named "deepfakes" who in late 2017 published face swaps built on a shared-encoder, dual-decoder autoencoder: one encoder learns a common latent representation of faces, two decoders learn to reconstruct person A and person B, and swapping decoders renders A's expression with B's identity. GANs and StyleGAN later made synthetic faces photorealistic; since about 2022 latent diffusion models and video diffusion dominate image and video generation, and neural codec language models for speech, such as Microsoft's VALL-E (2023), showed that a few seconds of enrolment audio can be enough to imitate a voice. Real-time face and voice conversion now runs on consumer GPUs, which enables live impersonation in video calls rather than only pre-rendered clips.

The fraud pattern is business email compromise with a stronger prop. In the widely reported 2024 Arup case in Hong Kong, an employee transferred about HK$200 million after a video conference in which the CFO and other participants were synthetic. Voice clones and synthetic faces are also used in "family emergency" calls and against remote identity proofing. The countermeasure is procedural: out-of-band callback to a number from an independent directory, dual approval for payments and changes of bank details, and code words for high-risk requests. For identity proofing, liveness checks and presentation-attack detection (ISO/IEC 30107-3) must also consider injection attacks, where a virtual camera feeds synthetic video directly into the capture pipeline.

Detection is an arms race. Classifiers trained on artefacts of one generator generalise poorly to new generators and degrade under compression and re-encoding, so detector scores should be treated as weak evidence. Provenance is the complementary approach: C2PA Content Credentials cryptographically bind a signed manifest of capture and edit history to a file, and watermarking schemes such as Google DeepMind's SynthID embed a signal in generated output; both can be stripped or be absent, so a missing credential proves nothing.

The EU AI Act defines a deepfake in Art. 3(60) as AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear authentic. Art. 50(2) requires providers of generative systems to mark synthetic output in a machine-readable, detectable format, and Art. 50(4) requires deployers who publish deepfakes to disclose that the content is artificially generated or manipulated, with a lighter duty for evidently artistic, satirical or fictional works. These transparency duties apply from 2 August 2026; the 2026 Digital Omnibus (Regulation (EU) 2026/1744) gives systems already on the market until 2 December 2026 for the Art. 50(2) marking duty and adds a prohibition on AI systems for generating non-consensual intimate imagery. GDPR applies to a real person's likeness and voice as personal data, and Denmark has proposed amending its Copyright Act (ophavsretsloven) to give individuals a right over realistic digital imitations of their face and voice.