{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/diffusion-model","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/diffusion-model/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/diffusion-model/"},"term":{"en":"Diffusion model","da":"Diffusionsmodel"},"aka":{"en":["denoising diffusion model"],"da":["støjfjernende diffusionsmodel"]},"domain":["ai"],"cluster":"model-architecture","layer":"model","status":"current","era":2020,"summary":{"en":"The kind of generative AI behind many image makers, which starts from random static and cleans it up step by step into a picture.","da":"Den slags generativ AI bag mange billedgeneratorer, der starter fra tilfældig sne og renser den trin for trin til et billede."},"body":{"formal":{"en":"A deep learning model taught to undo a process that slowly adds random static to real examples; to create something new it begins with pure static and removes a little at each of many steps, guided by a prompt.","da":"En deep learning-model, der er lært at fortryde en proces, som langsomt lægger tilfældig sne over rigtige eksempler; for at skabe noget nyt starter den med ren sne og fjerner lidt i hvert af mange trin, styret af en prompt."},"plain":{"en":"Like a sculptor who sees a figure in a rough block of stone and chips away a little at a time until only the figure is left.","da":"Som en billedhugger, der ser en figur i en grov stenblok og hugger lidt væk ad gangen, til kun figuren er tilbage."},"inPractice":{"en":"A history teacher at a Danish primary school types “a Viking market in Ribe, seen from above, in winter” and within seconds a diffusion model returns four pictures to choose from for her slides.","da":"En historielærer i en folkeskole skriver “et vikingemarked i Ribe set fra oven om vinteren”, og på få sekunder giver en diffusionsmodel hende fire billeder at vælge imellem til undervisningen."},"whyItMatters":{"en":"It made realistic pictures and video cheap for anyone to create, which helps design and teaching but also feeds fake evidence and deepfakes.","da":"Den har gjort realistiske billeder og video billige for alle at skabe, hvilket hjælper design og undervisning, men også giver næring til falske beviser og deepfakes."}},"deepDive":{"en":"A denoising diffusion probabilistic model (DDPM; Ho, Jain and Abbeel, 2020, building on Sohl-Dickstein et al., 2015) defines a fixed forward Markov chain that adds Gaussian noise over T steps: q(x_t | x_{t−1}) = N(√(1 − β_t) x_{t−1}, β_t I). Because Gaussians compose, any step can be sampled directly as x_t = √ᾱ_t x_0 + √(1 − ᾱ_t) ε, where ᾱ_t is the running product of (1 − β_t) and ε is standard normal noise. The DDPM paper used T = 1000 with β rising linearly from 10⁻⁴ to 0.02. A network ε_θ(x_t, t) is trained to predict the added noise with a simple mean-squared error, ‖ε − ε_θ(x_t, t)‖², which is equivalent to a reweighted variational bound and, as Song et al. (2021) showed, to learning the score (gradient of the log-density) of the noised data. Sampling runs the chain backwards from pure noise, each step subtracting predicted noise and adding a little fresh noise.\n\nPractical systems changed almost every piece. DDIM (Song, Meng and Ermon, 2020) gave a deterministic sampler that skips steps, cutting 1,000 steps to a few dozen; later ODE solvers and distillation (for example consistency models) reach one to four steps. Classifier-free guidance (Ho and Salimans) trains the network with and without the conditioning and at sampling time extrapolates, ε̃ = ε(x, ∅) + w · (ε(x, c) − ε(x, ∅)); a guidance scale w above 1 gives stronger prompt adherence at the cost of diversity and, at high values, oversaturated images. Latent diffusion (Rombach et al., 2022), the basis of Stable Diffusion, runs the process in the latent space of a variational autoencoder that downsamples images by a factor of 8 per side, with a U-Net denoiser that reads the text prompt through cross-attention to text-encoder embeddings; newer models replace the U-Net with a transformer (DiT) and often use the closely related flow-matching or rectified-flow objectives.\n\nThe contrast with autoregressive language models is in how output is formed: a diffusion model refines all pixels, audio samples or video frames jointly over many steps, so cost scales with the number of denoising steps rather than output length, and edits such as inpainting fall out naturally by fixing part of the input. Diffusion has also been applied to text, but autoregressive decoders remain dominant there.\n\nKnown problems include memorisation (Carlini et al., 2023 extracted near-copies of training images from Stable Diffusion), weak rendering of text, counts and spatial relations, and inherited bias from web-scraped training data. For provenance, output can be labelled with C2PA content credentials or invisible watermarks, and Art. 50(2) of the EU AI Act requires providers of systems that generate synthetic images, audio, video or text to mark outputs in a machine-readable, detectable way; watermarks, however, can often be removed by cropping, re-encoding or regeneration.","da":"En denoising diffusion probabilistic model (DDPM; Ho, Jain og Abbeel, 2020, med afsæt i Sohl-Dickstein m.fl., 2015) definerer en fast fremadrettet Markov-kæde, der lægger gaussisk støj til over T trin: q(x_t | x_{t−1}) = N(√(1 − β_t) x_{t−1}, β_t I). Fordi gaussiske fordelinger kan sammensættes, kan ethvert trin samples direkte som x_t = √ᾱ_t x_0 + √(1 − ᾱ_t) ε, hvor ᾱ_t er det løbende produkt af (1 − β_t), og ε er standardnormalfordelt støj. DDPM-artiklen brugte T = 1000 med β stigende lineært fra 10⁻⁴ til 0,02. Et netværk ε_θ(x_t, t) trænes til at forudsige den tilføjede støj med en simpel kvadratisk fejl, ‖ε − ε_θ(x_t, t)‖², hvilket svarer til en omvægtet variationsgrænse og, som Song m.fl. (2021) viste, til at lære scoren (gradienten af log-tætheden) for de støjede data. Sampling kører kæden baglæns fra ren støj, hvor hvert trin trækker forudsagt støj fra og lægger lidt ny støj til.\n\nPraktiske systemer har ændret næsten alle dele. DDIM (Song, Meng og Ermon, 2020) gav en deterministisk sampler, der springer trin over og skærer 1.000 trin ned til nogle få dusin; senere ODE-løsere og destillation (fx consistency models) når ned på ét til fire trin. Classifier-free guidance (Ho og Salimans) træner netværket både med og uden betingelsen og ekstrapolerer ved sampling, ε̃ = ε(x, ∅) + w · (ε(x, c) − ε(x, ∅)); en guidance-skala w over 1 giver tættere overensstemmelse med prompten på bekostning af variation og ved høje værdier overmættede billeder. Latent diffusion (Rombach m.fl., 2022), grundlaget for Stable Diffusion, kører processen i det latente rum hos en variational autoencoder, der nedsampler billeder med en faktor 8 pr. side, med en U-Net-støjfjerner, der læser tekstprompten via cross-attention til embeddings fra en tekst-encoder; nyere modeller erstatter U-Net med en transformer (DiT) og bruger ofte de nært beslægtede træningsmål flow matching eller rectified flow.\n\nForskellen fra autoregressive sprogmodeller ligger i, hvordan outputtet dannes: En diffusionsmodel forfiner alle pixels, lydsamples eller videobilleder samlet over mange trin, så prisen følger antallet af støjfjerningstrin frem for outputlængden, og redigering som inpainting følger naturligt af, at en del af inputtet holdes fast. Diffusion er også afprøvet på tekst, men dér dominerer autoregressive decodere stadig.\n\nKendte problemer er memorering (Carlini m.fl., 2023 udtrak næsten-kopier af træningsbilleder fra Stable Diffusion), svag gengivelse af tekst, antal og rumlige forhold samt nedarvet bias fra træningsdata skrabet fra nettet. Til proveniens kan output mærkes med C2PA-indholdslegitimationer (content credentials) eller usynlige vandmærker, og AI-forordningens art. 50, stk. 2, kræver, at udbydere af systemer, der genererer syntetiske billeder, lyd, video eller tekst, mærker outputtet i et maskinlæsbart og detekterbart format; vandmærker kan dog ofte fjernes ved beskæring, genkodning eller regenerering."},"edges":[{"type":"requires","to":"ai/deep-learning","confidence":"high","strength":"normal"},{"type":"kind-of","to":"ai/generative-ai","confidence":"high","strength":"normal"},{"type":"contrasts-with","to":"ai/large-language-model","why":{"en":"A diffusion model shapes the whole output at once over many cleaning steps; a large language model writes one token after another.","da":"En diffusionsmodel former hele outputtet på én gang over mange rensningstrin; en stor sprogmodel skriver ét token efter det andet."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/synthetic-data","why":{"en":"Diffusion models are used to make extra realistic images to train other models when real examples are scarce.","da":"Diffusionsmodeller bruges til at lave ekstra realistiske billeder til at træne andre modeller, når rigtige eksempler er få."},"confidence":"medium","strength":"normal"},{"type":"used-with","to":"ai/prompt","confidence":"high","strength":"normal"},{"type":"used-with","to":"ai/gpu","confidence":"high","strength":"normal"}],"depth":2,"sources":[{"title":"Ho, Jain & Abbeel (2020), Denoising Diffusion Probabilistic Models","tier":"reference"},{"title":"Sohl-Dickstein et al. (2015), Deep Unsupervised Learning using Nonequilibrium Thermodynamics","tier":"reference"}],"draft":true}