Phishing simulation

A simulation platform - commercial suites, Microsoft Defender for Office 365 Attack Simulation Training, or the open-source GoPhish - sends templated messages with per-recipient tracking tokens embedded in links, attachments or QR codes, and records opens, clicks, credential submissions, attachment opens and reports. Typical payload types mirror real techniques: credential harvest, malware attachment, link in attachment, drive-by URL and OAuth consent grant. Because production defences would otherwise block or detonate the mails, third-party simulations must be explicitly exempted, in Microsoft 365 via the advanced delivery policy rather than blanket transport-rule bypasses, and link scanners must be accounted for, since sandbox detonation produces false clicks that inflate results.

Results are only comparable when difficulty is controlled. The NIST Phish Scale (introduced in 2020, user guide NIST TN 2276, 2023) rates a template by counting observable cues (sender, formatting, technical and content anomalies) and assessing premise alignment, how well the pretext fits the recipient's actual work. A low-cue, high-alignment lure will produce far higher click rates than a clumsy one, so a falling click rate may reflect easier templates, not better staff. Report rate and time-to-report are more robust, and the difference between the two gives a picture of both susceptibility and detection. Small departments, one-off campaigns and seasonal effects add noise that is easily mistaken for trend.

The empirical evidence is sobering. Lain, Kostiainen and Čapkun (IEEE S&P 2022), with more than 14,000 employees over 15 months, found that embedded training shown after a click did not make employees more resilient and could make them more susceptible, while crowd-sourced reporting worked. Ho et al. (IEEE S&P 2025), in a randomised experiment with about 19,500 staff at UC San Diego Health, found no significant relationship between recent annual training and failure rates, and that most users spent a minute or less on embedded training pages. Simulations are thus better used to exercise reporting and to find processes that fail than as proof that training works.

Design and governance matter. Lures offering bonuses, pay rises or pandemic information have caused well-publicised staff backlash, for example at GoDaddy in 2020, and erode trust in the security team. Click data is personal data about employees, so under GDPR the programme needs a lawful basis (usually legitimate interest, Art. 6(1)(f)), transparent information to staff, data minimisation and, preferably, aggregated reporting to management rather than named lists. NIST SP 800-50 Rev. 1 places simulations under experiential learning alongside tabletop and cyber-range exercises, and CIS Controls v8 Control 14 expects training on recognising social engineering attacks.