SIEM

The term was coined by Gartner analysts Mark Nicolett and Amrit Williams in 2005, merging two earlier product categories: security information management (SIM), focused on log collection, retention and compliance reporting, and security event management (SEM), focused on real-time correlation and alerting. A SIEM pipeline has five stages. Collection uses agents, syslog (RFC 5424, usually over TLS per RFC 5425 rather than lossy UDP), Windows Event Forwarding, and API pulls from cloud and SaaS services. Parsing and normalisation map vendor-specific fields to a common schema - historically CEF or LEEF, today often the Elastic Common Schema, Microsoft's ASIM or the Open Cybersecurity Schema Framework (OCSF, launched in 2022). Enrichment adds context such as asset owner, user department, geolocation and threat-intelligence matches. Correlation and analytics apply rules and models. Storage tiers (hot, warm, cold or archive) balance query speed against retention cost.

Correlation logic ranges from single-event rules (a new member added to Domain Admins) through threshold and sequence rules (many failed logons followed by a success from the same source within ten minutes) to statistical baselines and user and entity behaviour analytics (UEBA), which flag deviations such as a user downloading far more data than their peer group. Sigma has become a vendor-neutral rule format that can be compiled to the query languages of different SIEMs (KQL, SPL, EQL and others), enabling shared detection content. Increasingly, SIEMs integrate SOAR (security orchestration, automation and response) so that playbooks can enrich an alert, open a ticket, disable an account or isolate a host automatically.

The dominant constraints are economic and operational. Licensing is typically by ingest volume (GB/day) or events per second, which pushes organisations to drop "noisy" sources - often exactly those needed in an investigation. Parsers break silently when a vendor changes its log format, so a source can appear healthy while its fields are empty. Detection engineering needs version control, testing against sample data and coverage mapping to MITRE ATT&CK. A SIEM with default rules and no tuning produces alert volume without insight.

Retention is driven by investigation needs and regulation: incident investigations routinely need months of history because dwell times are measured in weeks, while logs containing personal data must respect GDPR storage limitation. NIST SP 800-92 (2006) is the classic log-management guide, and ISO/IEC 27002:2022 controls 8.15 (logging), 8.16 (monitoring activities) and 8.17 (clock synchronisation) are the usual anchors. A SIEM is a platform, not a function: without a SOC or on-call analysts to triage its output, it is an expensive log archive.