AI supply chain attack

An ML system inherits a longer dependency graph than ordinary software: pretrained base models, fine-tuned derivatives and adapters (LoRA), tokenizers and config files, datasets and their download URLs, conversion and quantisation tools, inference servers, Python packages, and increasingly plugins, MCP servers and agent tools. OWASP LLM03:2025 (Supply Chain) covers tampered or vulnerable third-party components, outdated or deprecated models, unclear licensing, vulnerable LoRA adapters, and weak model provenance on public hubs; NIST AI 100-2 treats supply-chain compromise as an enabler for poisoning and backdoor attacks.

The most direct technical vector is unsafe deserialisation. PyTorch's legacy checkpoint format is a ZIP containing a Python pickle, and unpickling can invoke arbitrary callables through __reduce__, so torch.load on an untrusted .pt/.bin file is remote code execution. Similar issues exist with Keras Lambda layers (CVE-2024-3660) and joblib files. Researchers have repeatedly found malicious models on Hugging Face (JFrog reported around 100 in 2024), and ReversingLabs' 2025 "nullifAI" samples used deliberately broken pickles that evaded Picklescan. Mitigations are to load only weight-only formats such as safetensors, which stores raw tensors plus a JSON header and executes no code, to rely on torch.load(weights_only=True) (the default since PyTorch 2.6), and to run conversion jobs in sandboxes without credentials.

The subtler vector is behavioural: weights that execute no code but have been fine-tuned or edited to contain a backdoor or targeted misinformation. Mithril Security's 2023 PoisonGPT demonstration surgically edited one fact in GPT-J with ROME and uploaded it under a look-alike organisation name, and the model still scored normally on standard benchmarks. Such tampering cannot be found by malware scanning; detection requires provenance and behavioural evaluation. Look-alike and hijacked namespaces on model hubs play the role typosquatting plays in npm and PyPI, and agents that suggest non-existent packages open the related slopsquatting vector.

Controls follow classic software supply chain practice adapted to models: an internal model registry or proxy with an allow-list, pinning models by immutable commit hash rather than a mutable tag or "latest", verifying cryptographic signatures (for example OpenSSF model signing based on Sigstore), recording components in an ML-BOM (CycloneDX 1.5 added machine-learning BOMs; SPDX 3.0 has an AI profile), reviewing model cards and licences, re-running your own evaluation and red teaming on each new version, and treating datasets as versioned, hashed artefacts. Under the EU AI Act, providers of high-risk systems must document third-party components in their technical documentation, and Art. 25(4) requires written agreements with suppliers of AI tools, services, components or processes integrated into a high-risk system.