ML security (MLSecOps)
ML systems have an attack surface software doesn't: poisoned data, stolen models, malicious model files, and a fragile supply chain. The threats and the defenses that belong in your pipeline.
What you'll learn
- The ML-specific attack surface — poisoning, theft, evasion, supply chain
- Why loading a random model file is as risky as running a random script
- Defenses that belong in an MLOps pipeline
Before you start
Standard application security still applies to ML systems — but they add an attack surface ordinary software doesn’t have. The data can be poisoned, the model can be stolen or fooled, and the model file itself can carry malware. MLSecOps is folding these ML-specific threats into your security practice, and OWASP now maintains dedicated entries for them.
The ML attack surface
- Data & model poisoning — an attacker slips crafted samples into the training data to plant a backdoor: the model behaves normally until it sees a secret trigger, then misclassifies. OWASP tracks this as a top ML/LLM risk; the “PoisonGPT” demo made it concrete.
- Model theft / extraction — by querying your prediction API enough, an attacker reconstructs a copy of your model (stealing the IP), or infers whether a specific record was in the training set (a privacy leak).
- Evasion / adversarial inputs — small, deliberate perturbations to an input (imperceptible to humans) that flip the prediction — the classic “sticker on a stop sign” attack.
- Supply chain — malicious dependencies, or a malicious model file. This is the sharpest practical risk for MLOps teams.
The pickle problem
Defenses that belong in the pipeline
- Secure the supply chain — pin and scan dependencies, verify model provenance/signatures, and maintain an AI bill of materials (AIBOM) — the same SBOM/SLSA discipline as software, extended to data and models.
- Validate training data — data validation tests and data contracts catch anomalous samples before they poison a model.
- Lock down the model API — rate-limit and monitor prediction endpoints to make model extraction and membership-inference impractical.
- Govern artifacts — only load models that passed through the
registry with verified lineage; never
pip installa random model server and point it at production.
Quick check
Quick check
Next
Security and cost and governance round out the platform layer. Next: the data-quality gate — data contracts.
Practice this in an interview
All questionsMLSecOps extends security practices across the whole ML lifecycle rather than just the deployed app, covering data, training, the model artifact, and serving. Key threats include data and model poisoning, adversarial evasion inputs, model theft or extraction, membership-inference and privacy leakage, and supply-chain risks like malicious model files and dependencies. Defenses span provenance and validation, robustness testing, access control and signing of artifacts, input monitoring, and scanning, integrated into the MLOps pipeline.
No single fix is complete, so defenses are layered: separate trusted instructions from untrusted data, constrain and least-privilege the tools and actions the model can take, validate and sanitize inputs and tool outputs, add output guardrails and injection classifiers, and keep a human in the loop for sensitive actions. Treat all external or retrieved content as untrusted.
The ML lifecycle spans eight phases: problem framing, data collection and validation, feature engineering, training and experimentation, offline evaluation, deployment, production monitoring, and retirement or retraining. Each phase has distinct owners, artefacts, and failure modes that an MLOps practice must systematise.
LLMOps extends classical MLOps to handle foundation model scale, prompt-based configuration, non-deterministic outputs, and evaluation without a scalar ground truth. Key new concerns include prompt versioning, output quality evaluation via LLM judges or human review, hallucination monitoring, cost management, and RAG pipeline observability.