What is MLSecOps, and what are the main threats across the ML lifecycle?
MLSecOps extends security practices across the whole ML lifecycle rather than just the deployed app, covering data, training, the model artifact, and serving. Key threats include data and model poisoning, adversarial evasion inputs, model theft or extraction, membership-inference and privacy leakage, and supply-chain risks like malicious model files and dependencies. Defenses span provenance and validation, robustness testing, access control and signing of artifacts, input monitoring, and scanning, integrated into the MLOps pipeline.
How to think about it
The short answer
MLSecOps extends security across the entire ML lifecycle — data, training, the model artifact, and serving — not just the app wrapping the model. The main threats map to lifecycle stages: poisoning, evasion, theft/extraction, privacy leakage, and supply-chain attacks. Defenses are controls embedded into the MLOps pipeline.
Threats by lifecycle stage
- Data stage — poisoning: malicious/mislabeled training data creating backdoors or bias.
- Training/artifact — supply chain: malicious model files (pickle code execution), tampered dependencies, compromised pretrained weights from public hubs.
- Serving — adversarial evasion: crafted inputs that fool the model at inference (e.g., perturbations that flip a classifier).
- Serving — model theft / extraction: querying an API to reconstruct or clone the model.
- Serving — privacy / membership inference: probing whether a specific record was in the training set, or extracting memorized data.
The OWASP Secure AI Model Ops guidance frames protecting the lifecycle from development through production.
Defenses (mapped)
- Provenance + validation for data; dataset versioning for auditability.
- Sign and verify artifacts, scan model files, prefer safetensors, pin dependency hashes.
- Access control & secrets hygiene around the registry and serving infra.
- Robustness testing (adversarial/invariance tests) in CI.
- Input monitoring & rate limiting to blunt extraction and evasion at serving time.
- Privacy techniques (differential privacy, output filtering) where memorization is a risk.
Concrete example
A fraud model is exposed via API. An attacker rate-floods it with probing queries to reverse-engineer the decision boundary (extraction), then crafts transactions that evade it. Rate limiting, anomaly detection on query patterns, and adding adversarial examples to training raise the cost of both attacks.
Common follow-up / trap
A frequent probe: “How is this different from normal AppSec?” Classic AppSec secures code and infra; MLSecOps adds threats unique to learned behavior and data — poisoning, evasion, extraction, membership inference — that don’t exist in ordinary software. The trap is securing only the endpoint while ignoring the data and training supply chain, which is where the most damaging attacks originate.