ML security (MLSecOps)

The last lesson defended against harm the model causes by behaving exactly as trained — an unfair decision, an unexplained denial. But we ended by naming the threat a bias report can’t see: someone attacking the model on purpose. That adversary is this lesson. Where responsible-AI ops asks “is what the model honestly does acceptable?”, ML security asks the opposite question — “what happens when someone is actively trying to corrupt, steal, or weaponize it?”

Standard application security still applies to ML systems — but they add an attack surface ordinary software doesn’t have. The data can be poisoned, the model can be stolen or fooled, and the model file itself can carry malware. MLSecOps is folding these ML-specific threats into your security practice, and OWASP now maintains dedicated entries for them.

The ML attack surface

• Data & model poisoning — an attacker slips crafted samples into the training data to plant a backdoor: the model behaves normally until it sees a secret trigger, then misclassifies. OWASP tracks this as a top ML/LLM risk; the “PoisonGPT” demo made it concrete.
• Model theft / extraction — by querying your prediction API enough, an attacker reconstructs a copy of your model (stealing the IP), or infers whether a specific record was in the training set (a privacy leak).
• Evasion / adversarial inputs — small, deliberate perturbations to an input (imperceptible to humans) that flip the prediction — the classic “sticker on a stop sign” attack.
• Supply chain — malicious dependencies, or a malicious model file. This is the sharpest practical risk for MLOps teams.

The pickle problem

Defenses that belong in the pipeline

• Secure the supply chain — pin and scan dependencies, verify model provenance/signatures, and maintain an AI bill of materials (AIBOM) — the same SBOM/SLSA discipline as software, extended to data and models.
• Validate training data — data validation tests and data contracts catch anomalous samples before they poison a model.
• Lock down the model API — rate-limit and monitor prediction endpoints to make model extraction and membership-inference impractical.
• Govern artifacts — only load models that passed through the registry with verified lineage; never pip install a random model server and point it at production.

In one breath

ML adds four attack vectors ordinary software doesn’t have: poisoning (crafted training samples that plant a backdoor — normal behavior until a secret trigger fires), theft/extraction (probing the prediction API to clone the model or infer who was in the training data), evasion (imperceptible adversarial perturbations that flip a prediction), and supply chain (malicious dependencies or a model file that runs code on load — pickle is as dangerous as a random shell script); the defenses are security discipline applied to ML artifacts — safe formats (Safetensors/ONNX), pinned-and-scanned dependencies with provenance/signing (an AIBOM), data-validation gates, locked-down APIs, and loading only registry-verified models — and the pragmatic order is to stop the practical attacks first, not every exotic one.

Practice

Before the quiz, sit with the sharpest practical risk the lesson names: the model file itself. Explain, to a teammate about to pip install a random model server and point it at production, why loading an untrusted checkpoint is “exactly as dangerous as running a random shell script,” and name the one format choice that defuses it. Then prioritize: of the four attack vectors, which two would you harden first for a typical production API, and why those over chasing adversarial-perturbation defenses?

A question to carry forward

That closes MLOps. Look back at the whole arc: a model is 5% of the work, and the other 95% is the loop you built around it — a lifecycle, baselines, tracking, containers, CI/CD, serving, monitoring, retraining, a platform, and now its guardrails of cost, governance, and security. You can take a model from a notebook to a system that keeps working, safely and accountably, in production.

But every single lesson in this chapter quietly stood on something it never had to build: data, already cleaned, joined, and waiting. The feature store assumed a table of features; the training job assumed a dataset; the retraining window assumed rows by the million arriving on schedule. We have spent a whole section operating models and never once asked how the data that feeds them gets processed — and at the scale these systems run, that is not a small question. A real training set isn’t a CSV; it’s terabytes that won’t fit on one machine, let alone in pandas. So the question to carry forward, out of MLOps and into the next section, is the one underneath everything we just built: when the data is too big for a single computer, how do you process it at all? That is big data, and the section that answers it — Hadoop, the cluster, and Spark — begins with the most basic question of all: what does “big data” actually mean?