Question 1

What does experiment tracking solve, and how do MLflow and Weights and Biases differ in practice?

Accepted Answer

Experiment tracking captures the full reproducibility context of a training run — code version, hyperparameters, dataset hash, environment, and metrics — so any result can be reproduced and compared. MLflow is an open-source, self-hosted lifecycle platform; Weights and Biases is a hosted, collaboration-first product with richer real-time visualisation.

Question 2

Walk me through the full ML lifecycle from problem definition to model retirement.

Accepted Answer

The ML lifecycle spans eight phases: problem framing, data collection and validation, feature engineering, training and experimentation, offline evaluation, deployment, production monitoring, and retirement or retraining. Each phase has distinct owners, artefacts, and failure modes that an MLOps practice must systematise.

Question 3

How does autoscaling work for ML inference services, and what metrics should drive it?

Accepted Answer

ML inference services should scale on request queue depth or GPU utilization rather than CPU utilization alone, because GPU-heavy workloads keep CPU near-idle even under full load. Horizontal Pod Autoscaler in Kubernetes can be configured with custom metrics, and scale-to-zero with a warm-up buffer prevents cold-start latency spikes.

Question 4

What are the differences between batch, online, and streaming inference, and when should you use each?

Accepted Answer

Batch inference runs predictions on large datasets on a schedule, optimizing for throughput. Online inference serves individual requests in real time, optimizing for low latency. Streaming inference processes continuous event streams with bounded latency requirements between the two extremes.

Question 5

How does CI/CD for ML differ from standard software CI/CD, and what stages should an ML pipeline include?

Accepted Answer

ML CI/CD must validate not just code correctness but also model quality — automated retraining triggers, data validation, model evaluation gates, and canary deployment checks that standard software pipelines have no equivalent for. A regression in model AUC is as much a deployment failure as a 500 error.

Question 6

What is the difference between data drift, concept drift, and label drift — and how do you detect each?

Accepted Answer

Data drift is a change in the statistical distribution of model inputs; concept drift is a change in the relationship between inputs and the target; label drift is a shift in the marginal distribution of the target itself. They require different detectors and carry different business urgency.

Question 7

How do Docker and ONNX complement each other for packaging and deploying ML models portably?

Accepted Answer

Docker encapsulates the full runtime environment — OS libraries, Python version, system packages — so the model runs identically everywhere. ONNX provides a hardware- and framework-agnostic model format so a model trained in PyTorch can be executed by a high-performance runtime like ONNX Runtime without the training framework as a dependency.

Question 8

What is a model registry, and how does model versioning work in production ML systems?

Accepted Answer

A model registry is a centralised store that tracks every trained model artifact alongside its metadata — hyperparameters, training data version, evaluation metrics, and lineage. Versioning assigns unique identifiers to each artifact and manages lifecycle stages so teams can promote, roll back, and audit models without manual file management.

Question 9

How do you safely roll back a model in production and what triggers a rollback?

Accepted Answer

A rollback reverts serving traffic to a known-good model version when the newly deployed model shows metric regression beyond a tolerance threshold. Safe rollback requires versioned model artifacts, traffic-routing control, and pre-defined automated or manual triggers — not ad hoc decisions under pressure.

Question 10

What are the security and compatibility risks of using pickle for model serialization, and what are the safer alternatives?

Accepted Answer

Pickle executes arbitrary Python bytecode during deserialization, so loading an untrusted pickle file is equivalent to running arbitrary code on your machine. Beyond security, pickle artifacts are tightly coupled to the exact Python and library versions used to create them, making them fragile across environments.

Question 11

What metrics should you monitor for a production ML model, and at what layer?

Accepted Answer

Production ML monitoring spans four layers: data quality (schema, distributions, null rates), model behaviour (prediction drift, confidence calibration), operational health (latency, error rate, throughput), and business KPIs (conversion, revenue impact). Each layer has different owners and different alert thresholds.

Question 12

How do you achieve reproducibility in ML training pipelines — covering seeds, environment, and data versioning?

Accepted Answer

Full ML reproducibility requires locking three layers: the random seed across all frameworks, the software environment via pinned dependency manifests or container images, and the training data via content-addressed versioning. Missing any one layer means the same code can produce different models on different runs or machines.

Question 13

When would you choose gRPC over REST for model serving, and what are the practical trade-offs?

Accepted Answer

gRPC uses HTTP/2 and Protocol Buffers to deliver lower latency, strongly typed contracts, and built-in streaming, making it the better choice for high-throughput internal model services. REST remains the standard for public-facing APIs where broad client compatibility and human-readable payloads matter more than raw performance.

Question 14

What is the difference between shadow deployment and canary deployment for ML models, and when do you use each?

Accepted Answer

Shadow deployment mirrors live traffic to the new model and discards its predictions, so you can evaluate performance and load without any user impact. Canary deployment routes a small real slice of traffic to the new model and uses its predictions, so real user impact is possible but limited and monitored.

Question 15

What is a feature store and why is it critical for production ML systems?

Accepted Answer

A feature store is a shared data platform that computes, stores, and serves ML features consistently for both training and serving. It eliminates training-serving skew by ensuring the same transformation code runs in both contexts, and it reduces duplicated work by letting teams share and discover features across models.

Question 16

When and how should you trigger model retraining — scheduled vs. event-driven?

Accepted Answer

Scheduled retraining is simple and predictable but wastes compute when nothing has shifted and reacts slowly when drift is sudden. Event-driven retraining ties compute to evidence — a drift alarm, a performance threshold breach, or a data volume trigger — and is more efficient at scale. Most mature systems combine both.

Question 17

Why does a model that performed well in offline evaluation degrade in production?

Accepted Answer

Production degradation stems from distributional shift between training and serving data, upstream pipeline changes, feedback loops, and the static nature of a trained model against a changing world. Offline evaluation on a held-out slice of historical data cannot simulate these dynamics.

Question 18

How do you optimise GPU utilization for model serving, and what role does dynamic batching play?

Accepted Answer

GPUs execute tensor operations efficiently only when the batch dimension is large enough to saturate all CUDA cores. Dynamic batching collects individual requests arriving within a short window and fuses them into a single GPU call, dramatically improving throughput and cost efficiency without sacrificing per-request latency beyond the configured wait threshold.

Question 19

How do you monitor a model when ground-truth labels are delayed or never arrive?

Accepted Answer

When true labels are unavailable or arrive weeks late, you monitor leading indicators instead: input distribution drift, output score distribution shift, proxy business metrics, and inter-model disagreement. These act as early-warning signals before any labelled evaluation becomes possible.

Question 20

How do you balance latency and throughput trade-offs when designing a model serving system?

Accepted Answer

Latency is the time to serve a single request; throughput is the number of requests served per second. They are in tension because batching requests improves GPU utilization and throughput but adds queuing delay. The design goal is to meet the latency SLA at the highest possible throughput.

Question 21

How does LLMOps differ from classical MLOps, and what new operational challenges do LLMs introduce?

Accepted Answer

LLMOps extends classical MLOps to handle foundation model scale, prompt-based configuration, non-deterministic outputs, and evaluation without a scalar ground truth. Key new concerns include prompt versioning, output quality evaluation via LLM judges or human review, hallucination monitoring, cost management, and RAG pipeline observability.

Question 22

A model is live and you cannot get labels quickly. How do you set up alerting to catch performance problems early?

Accepted Answer

Without labels, alerting relies on three proxy signal layers: input distribution tests, output score distribution tests, and business proxy metrics. You define thresholds on each layer pre-deployment and set up automated alerts so that degradation triggers investigation before it compounds.

Question 23

What is train/serve skew and how do you prevent it?

Accepted Answer

Train/serve skew occurs when the feature values a model sees at training time differ from those it sees at inference time, even for the same raw input — caused by divergent preprocessing code paths, different data sources, or temporal leakage. It silently degrades performance without raising obvious errors.