Model registry & promotion

At 3:07 a.m., a customer reports that the churn dashboard says one thing and the production API says another. Two engineers search Slack. One finds model.pkl attached to an old ticket. Another finds a notebook named final_final_v4. Nobody can answer the only question that matters: which model is serving customers right now, and why was it allowed to serve them?

A model registry exists for this moment.

MLflow tracking answers, “What did we try?” A model registry records named model versions and release pointers, and can link each version to lineage and approval evidence. A surrounding governance and deployment workflow must enforce approvals, permissions, auditability, and production access. Managed platforms may integrate those capabilities; MLflow OSS generally requires external IAM, CI/CD, policy, and deployment controls.

Without one, production model identity lives in filenames, deployment scripts, and memory. All three are poor databases.

What a registry actually tracks

A registry is more than a folder for model files. It stores model-version metadata and release pointers, such as aliases or stages, alongside links to the artifact store and experiment tracker. It does not, by itself, enforce every approval, permission, audit, or deployment rule; those controls usually live in CI/CD, IAM, policy systems, deployment platforms, or managed integrations.

For a churn system, you might register one logical model called churn-classifier. That logical model has versions:

• Version 17 points to one specific trained artifact and tracked run.
• Version 18 points to another artifact and run.
• Each version has its own metadata, evaluation evidence, and release history.

The important word is specific. Version 18 should point to artifact bytes that your storage and release process make immutable, not to “whatever file currently sits at s3://models/churn/latest.” A registry version ID does not make object-store contents immutable. Use write-once or content-addressed artifacts where appropriate, and store or verify a checksum or digest so changed bytes cannot quietly serve under the same identity.

A useful version record contains, or links to:

• The model artifact and format.
• The source experiment run, parameters, and metrics.
• The training-data snapshot.
• The code commit, preprocessing and feature definitions, and runtime image.
• Evaluation reports, model cards, risk assessments, and approval evidence.

A registry cannot recover lineage that the training pipeline never recorded. Registering a file later does not reveal which data produced it. Log the run first, then register the resulting artifact with its lineage. For dataset snapshots and model artifacts, see data and model versioning.

The registry gives the model a stable name with separate numbered identities. A service can ask for models:/churn-classifier@champion instead of hard-coding “version 17.” That means: resolve the champion pointer for this named model, then load the version it identifies.

An alias is only a pointer. Unless an external approval workflow and access policy protect its updates, any sufficiently privileged client may move it. The service need not change its code for every new model; an authorized release process changes the pointer, and deployment decides when to reconcile it.

Stages, aliases, and the naming trap

Older MLflow installations commonly use stages such as None, Staging, Production, and Archived. A stage describes a broad lifecycle position and remains common in existing systems.

For new designs, aliases are usually more precise. An alias is a movable label attached to one version. champion can point to version 17 today and version 18 tomorrow. challenger can point to the candidate being tested, and shadow to a model receiving copied requests without deciding customer outcomes.

Aliases describe a role; stages describe a workflow state. Those are not always the same. Two production candidates might receive 90 percent and 10 percent of traffic, which a single Production stage cannot express. Aliases identify the models, while the deployment system controls the split.

For new MLflow work, aliases are the safer default; stages remain useful in older registries and integrations. Either way, production code resolves a controlled release pointer instead of guessing which file is current.

That separation prevents an approval click from silently restarting 200 servers. It also lets you test deployment without changing governance, and review a model without granting the reviewer infrastructure access.

A worked promotion: the better average that should not ship

Suppose version 17 is the current champion for a subscription business. The model predicts which customers are likely to cancel in the next 30 days, and a retention team uses the prediction to decide whom to call.

Version 18 looks better in the headline report:

• The fixed evaluation set contains 20,000 customers, including 2,000 who churned.
• Version 17 catches 1,540 churners: recall is 1,540 divided by 2,000, or 77 percent.
• Version 18 catches 1,580 churners: recall is 79 percent.
• The overall gain is 2 percentage points.

A hurried team promotes version 18. A careful gate asks one more question: where did the gain come from?

Among 400 churners who are new customers, version 17 catches 160, or 40 percent. Version 18 catches only 120, or 30 percent. Among existing customers, version 18 improves substantially, so the aggregate score rises. The model is better on average and worse for a group the business may care about.

A sensible promotion policy might require all of the following:

1. The candidate beats the incumbent on the fixed evaluation set.
2. No monitored customer slice loses more than 5 percentage points of recall.
3. Input schema and feature-freshness checks pass.
4. Evaluation uses the approved data snapshot and code commit.
5. A reviewer signs the model card and relevant risk assessment.

Version 18 fails rule two. It remains registered but does not become the champion. Nothing is deleted: the evidence stays attached so the team can inspect the failure, improve the data, or deliberately accept the trade-off later.

This is the difference between a registry and a leaderboard. A leaderboard says which number is largest. A registry records which candidate was allowed to take a real role, under which rules and evidence.

The gate can automate repeatable checks and require a human for risk-sensitive decisions. A model card describes intended use, evaluation results, limitations, and known risks. A bias audit checks performance or outcomes across relevant groups rather than only across the full population. Neither makes a model safe alone; both make reasoning visible and reviewable.

Try the gate:

The useful output is not just “pass” or “fail.” A failed gate should say which rule failed, which data and code it evaluated, and where the report is stored. “Model rejected” is an error message. “New-customer recall fell from 40 percent to 30 percent on evaluation snapshot customers-2026-08-12” is a decision.

The release pattern in production

A dependable release has four separate records:

1. Training record: run, data snapshot, code commit, parameters, metrics, and artifact.
2. Registry record: numbered version of a named model linked to that artifact.
3. Approval record: automated tests and human reviews deciding whether it may receive an alias or stage.
4. Deployment record: where and when the approved version was loaded, plus its traffic share.

The transition usually looks like this: training creates version 18; evaluation attaches reports and checks; a reviewer approves it; an authorized release step moves challenger to version 18; and deployment rolls it out to a small slice or shadow environment. Observability checks error rate, latency, feature validity, and business metrics. If healthy, deployment expands traffic and the authorized workflow moves champion to version 18.

Some teams move the alias before deployment; others deploy the exact approved version first and move the alias after health checks. Either can work if both states are recorded clearly. What must not happen is allowing deployment to resolve “latest” independently from the approved registry version.

Every serving process should expose the resolved model name, version, run identifier, and artifact digest in logs or a health endpoint. “The churn API is slow” is not enough. “Pod churn-7f8 loaded churn-classifier version 17, digest abc123, at 02:54 UTC” turns a Slack search into a diagnosis.

Failure modes you will actually see

The alias moved, but the API still serves the old model

First symptom: the registry says champion points to version 18, but API predictions still match version 17.

Usually the service resolved the alias at startup, loaded version 17, and never received a reload or restart signal. Make alias resolution part of a deliberate deployment operation: log the resolved version at startup, roll the service when the approved pointer changes, and verify the loaded version through a health endpoint. Do not resolve on every request; that adds latency and makes registry availability part of prediction availability.

Someone overwrote an artifact behind a stable name

First symptom: a rollback produces different predictions, or its digest no longer matches the deployment log.

A path such as models/churn/latest.pkl is a mutable pointer with no trustworthy history. Use immutable versioned artifacts and verify checksums or content digests during promotion and deployment. A rollback should select the old registered version, not whatever currently answers to “latest.”

Production bypasses the registry

First symptom: a server is running a model absent from the registry’s release history.

A deployment script may accept a raw object-store path, a notebook may copy a file directly to a host, or a developer may change a container image without recording the model identity. Restrict production deployment permissions, require a registered version or approved alias, and reject unregistered artifacts at startup. Deployment logs should include the registry identity and digest.

The evidence is missing

First symptom: weeks later, nobody can reproduce the metric that justified promotion. The version has a name but no data snapshot, code commit, threshold, or slice report.

Make promotion depend on required metadata. The gate should fail when the dataset version, code revision, evaluation report, or runtime information is absent. This is where model lineage and data contracts become practical rather than decorative.

Two releases race to become champion

Two pipelines can read the same old state and both attempt an update. Version 18 may be approved, then version 19 approved, while the alias ends on an unexpected version. Serialize promotion decisions, require an explicit reviewed version, and make the update auditable. “Latest successful pipeline” is not a sufficient approval rule.

The honest cost

A registry introduces another service, a permissions model, a metadata schema, and a failure surface. Teams must agree on what counts as a version, what evidence is mandatory, and how aliases change.

That cost is worthwhile when a wrong model can lose money, harm a customer, trigger a compliance investigation, or waste a week of debugging. It is often excessive for a private notebook or one-off batch analysis. Record immutable artifacts and lineage early, but add a formal gate when there is a real release decision to govern.

Keep the registry out of the hot prediction path. Resolve and authorize the model during deployment, load it locally, and monitor the resulting process. The registry is the release ledger, not a low-latency feature database.

In one breath

A registry gives a model a durable name and version identity. Immutable artifacts, lineage, access controls, and an approval workflow turn that identity into reliable release control. Aliases such as champion and challenger move release roles without changing application code. A promotion gate checks evaluation, data, runtime, risk, and approval evidence; the registry records the resulting pointer; and a separate deployment system rolls it out, records what loaded, and controls traffic and rollback. If production can bypass that workflow, or the registry can point to mutable bytes, you have a catalogue rather than release control.

Practice

Take the churn example and write the release record needed to reconstruct the decision six weeks later: data snapshot, code revision, model version, evaluation report, approval, and deployment event.

Why is “version 18 is approved” not the same as “version 18 is serving 10 percent of traffic”? The first is a governance and registry decision. The second is a deployment fact. A reliable system records both.

A question to carry forward

The promotion gate is only as trustworthy as the evidence behind it. “F1 is 0.91” does not tell you whether evaluation data leaked training examples, a small but important slice collapsed, serving features match training, or the model still works after six weeks of changing traffic.

The next question is larger than model accuracy: how do you test the data, model, pipeline, and serving system together, so the gate has evidence worth trusting? That is ML testing and the ML Test Score.