all posts

The best ways to host MLflow in 2026

Ajay Kumar··11 min read

MLflow is the closest thing machine learning has to a default, and it is also one of the most commonly misdeployed pieces of infrastructure in the field. Not because it is hard to run — pip install mlflow followed by mlflow server gets you a working UI in about a minute — but because that one-minute command quietly makes four decisions on your behalf, and every one of them is the decision you would not make for something you intend to keep.

The reason teams get the hosting question wrong is that "host MLflow" sounds like "host a web app." It isn't. MLflow is three services wearing one name, and the hosting question is almost entirely about two of them — neither of which is the part with the UI. Then there is a second question, hiding in the model registry, that doesn't look like a hosting question at all until the day it becomes an incident.

This is a comparison of the realistic places to put MLflow, what each one costs you in operational attention, and where the managed option is simply the right answer. I build a compute platform, so I have a horse in this race; I've tried to be useful rather than promotional, and I'll say plainly where we're a bad fit. Everything I say about other products is qualitative on purpose — verify the specifics against their current documentation, because this whole category moves.

MLflow is three services wearing one name

Before comparing hosts, be precise about the shape of the thing you are hosting. A production MLflow deployment is:

  • The tracking server. A Python web application — Flask behind Gunicorn — serving the REST API your client library talks to and the UI you stare at during a training run. It holds no state of its own. It is, genuinely, the boring part, and it is the part everyone spends their planning time on.
  • The backend store. A real relational database holding experiments, runs, parameters, metrics, tags, and the model registry's versions, aliases and stage transitions. In any serious deployment this is Postgres. It is tiny in bytes and total in consequence: it is the entire record of what you tried, what happened, and which artifact is the current champion.
  • The artifact store. Object storage holding the things runs actually produce: serialized models, checkpoints, plots, confusion matrices, HTML reports, and the occasional dataset someone logged "just for reproducibility" that turned out to be forty gigabytes. This is where your storage bill lives.

Run mlflow server with no arguments and you get a SQLite file in the working directory as the backend store and a local ./mlruns directory as the artifact store. Both are excellent defaults for a laptop. Both are, on a server, a data-loss incident with a start date. The interesting hosting decisions are entirely about replacing those two defaults with things that survive the machine.

The single most common self-hosted MLflow failure is not an outage. It is discovering, months in, that the tracking server has been writing to a SQLite file and a local directory on a machine that was never in the backup policy because it was "just the MLflow box."

The backend store: small, chatty, and unrecoverable

MLflow owns this schema and migrates it with its own Alembic revisions, applied by mlflow db upgrade. That is a mundane fact with an operational consequence: an MLflow version bump is a database migration, and it needs the same care you would give any other production migration. Teams that treat the tracking server as a stateless container they can roll forward casually eventually roll forward into a schema their old replicas cannot read.

The write pattern is not what people expect either. Nothing here is large, but there is a lot of it. Metric logging is fundamentally one row per metric per step, so a training loop that logs loss and four other metrics every step for twenty epochs writes an unglamorous mountain of tiny rows. Turn on autologging and a framework will happily log more than you asked for. This is not a scaling crisis — it is a normal Postgres workload — but it is why SQLite stops being cute the moment two training jobs run at once, and why "database is locked" shows up in a training log that took nine hours to produce.

Here is the sentence to internalise before choosing a host. If you lose the backend store, the artifact store becomes a bucket full of anonymously named UUID directories. The artifacts are the payload; the backend store is the index. A bucket without its index is not a backup, it is archaeology — and you will be the archaeologist, on a Friday, trying to work out which of four hundred model.pkl files was the one in production.

The artifact store: large, dumb, and quietly expensive

S3, GCS, Azure Blob, or anything S3-compatible. It can also be a local filesystem or an NFS mount, and the fact that it can is responsible for a great deal of human suffering. Put it in object storage, in the same region as everything else, and stop thinking about it.

There is one real architectural choice here, and it is worth understanding before you pick a host: how clients reach the bucket. In the direct mode, the tracking server hands the client an artifact URI and the client talks to object storage itself — which means every training box needs bucket credentials. In the proxied mode, enabled with --serve-artifacts and an --artifacts-destination, uploads and downloads flow through the tracking server, so training boxes hold one MLflow credential and nothing else.

Proxied access is genuinely nicer for access control, and it is the mode most teams should want. It also does something that undermines the comfortable claim that the tracking server is a small stateless web app: it turns that app into a data plane streaming multi-gigabyte checkpoints through a Gunicorn worker. Every host option below is affected by this, and the failure mode is always the same — a large upload dies and the error message is about your proxy's body-size limit, not about MLflow.

Finally: nothing prunes the artifact store for you. mlflow gc exists and permanently deletes artifacts for runs that were already soft-deleted in the backend store — note the dependency, again, on the database knowing what is safe to remove. If you never set a retention policy, your artifact bill grows monotonically forever, which is a fun thing to explain in a cost review.

The tracking server is the least interesting part of the problem

With the two stores externalised, the tracking server becomes genuinely disposable: a Python process with some workers, holding nothing. You can run two of them, you can roll them, you can delete one mid-afternoon and nobody notices. "How do I make MLflow highly available" mostly resolves into "how do I make Postgres and my bucket highly available," plus a second container, plus a load balancer that does not truncate large uploads.

The one thing that is genuinely underbuilt is authentication. MLflow ships a basic auth capability, and it is basic in both senses. Almost every team that runs this in production puts an identity-aware proxy, an OAuth2 proxy, or their existing SSO gateway in front of it and treats MLflow's own auth as defence in depth rather than the front door. Plan for that as part of the hosting decision rather than discovering it after the URL has been shared in a channel with four hundred people in it.

#!/usr/bin/env bash
# The three stores, made explicit. Every flag below replaces a default that is
# fine on a laptop and an incident on a server.
set -euo pipefail

# 1. BACKEND STORE -- experiments, runs, params, metrics, tags, registry.
#    Small in bytes, total in consequence. Postgres, not SQLite: several
#    training jobs writing metrics concurrently is precisely the workload
#    SQLite is worst at, and "database is locked" is a miserable line to find
#    in a training log that took nine hours to produce.
export MLFLOW_BACKEND_URI="postgresql+psycopg2://mlflow:${PGPASSWORD}@db.internal:5432/mlflow?sslmode=require"

# 2. ARTIFACT STORE -- models, checkpoints, plots. Where the terabytes go.
export MLFLOW_ARTIFACT_ROOT="s3://acme-ml-artifacts/mlflow"

# Run the schema migrations deliberately, once, from one place, with a database
# snapshot already taken -- not implicitly at boot from three replicas racing
# each other through the same Alembic revisions.
mlflow db upgrade "$MLFLOW_BACKEND_URI"

mlflow server \
  --backend-store-uri "$MLFLOW_BACKEND_URI" \
  --artifacts-destination "$MLFLOW_ARTIFACT_ROOT" \
  --serve-artifacts \
  --host 0.0.0.0 \
  --port 5000 \
  --workers 4

# --serve-artifacts proxies artifact upload and download THROUGH this process,
# so training boxes carry one MLflow credential instead of bucket credentials.
# Lovely for access control. It also means this "stateless web app" is now
# streaming multi-gigabyte checkpoints through itself. Size it accordingly, and
# check the body-size limit and read timeout on whatever proxy sits in front,
# because a failed 6 GB upload produces an error message about nginx.

The half people get wrong: the registry is a code distribution channel

Now the second question, which nobody puts on the hosting spreadsheet.

The MLflow Model Registry is a beautiful idea. Register a model version, give it an alias, and let every downstream system load "the current champion" by name without knowing anything about paths or buckets. One line: mlflow.pyfunc.load_model("models:/fraud-detector@champion"). It is the kind of API that makes a platform team's week.

Look at what that line does. It resolves the alias against the backend store, fetches the artifacts, reads the MLmodel manifest, and hands the payload to a flavour loader. For the sklearn and generic pyfunc paths, that loader unpickles — usually via cloudpickle. And pickle is not a data format. It is a small stack machine whose instruction set includes "import this module and call this callable with these arguments." Deserializing a pickle is executing a program that the pickle chose.

I've written the long version of that argument elsewhere — the __reduce__ primitive, what weights_only=True does and does not fix, why safetensors is real progress and still not a boundary — and it is linked at the bottom rather than re-derived here. The short version is that there is no flag which makes loading an untrusted pickle safe, because executing code is what the format is for.

MLflow then adds a second path that gets discussed much less. A logged model carries its own environment declaration — python_env.yaml, requirements.txt, conda.yaml — and mlflow models serve will by default materialise that environment before serving. Which means resolving and installing packages whose names were chosen by the artifact. The artifact gets a vote on what is installed on the machine that serves it, and pip install is not a spectator sport.

# MLmodel -- the manifest at the root of a logged model's artifacts.
artifact_path: model
mlflow_version: "2.x"
model_uuid: 8f1e2c4a9b7d4e1fa0c3d5e6f7089abc
flavors:
  python_function:
    loader_module: mlflow.sklearn
    python_version: 3.11.9
    env:
      conda: conda.yaml
      virtualenv: python_env.yaml
    model_path: model.pkl      # <-- a pickle, i.e. a program the artifact wrote
  sklearn:
    sklearn_version: 1.5.1
    serialization_format: cloudpickle

---
# python_env.yaml -- shipped alongside it. mlflow models serve builds this
# environment before serving unless you pass --env-manager local.
python: 3.11.9
build_dependencies:
  - pip==24.2
dependencies:
  - -r requirements.txt        # <-- package names chosen by the artifact

# So "serve the champion" can mean: resolve an alias, download a blob, install
# whatever that blob's requirements file names, then execute a pickle. Four
# steps, three of which run code the registry never inspected.

So a model serve is up to four things, and only the last one is inference:

  1. Resolve the alias or version against the backend store and fetch the artifacts from the artifact store.
  2. Materialise the declared environment — pip or conda resolving package names the artifact supplied — unless you pass --env-manager local, which skips this step and keeps the next one.
  3. Deserialize the model object. For pickle-based flavours this is arbitrary code execution by construction, not by vulnerability.
  4. Actually run inference, which is the only step anybody was thinking about.
"Load whatever is in the registry and serve it" is a remote code execution path with a friendly UI, and it is not a bug — it is what the feature does. The registry can authenticate who is allowed to write a model version. It cannot constrain what that version does when it is loaded. Those are two different questions and only the first one has a setting.

Whether that matters depends entirely on who can write into your registry. If it is four people on one team logging models built from a repo they all review, this is theoretical and you should not spend a cent on it. It stops being theoretical the moment any of these are true: you accept models from customers, tenants, or a marketplace; you run a platform where users train and register their own models; a CI pipeline registers models built from a repo where any contributor can change the training script; or you re-register community checkpoints into your internal registry, which quietly makes a stranger's serialization your production artifact.

The other half of the risk is not the code, it is where it runs. Model loading conventionally happens inside your serving deployment: a pod carrying an IAM role that can read the artifact bucket, a service account token, a database URI in its environment, and network reach to the feature store and half your internal namespace. The pickle does not need a clever exploit. It has been handed a shell in the most privileged part of your ML stack and asked politely to only do inference.

There are two independent decisions here, and conflating them is why hosting comparisons for MLflow tend to be unsatisfying. Where the tracking server and its two stores live is a straightforward infrastructure choice. Where registry artifacts get deserialized is a security boundary question, and you can answer it differently — on any of the platforms below.

The realistic hosting options

1. Databricks-managed MLflow

Stated first because a lot of self-hosting is reflexive, and this is the option that removes the most work. The people who maintain MLflow run it; the backend store and artifact store are theirs, backed up, migrated and version-matched together; permissions, lineage and workspace identity are integrated rather than bolted on with a proxy. If your data already lives in Databricks, the gravity is real, and fighting gravity is a hobby rather than a strategy.

The underrated win is version skew simply disappearing. Self-hosters learn — usually at an inconvenient moment — that client version, server version and schema revision form a compatibility surface, and that the mlflow db upgrade nobody ran is why the UI started returning 500s after a routine image bump. On a managed service that is somebody else's Tuesday.

Reasons not to: hard data-residency or air-gap requirements; wanting to query the tracking database directly for your own reporting, which a managed product may not expose; or simply not being a Databricks shop and not wanting to become one in order to have an experiment tracker. On the serving side, if you use their model serving, ask what the isolation boundary is for your tier and get the answer from their current documentation rather than from a blog post — "whose code runs next to mine" is a fair question to put to any managed serving product, including this one.

2. Self-hosted on Kubernetes

The right answer if you already run Kubernetes and have somewhere to put stateful things. A Deployment for the tracking server, an external managed Postgres, bucket access through workload identity or IRSA so there are no long-lived keys in a Secret, an Ingress with SSO in front. Community Helm charts exist and vary enormously in quality; read the templates before you apply them, because you are adopting somebody's opinions about your database connection pooling.

Two things bite reliably. The first is the artifact data plane: with --serve-artifacts on, your ingress controller is now in the path of every checkpoint upload, and the defaults for body size and read timeout on most controllers were not chosen with 8 GB files in mind. The second is migrations at boot — N replicas each running mlflow db upgrade in an init container, racing each other through Alembic. Run migrations as a Job, once, before the rollout, and make the app containers assume the schema is already correct.

For serving, Kubernetes gives you a pod per model, which is a container sharing the node's kernel with every other pod scheduled there. That is entirely appropriate for models your own team trained. It is a thin wall if the artifact came from outside, because a container is ultimately a polite suggestion to a kernel you are also asking to keep the rest of your workloads safe.

3. One plain VM

More defensible than its reputation. A systemd unit, Gunicorn, a managed Postgres somewhere else, a bucket somewhere else, and an hour of your life instead of a quarter. A well-run single VM beats a badly-run Kubernetes deployment on every axis that matters, including the one where you can explain it to a new hire.

The non-negotiables are exactly the two defaults from earlier: the backend store must not be a SQLite file on that box, and the artifact store must not be a directory on that box. Get those right and the machine becomes replaceable from a script, which is the whole point. Get them wrong and you have built a pet with your entire experimental history in its bladder.

What you are accepting: one host is your availability story, and there is no per-model isolation unless you build it. Fine for a team's internal tracker. Not fine when the registry feeds a customer-facing prediction service.

4. Container PaaS plus managed Postgres plus a bucket

Where most production deployments actually converge, and for good reason. Push an image or a repo, get a URL and TLS, point it at a managed Postgres and object storage, and let three different vendors handle three different sets of backups. More expensive in dollars, dramatically cheaper in attention, and the tracking server genuinely becomes the disposable thing it always claimed to be.

Two watch-outs. The artifact proxying problem returns in a new costume: many PaaS platforms have request body limits and hard request timeouts you cannot raise, and a 4 GB checkpoint upload is a long-lived request. Test that before you migrate, not after. And most container platforms bill a floor whether or not anyone is using the service, which matters for an internal tracker with a usage pattern of "hammered for two hours during a training run, untouched for the next fourteen."

Serving is typically a second service on the same platform, isolated by the platform's container boundary. Ask the vendor what that boundary actually is; the answers range from a shared-kernel container to a per-tenant virtual machine, and the marketing language for both is identical.

5. PandaStack (where we fit, and where we don't)

Since I build one of these, let me be specific rather than enthusiastic. The fit is three things.

First, the tracking server as a git-driven app with a managed Postgres backend store next to it. Push the repo, get a URL, get a database. The part I actually care about is what the database gives you operationally: clone and point-in-time restore. Before an MLflow version bump, clone the backend store, run mlflow db upgrade against the clone, and look at the result — the migration stops being a thing you find out about in production. And when someone runs a well-meaning cleanup script against the wrong experiment, point-in-time restore is the difference between an afternoon and a permanent hole in your experimental record. Creating one takes 30 to 90 seconds, because it is a real Postgres being bootstrapped rather than a row in a table.

Second, scale-to-zero. An experiment tracker is the most boringly bursty internal service in existence: saturated during a training run, then untouched overnight while everyone sleeps. On a platform where idle costs nothing, the tracker sleeps and wakes on the next request. This is not a headline feature, it is just the difference between paying for a service twenty-four hours a day and paying for the hours anyone used it.

Third — and this is the part the rest of this post has been building towards — per-load hardware isolation for the model-loading path. A Firecracker microVM with its own guest kernel, its own network namespace, no cloud role, no registry write token, and a TTL so it deletes itself even if your orchestrator forgets. Creating one is a snapshot restore rather than a boot: roughly 179ms at p50 and 203ms at p99, with the genuine three-second cold boot happening once when the template is baked. That number matters for exactly one reason — it is what makes "a fresh machine per artifact load" affordable enough that nobody quietly starts reusing one to hide the latency.

Where we are a bad fit, plainly. We do not offer managed object storage, so the artifact store is your bucket and you point at it — same as most of the options above, but I would rather you know before you plan a migration. Firecracker is not a GPU story: this shape covers the CPU-side work — loading, scanning, format conversion, metadata extraction, quarantine decisions, and light CPU inference — not your accelerator fleet. And if you already live in Databricks, their managed MLflow is the boring correct answer and I am not going to pretend otherwise to win a comparison table.

At a glance, on the five dimensions that decide this

Same five questions, asked of every option. Everything here is qualitative by design — check current pricing, limits and isolation claims against each vendor's own documentation, because all of it moves.

  • Databricks-managed MLflow — Backend store: theirs, managed, version-matched to the server; you generally don't get direct SQL access to it. Artifact store: theirs, integrated with workspace permissions. Idle cost: a platform floor; check current pricing for your tier. Model-serving isolation: their serving product's boundary — ask them what it is for your tier rather than assuming. Ops burden: lowest on this list by a wide margin, which is the entire point.
  • Self-hosted on Kubernetes — Backend store: yours; bring a managed Postgres and run migrations as a Job, not from an init container in every replica. Artifact store: your bucket, ideally via workload identity so no keys sit in a Secret. Idle cost: the cluster, always, plus the tracking Deployment. Model-serving isolation: a pod per model — a container on a shared node kernel; adequate for your own models, thin for anyone else's. Ops burden: moderate if you already run Kubernetes, absurd if you would be adopting it for this.
  • One plain VM — Backend store: yours, and it must be an external managed Postgres, not a SQLite file on the box. Artifact store: your bucket, and emphatically not a local directory. Idle cost: one instance running permanently, which is small and non-zero. Model-serving isolation: none unless you build it; the serving process is a process on the same host. Ops burden: low to run, entirely on you when something breaks, and the box is your availability story.
  • Container PaaS plus managed deps — Backend store: a managed Postgres from your cloud or a database vendor, with someone else's backups. Artifact store: your bucket. Idle cost: a per-service floor on most platforms whether or not anyone opens the UI. Model-serving isolation: the platform's container boundary — verify whether that means shared-kernel containers or per-tenant VMs. Ops burden: lowest of the self-managed options; the tracking server becomes genuinely disposable.
  • PandaStack — Backend store: managed Postgres with clone and point-in-time restore, so migrations get rehearsed on real data and a bad cleanup script is recoverable. Artifact store: your bucket — we don't offer managed object storage, and that is a real gap. Idle cost: scale-to-zero; an overnight-idle tracker costs nothing while it sleeps. Model-serving isolation: a Firecracker microVM per load with its own guest kernel, own network namespace and a TTL — the strongest boundary here for untrusted artifacts, and CPU-only. Ops burden: low for the tracking and database tiers; the bucket and any GPU serving remain yours.

Loading a registry model without handing it your cluster

The pattern that works is to stop treating "load the model" as an ordinary function call inside your privileged serving process, and start treating it as a job you dispatch to a machine you are willing to lose. Not because you expect every artifact to be hostile — almost none are — but because the cost of being wrong once is unbounded, and the cost of being right about the topology is roughly two hundred milliseconds.

import json

from pandastack import Sandbox

# The URI came from the registry, which tells us who wrote it and nothing
# whatsoever about what it does when loaded.
MODEL_URI = "models:/fraud-detector@champion"

LOADER = '''
import json, sys
import mlflow

# We do NOT let the artifact's python_env.yaml choose what gets installed --
# the template already carries the runtime. This is the --env-manager local
# posture: it removes the pip step and keeps the unpickling one, which is why
# the unpickling happens inside a disposable machine instead of a pod.
model = mlflow.pyfunc.load_model(sys.argv[1])   # <-- arbitrary code runs HERE

meta = model.metadata
sig = meta.signature
print(json.dumps({
    "flavors": sorted(meta.flavors.keys()),
    "mlflow_version": meta.mlflow_version,
    "signature": sig.to_dict() if sig else None,
}))
'''

sbx = Sandbox.create(
    template="base",
    ttl_seconds=900,  # backstop: the guest reaps itself if our process dies
    metadata={"kind": "mlflow-load", "model_uri": MODEL_URI},
)
try:
    sbx.filesystem.write("/work/inspect.py", LOADER)

    # What crosses the boundary: a tracking URI and a read-only, single-prefix,
    # short-lived artifact credential. What does not: your serving IAM role,
    # the backend store's connection string, the registry write token, or a
    # mounted model cache shared with anything else.
    res = sbx.exec(
        "cd /work && "
        f"MLFLOW_TRACKING_URI={TRACKING_URI} "
        "AWS_ACCESS_KEY_ID=$ARTIFACT_RO_KEY "
        "AWS_SECRET_ACCESS_KEY=$ARTIFACT_RO_SECRET "
        f"python inspect.py '{MODEL_URI}'",
        timeout=300,
    )
    if res.exit_code != 0:
        raise RuntimeError(f"quarantine {MODEL_URI}: load failed\n{res.stderr}")

    # DATA comes back out, parsed by us. Never import what the guest produced,
    # never exec the helpful conversion script it thoughtfully generated.
    report = json.loads(res.stdout)
    promote_if_clean(MODEL_URI, report)
finally:
    sbx.kill()

Three things are load-bearing here and none of them is the sandbox itself. The credential set is scoped down to "read this one artifact prefix." The result comes back as JSON that your side parses, never as code your side runs — if the guest emits a helpful conversion script, you read it, you do not execute it. And there are two independent stop conditions: the exec timeout for a load that hangs, and the create-time TTL for the case where your own process dies between spawning the guest and reaping it. Neither depends on the other being correct.

Egress is the other half. Because each sandbox gets its own Linux network namespace — the platform pre-allocates 16,384 /30 subnets per host so this is the default shape rather than a special request — the egress policy is a host-side rule the guest cannot argue with. Allow the artifact store prefix and DNS. Deny the rest. The difference between a malicious pickle being a contained annoyance and being an incident is almost always whether it could reach the network to tell anyone what it found.

If you are serving rather than inspecting, the same guest becomes the serving process and gains a property worth having: snapshot it once the model is deserialized and in memory, and every subsequent replica is a restore of that already-loaded process rather than a fresh execution of the pickle. You pay for the untrusted deserialization once, deliberately, in a place you chose — and then scale out from a frozen, known state. Copy-on-write memory means the tenth replica does not copy gigabytes to exist.

Treat the backend store like the production database it is

Everything above assumes you got this part right, so here it is as a list rather than an implication. None of it is exotic; all of it is skipped regularly because the database is labelled "MLflow" instead of "production."

  • Postgres from day one. Not "SQLite until we're bigger" — the migration from SQLite happens exactly when you are busiest, and MLflow's own tooling does not move your data for you.
  • Rehearse every mlflow db upgrade on a clone. A migration that runs in two seconds on a demo dataset can take a great deal longer against a table with a row per metric per step per run, and you want to learn that on a copy.
  • Point-in-time restore, tested. The realistic disaster here is not hardware, it is a delete-experiments script pointed at the wrong workspace. Backups you have never restored are a feeling, not a plan.
  • A retention policy, decided early. Soft-delete old runs and let mlflow gc reclaim the artifacts, or accept that storage grows forever. Deciding this at month two is trivial; deciding it at year two is a project.
  • A least-privilege database user. The tracking server needs DML on its own schema; it does not need to be a superuser, and the migration job's credentials do not need to be the runtime's.
  • Connection pool sizing that accounts for autologging. Frameworks log more than you asked them to, and the tracking server's pool is the thing between a chatty training loop and a Postgres running out of connections.
# The tracking server as a git-driven app; the backend store as a managed
# Postgres. Two calls, no cluster, and nothing on a local disk.
curl -sS -X POST https://api.pandastack.ai/v1/databases \
  -H "Authorization: Bearer $PANDASTACK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"label": "mlflow-backend", "size": "4g"}'
# 202 -> poll GET /v1/databases/{id}. Expect 30-90s: it is a real Postgres
# being bootstrapped, not a row appearing in a control-plane table.

curl -sS -X POST https://api.pandastack.ai/v1/apps \
  -H "Authorization: Bearer $PANDASTACK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "mlflow-tracking",
    "git_url": "https://github.com/acme/mlflow-server",
    "git_branch": "main",
    "port": 5000,
    "auto_deploy": true,
    "env": {
      "MLFLOW_BACKEND_URI": "postgresql+psycopg2://...",
      "MLFLOW_ARTIFACT_ROOT": "s3://acme-ml-artifacts/mlflow"
    }
  }'

# Before an MLflow version bump: migrate a COPY first. A clone is a real
# database with your real row counts, so "will this Alembic revision finish
# before anyone notices" stops being a question you answer live, on a
# Thursday evening, with the training team watching.
curl -sS -X POST https://api.pandastack.ai/v1/databases/$DB_ID/clone \
  -H "Authorization: Bearer $PANDASTACK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"label": "mlflow-backend-premigration"}'

Choosing

  • Already on Databricks, or your objection to managed is a preference rather than a requirement: use their managed MLflow and spend the saved quarter on the models.
  • One team, internal, models you trained yourselves: a plain VM or a container PaaS, with managed Postgres and a bucket. Do not build a platform for this.
  • Production tracker several teams depend on, and you already run Kubernetes: the Helm route, with migrations as a Job and an ingress configured for large uploads.
  • You want it near your workloads with no idle bill, and the tracker sleeps most of the day: a microVM platform for the app and database tiers, your own bucket for artifacts.
  • Anyone outside your team can write into the registry — customers, tenants, a marketplace, an open contributor base: whatever you choose for the tracking server, do not load registry artifacts inside your privileged serving process. That is the decision that actually matters, and it is independent of every row above.

The short version

MLflow hosting is a stateful-dependency question in an app-hosting costume. The tracking server will run anywhere and is not worth arguing about; the backend store determines whether your experimental history survives, and the artifact store determines your bill. Decide who operates Postgres before you decide where the web process goes, and everything downstream follows.

Then decide the question the hosting spreadsheet does not have a column for: who can put a model version into your registry, and what happens on the machine that loads it. If the answer to the first is "only us," relax and pick on convenience. If the answer is anything broader, the load is untrusted code execution by construction, and the only durable fix is to give it a machine of its own — one with its own kernel, no credentials worth stealing, and a deletion time already set.

Frequently asked questions

What does a production MLflow deployment actually need?

Three things that people conflate into one. A tracking server, which is a Flask application behind Gunicorn serving the REST API and UI and holding no state of its own. A backend store, which should be Postgres, holding experiments, runs, parameters, metrics, tags and the model registry — small in bytes and the thing you genuinely cannot lose. And an artifact store, which should be S3-compatible object storage, holding models, checkpoints and plots. Running mlflow server with no flags gives you SQLite and a local directory instead of the last two, which is correct for a laptop and a data-loss incident on a server.

Should I use SQLite or Postgres for the MLflow backend store?

Postgres, from the first day you run MLflow on a server rather than a laptop. Metric logging writes a row per metric per step, so concurrent training jobs produce exactly the pattern SQLite handles worst, and "database is locked" appearing at hour nine of a training run is a bad way to learn this. The migration from SQLite to Postgres also tends to become urgent precisely when you are busiest, and MLflow's own tooling does not move your data for you. Beyond concurrency, a managed Postgres gets you backups, point-in-time restore and the ability to rehearse an mlflow db upgrade on a clone before running it for real.

Is it safe to serve a model straight from the MLflow Model Registry?

It is safe if and only if you trust everyone who can write a model version into it. Loading a pyfunc or sklearn model deserializes a pickle, and pickle is a code format rather than a data format — deserializing it executes instructions the artifact chose. On top of that, mlflow models serve will by default build the environment declared in the artifact's own python_env.yaml or conda.yaml, meaning the artifact influences what gets pip-installed on the serving machine. --env-manager local removes the install step but not the unpickling one. The registry controls who may write a version; it cannot constrain what that version does when loaded, and no flag makes an untrusted pickle safe.

How do I isolate MLflow model loading from my production infrastructure?

Move the load off your privileged serving process and onto a machine you are willing to lose. In practice that means a fresh microVM or equivalent per artifact load, carrying only a short-lived read-only credential scoped to the artifact prefix — no serving IAM role, no backend store connection string, no registry write token — with an egress policy that allows the artifact store and nothing else, and a TTL so the machine reaps itself if your orchestrator crashes. Results must come back as data your side parses, never as code your side executes. On PandaStack a create is a snapshot restore at roughly 179ms p50, which is what makes a per-load machine affordable rather than something you batch and compromise on.

Is managed MLflow worth it, or should I self-host?

Self-host when you have a concrete reason: data residency, air-gapped operation, wanting direct SQL access to the tracking database for your own reporting, or an existing platform team that will genuinely own it. Absent one of those, the managed option removes an entire class of work — version skew between client, server and schema; Alembic migrations; artifact-proxy timeouts; putting SSO in front of a service whose built-in auth is minimal. Self-hosting is very reasonable and quite cheap once the backend store and artifact store are externalised; the trap is treating a stateful, migration-carrying service as a stateless container because it happens to have a web UI.

Keep reading

Run code in a microVM in one API call.

49ms p50 cold start. Fork, snapshot, and scale to zero.

Start free
Written by Ajay Kumar, Founder, PandaStack.