Custom domains and automatic TLS, done properly

At some point a customer asks to use their own domain. It sounds like a DNS record and a certificate, and the demo version genuinely is. The production version has a security property that's easy to miss and expensive to miss: without it, one tenant can claim a hostname belonging to another and start receiving their traffic.

I built this for PandaStack apps and a review pass caught fourteen real bugs before it shipped, domain squatting among them. This is the design, the failure modes, and the ordering that makes them impossible rather than unlikely.

First, the wildcard certificate constraint

Before custom domains, you have a per-app hostname on your own domain covered by a wildcard certificate. That works, with one rule people design around too late: a wildcard covers exactly one label.

`*.example.com` matches `app-123.example.com`. It does not match `app-123.apps.example.com`. Every extra dot needs its own wildcard, and multi-level wildcards don't exist. So if your URL scheme has an infix — `<id>.apps.` or `<port>-<id>.preview.` — you either issue more certificates or restructure the hostname.

We serve apps at the apex of the suffix, `<app-id>.<suffix>`, specifically so a single-level wildcard covers them. That's not elegance, it's the certificate deciding the URL scheme. Learn it before building the router rather than after.

The flow, in the order that matters

Five steps. The ordering is the security design.

1. The tenant claims a hostname. Record it as pending. It must not be routable yet.
2. The tenant proves control — a TXT record at a name you specify, or a CNAME to your endpoint. The proof value is unique per claim and unguessable.
3. You verify it by resolving DNS yourself, server-side. Only now does the hostname become active for that tenant.
4. You obtain a certificate for it, typically via ACME with an HTTP or DNS challenge, and store it for the router to serve.
5. Your edge terminates TLS, reads the SNI hostname from the handshake, looks up which app owns it, and proxies. Renewal runs on a timer well ahead of expiry.

Verification, and the mistakes in it

TXT-record verification is the standard approach: you generate a random token, the tenant publishes it at a known name under their domain, you resolve and compare. Four details separate a correct implementation from a plausible one.

• Resolve server-side, from your infrastructure. Anything the client reports about DNS is a claim, not evidence.
• Make the token unguessable and single-use. A predictable token means anyone can pre-publish it.
• Re-verify periodically. Domains change hands, and a hostname verified two years ago may now belong to someone else — one of the ways subdomain takeover happens.
• Cap the number of pending claims per tenant, and expire them. Unbounded pending claims are a memory-exhaustion vector and a way to squat the namespace, which is precisely the pair of bugs review caught in ours.

Tenant adds:
  _pandastack-challenge.app.acme.com.  TXT  "ps-verify=9f2c...unique-per-claim"
  app.acme.com.                        CNAME <app-id>.pandastack.app.

Platform, server-side:
  resolve TXT _pandastack-challenge.app.acme.com
  constant-time compare against the stored token for THIS claim
  on match  -> mark verified, create the routing entry, request a cert
  no match  -> stay pending; NEVER create a routing entry

Periodically:
  re-resolve; if the record is gone, mark the domain unverified
  and remove the route -- domains change owners

Certificates at multi-tenant scale

One certificate per customer domain, issued automatically and renewed automatically. Conceptually simple; operationally it's the part that pages you at 3am if you get it wrong.

• Rate limits are real. Public ACME providers cap issuance per domain and per account per week. A retry loop on a failing domain will consume your budget and block issuance for unrelated customers. Back off exponentially and stop retrying a domain that has failed repeatedly.
• Renew early. Thirty days before expiry, not three. Renewal fails for boring reasons — the customer changed DNS, their firewall blocks the challenge — and you want weeks of runway to notice and email them.
• Store certificates where every edge node can read them, and make sure a node that restarts loads current material rather than something stale from disk.
• Handle the handshake miss deliberately. A TLS connection for an unknown hostname must fail cleanly, not serve another tenant's certificate — which is both a mismatch error for the user and an information leak about who else you host.
• Monitor expiry independently of the renewal system. If renewal is broken, renewal will not tell you. An external check that alerts on any served certificate within fourteen days of expiry costs nothing and has saved every team that has one.

SNI routing: hostname to backend

At the edge, the request arrives, TLS begins, and the client's SNI extension names the hostname before any HTTP is exchanged. Your edge selects the certificate for that hostname, completes the handshake, then looks up which backend owns the hostname and proxies to it.

Two properties matter. The lookup must be fast and cached, because it's on every connection. And it must be updated atomically on deploys, since the backend behind a hostname changes every time a blue-green flip happens — the mapping updates in one step, and a request either goes to the old machine or the new one, never to a half-updated entry.

The same mechanism works beyond HTTP. We route managed Postgres connections by SNI too: `<id>.db.pandastack.ai` terminates TLS at a proxy that maps the hostname to a database VM. Same idea, different protocol.

The interaction with sleeping apps

If your platform sleeps idle apps, custom domains interact with that in a way that will surprise you. A custom domain is public, so it attracts the full internet background radiation: scanners, crawlers, certificate-authority validation probes, and whatever is looking for WordPress admin panels this month.

If any of that resets the idle timer, apps on custom domains never sleep. We shipped exactly that. The initial suspicion fell on the domain machinery, which was innocent — the cause was automated traffic keeping the timer warm, and the fix was a classifier distinguishing traffic that keeps an app awake from traffic that may wake a sleeping one. Those are separate decisions: a scanner should do neither, or you've given the internet a button that boots your infrastructure.

The checklist

If you're building this, these are the properties I'd hold a review to.

1. A hostname is never routable before verification succeeds.
2. Verification tokens are random, per-claim, and checked server-side.
3. Pending claims are capped per tenant and expire.
4. Verification is re-checked periodically, and losing it removes the route.
5. Certificate issuance backs off and stops retrying persistently failing domains.
6. Renewal starts weeks before expiry, with independent external expiry monitoring.
7. An unknown SNI hostname fails cleanly rather than serving someone else's certificate.
8. The hostname-to-backend mapping updates atomically on deploy.
9. Automated traffic to a custom domain does not silently defeat idle sleeping.

Nine bullets, several days of work, and one of them is the difference between a feature and a vulnerability disclosure. The first one is that bullet.