Hosting apps that hold persistent connections

Request-response hosting has a comfortable property: every request is short and independent, so the platform can restart, reschedule, or scale anything between any two of them. WebSockets remove that property. A connection can last hours, it carries state, and killing it is visible to a user immediately.

That single change ripples through deploys, idle detection, scaling, and where you're allowed to keep state. Here's what actually needs handling.

Deploys sever every connection

With blue-green deploys, a new version boots, gets health-checked, and traffic flips. HTTP clients don't notice. WebSocket clients absolutely do: their connection was to a process that is being torn down, and no amount of routing cleverness moves a live TCP connection between machines.

So the goal isn't to avoid disconnection — it's to make disconnection a non-event.

1. Close deliberately on shutdown. On SIGTERM, send a close frame with a code your client understands before the process exits. A clean close reconnects fast; a dropped TCP connection makes the client wait for a timeout first.
2. Reconnect with backoff and jitter on the client. Without jitter, every client in a deploy reconnects at the same millisecond and your new version takes a thundering herd on its first breath.
3. Make reconnection resumable. The client stores the last message id it saw; on reconnect it sends that id and the server replays what it missed. Without this, a reconnect is a gap in the user's data that nothing reports.
4. Keep a drain window. Let the old version hold existing connections for a period after the flip while the new one takes new ones. Users trickle over instead of moving in one jolt.

// Server: close cleanly so clients reconnect fast instead of timing out
process.on("SIGTERM", () => {
  for (const ws of wss.clients) {
    ws.close(1012, "server restarting");  // 1012 = Service Restart
  }
  server.close(() => process.exit(0));
});

// Client: backoff with jitter, and resume from the last seen message
let attempt = 0;
function connect() {
  const ws = new WebSocket(`${URL}?since=${lastMessageId ?? ""}`);
  ws.onopen = () => { attempt = 0; };
  ws.onclose = () => {
    const base = Math.min(1000 * 2 ** attempt++, 30_000);
    setTimeout(connect, base * (0.5 + Math.random()));  // jitter matters
  };
}

Idle detection gets it wrong

On a platform that sleeps idle apps, the question 'is this app in use?' is normally answered by looking at recent requests. A WebSocket app breaks that in both directions, which is why it's worth thinking about explicitly.

• False idle. A hundred connected users sitting quietly generate no new requests. Naive request-counting sees an idle app and puts it to sleep, disconnecting everyone. This is the dangerous direction.
• False busy. Ping and pong frames every thirty seconds look like continuous traffic, so an app with zero real users never sleeps and you pay for it permanently.

The workable definition is connection-aware: an app with open connections is in use, and an app with zero open connections and no recent requests is idle. Application-level pings that carry no payload shouldn't count as activity, the same way an uptime monitor's health check shouldn't. We ran into exactly this on the HTTP side — automated monitoring traffic was resetting the idle timer and keeping apps awake forever — and the fix was to classify traffic by what it is rather than counting it.

If you're evaluating a platform for a realtime app, this is the specific question to ask: does idle detection look at open connections, or only at request counts?

Everything between you and the client has a timeout

A connection passes through a CDN, a load balancer, a reverse proxy, possibly a corporate middlebox, and each one is willing to close an idle connection. Sixty seconds is a common default and the failure looks maddeningly like an application bug: connections dying at suspiciously round intervals, only for some users.

// Heartbeat below every intermediary's idle timeout, and detect half-open
// connections — a TCP peer that vanished looks identical to a quiet one.
setInterval(() => {
  for (const ws of wss.clients) {
    if (ws.isAlive === false) { ws.terminate(); continue; }
    ws.isAlive = false;
    ws.ping();
  }
}, 25_000);

wss.on("connection", (ws) => {
  ws.isAlive = true;
  ws.on("pong", () => { ws.isAlive = true; });
});

The half-open detection matters as much as the keepalive. A client whose laptop lid closed doesn't send a close frame — the socket stays open on your side, holding memory and appearing in your connection count, until something proves it's gone. On a server with thousands of connections, ghosts accumulate quickly.

State in process memory is a trap

The natural way to write a chat room or a collaborative document is a `Map` of room id to connected sockets, held in memory. It works perfectly with one instance and breaks the moment there are two: user A connects to instance 1, user B to instance 2, and they can't see each other.

There are two honest answers and one bad one.

• Externalise the fan-out. Publish messages to Redis, Postgres LISTEN/NOTIFY, or a message bus; every instance subscribes and forwards to its own local sockets. Instances stay stateless, scaling is normal, and this is the default choice.
• Partition deliberately. Route each room to exactly one instance so all its members share a process — genuinely correct for things like game sessions or collaborative editing with a single authoritative document state, and much simpler than distributed consensus. It requires routing by room id rather than round-robin.
• Sticky sessions as a substitute for either of the above. This is the bad one. Stickiness keeps a client on one instance, which does not help when two clients in the same room landed on different instances, and it makes deploys and scaling worse. It solves connection affinity, not shared state.

For the partitioned model, per-room isolation is where microVMs are a genuinely good fit: each room or session gets its own machine with its own memory, one room's runaway memory usage can't affect another, and the room dies cleanly when everyone leaves. It's the same shape as per-tenant isolation, applied to a session instead of a customer.

Consider not using WebSockets

A significant fraction of realtime features are one-directional: the server pushes updates and the client occasionally posts something. That's Server-Sent Events, which is plain HTTP, reconnects automatically, has resumption built into the protocol via the `Last-Event-ID` header, and passes through proxies without special handling.

// SSE: the browser reconnects on its own and replays from Last-Event-ID
const es = new EventSource("/api/updates");
es.onmessage = (e) => render(JSON.parse(e.data));
// no backoff logic, no ping/pong, no close-code handling

You still get the long-lived-connection problems — deploys, timeouts, idle detection — but you get the reconnection and resumption machinery for free instead of writing it. Reach for WebSockets when you need genuinely bidirectional, low-latency traffic; use SSE when you're pushing a feed and the client mostly listens.

The through-line: persistent connections don't require exotic infrastructure, they require you to stop assuming your process is immortal. Close cleanly, reconnect with jitter, resume from a cursor, keep shared state outside the process, and heartbeat below every timeout between you and the user.