Your Scale-to-Zero App Never Sleeps, and Bots Are Why

A customer opened a ticket that I have since learned is a genre. Their app was set to scale to zero after fifteen idle minutes. Their invoice said it had run continuously for thirty-one days. They were sure nobody was using it — internal tool, three people knew the URL, two of them on holiday. They wanted to know why our idle timer was broken.

The idle timer was not broken. It was doing exactly what it was told: any request resets it. What nobody had accounted for is that on the public internet, an unloved hostname still receives a steady drizzle of requests forever. Their uptime monitor hit /health every thirty seconds. Four vulnerability scanners were walking /wp-login.php, /.env and /actuator/env on a loop, having found the hostname in a certificate transparency log minutes after the TLS certificate was issued. AI crawlers re-read the same pages weekly. A stale browser tab was firing CORS preflights. And — the part I had to apologise for — our own platform health check was in there too. The only thing keeping the app alive was a robot checking whether it was alive.

I'm Ajay; I built PandaStack, a Firecracker microVM platform where git-driven apps scale to zero by actually deleting the running machine and restoring it from a snapshot on the next request. Idle-versus-awake is not a billing abstraction for me, it is a scheduling decision I have to get right a few thousand times a day. This post is the design we shipped: why the obvious rule is wrong, why the obvious fix is also wrong, and the two-predicate classifier that resolved it — with code you can lift and a query for finding your own offenders.

The app that would not sleep

Start with the shape of the traffic, because it is consistent across apps and it is not what people picture when they hear "bot traffic." It is not a DDoS. It is a metronome. Here is the census from that customer's request log for one representative day:

• Uptime monitoring — a synthetic checker polling /health on a fixed interval, often from several regions at once, so the effective interval is shorter than the one configured. Enough on its own to defeat any idle timer longer than the poll interval.
• Vulnerability and credential scanners — sweeps for /wp-login.php, /.env, /.git/config, /phpmyadmin, /vendor/phpunit. They do not care what your app is. They found you because a hostname appeared in a certificate transparency log: a public, append-only, real-time feed of every certificate issued. Publish a subdomain and you can be scanned before DNS finishes propagating.
• Search and AI crawlers — Googlebot, Bingbot, and the newer answer-engine fetchers. Low volume, high consequence: these are the requests you least want to fail.
• Preflight and probe requests — OPTIONS from a browser doing CORS, HEAD from a link unfurler, /favicon.ico and /robots.txt from anything that has ever heard of HTTP.
• Platform-internal health checks — the hosting provider's own liveness probe, politely resetting the idle timer of the app it was checking. Our bug, and the most embarrassing entry on the list.

Add those up and the median gap between requests was under twenty seconds. A fifteen-minute idle timeout will never fire against that; neither will a sixty-minute one. The app was not misconfigured, and the customer was not wrong about nobody using it. Both were true at once, which is what made the ticket interesting rather than annoying.

Two naive rules, both wrong

The rule we started with — the rule almost every platform starts with — is: any request resets the idle timer. One line of code, obviously correct in the sense that a request means somebody wants the app, and it has the failure mode above. It is wrong because it conflates "a packet arrived" with "someone is using this," and on the modern internet those are very different populations.

So the obvious fix is: ignore requests from bots, do not wake for them, do not serve them. This is worse. It serves an error page to Googlebot, and crawlers do not retry politely forever — a run of 502s reads as an unhealthy site, so your app quietly falls out of the index while your dashboards look fine, because your error rate is measured over requests you decided not to serve. It also breaks your own uptime monitoring, which now reports the app as down: technically accurate, operationally useless. Worst of all it converts a cost optimisation into an availability decision. You took something that was costing you money and made it cost you traffic.

Both rules fail for the same reason. They assume there is one question — "does this request matter?" — when there are two, and the answers are frequently different.

Two predicates, not one

The insight that makes the whole thing tractable is that waking and staying warm are separate decisions with separate costs, and they deserve separate predicates. Name them explicitly, in the code, so nobody collapses them again six months from now:

• allowsWake — should this request be allowed to boot a sleeping app? The answer is almost always yes. A served page beats an error page for basically every requester, including ones you find annoying. Googlebot should get a 200. Your uptime monitor should get a 200. Even a scanner getting a 404 from a woken app is not a disaster.
• keepsWarm — should this request reset the idle timer and hold the machine open? The answer is only yes if the request plausibly represents a human session or a real workload. A monitor poll does not. A crawler pass does not. A logged-in user loading a dashboard absolutely does.

Written that way, the bill makes sense: every one of those robotic requests was correctly answered with allowsWake = true, and every one was incorrectly credited with keepsWarm = true. The app woke for the monitor, which is right, then stayed awake for it, which is what cost money. Splitting the predicate means the app can answer every probe on earth and still be asleep between them — it sleeps fifteen minutes after the last request that mattered, not fifteen minutes after the last packet.

Here is the classifier, roughly as it exists in production, with the rules ordered so the cheapest and most certain checks run first.

// classify.ts -- one request in, two independent decisions out.
//
//   allowsWake : may this request boot a sleeping app?  default TRUE  (fail open)
//   keepsWarm  : may this request reset the idle timer?  default FALSE (fail closed)
//
// The asymmetry IS the design. A wrong keepsWarm=false costs one cold start.
// A wrong keepsWarm=true costs money for as long as the bot keeps polling.

export type Decision = { allowsWake: boolean; keepsWarm: boolean; reason: string };

// Endpoints that are, by construction, nobody's browsing session.
const PROBE_PATHS = new Set([
  "/health", "/healthz", "/readyz", "/livez", "/ping", "/status", "/metrics",
  "/robots.txt", "/favicon.ico", "/sitemap.xml", "/.well-known/security.txt",
]);

// Paths that exist only in a scanner's wordlist. If you see one of these, the
// requester is definitionally not a customer -- no app of ours serves them.
const SCANNER_PREFIXES = [
  "/wp-login", "/wp-admin", "/wp-content", "/xmlrpc.php", "/.env", "/.git/",
  "/phpmyadmin", "/vendor/phpunit", "/actuator/", "/cgi-bin/", "/.aws/",
];

// Weak, cheap, trivially spoofed -- and fine. A bot that lies about its UA is a
// billing problem, not an adversary. This is a cost heuristic, not a security
// boundary; never put an authorization decision behind this regex.
const BOT_UA =
  /bot|crawl|spider|slurp|monitor|uptime|pingdom|checkly|curl|wget|python-requests|go-http-client|headless|probe/i;

export function classify(req: Req): Decision {
  const path = req.path.toLowerCase();
  const ua = (req.headers["user-agent"] ?? "").toString();

  // 1. OUR OWN PROBES. Tag them, never guess at them. The platform health
  //    checker sets X-Probe on every request it sends, so it is *identified*
  //    rather than pattern-matched. This is the single highest-value rule here
  //    and the one most often missing -- your own liveness check is usually the
  //    most frequent client your app has.
  if (req.headers["x-probe"] === "platform") {
    return { allowsWake: false, keepsWarm: false, reason: "platform-probe" };
  }

  // 2. Scanner wordlist. The one place we do NOT fail open: the edge router can
  //    answer 404 itself without booting anything, because there is no version
  //    of this app that would have served /wp-login.php anyway.
  if (SCANNER_PREFIXES.some((p) => path.startsWith(p))) {
    return { allowsWake: false, keepsWarm: false, reason: "scanner-path" };
  }

  // 3. Monitoring and well-known probe endpoints. Wake for them -- an uptime
  //    check that reports you down is worse than a cold start -- but never let
  //    them hold the machine open.
  if (PROBE_PATHS.has(path)) {
    return { allowsWake: true, keepsWarm: false, reason: "probe-path" };
  }

  // 4. Method-only requests. A CORS preflight is usually followed within
  //    milliseconds by the real request, which will be judged on its own merits.
  //    HEAD is link unfurlers and monitors. Wake, don't warm.
  if (req.method === "OPTIONS" || req.method === "HEAD") {
    return { allowsWake: true, keepsWarm: false, reason: `method-${req.method}` };
  }

  // 5. Declared bots -- crawlers included. Serving Googlebot a 200 from a cold
  //    start is cheap. Serving it a 502 is expensive in a currency you cannot
  //    top up. Wake, don't warm.
  if (BOT_UA.test(ua)) {
    return { allowsWake: true, keepsWarm: false, reason: "bot-user-agent" };
  }

  // 6. Authenticated traffic. A session cookie or bearer token means someone (or
  //    some paying integration) got past a login. Strongest warm signal we have.
  if (req.headers["authorization"] || hasSessionCookie(req)) {
    return { allowsWake: true, keepsWarm: true, reason: "authenticated" };
  }

  // 7. Real browser navigation. Sec-Fetch-* is set by the browser, not the page,
  //    so a document navigation asking for text/html is a decent human proxy.
  //    Scanners send neither.
  if (
    req.method === "GET" &&
    req.headers["sec-fetch-mode"] === "navigate" &&
    (req.headers["accept"] ?? "").includes("text/html")
  ) {
    return { allowsWake: true, keepsWarm: true, reason: "browser-navigation" };
  }

  // 8. Burst shape. A real browser fetches the page and then its CSS/JS/fonts
  //    within a second or two. A scanner fetches one path and leaves. Look back
  //    over a short per-client window rather than judging the request alone.
  if (sawSubresourceFollowUp(req.clientKey, { windowMs: 10_000, minHits: 3 })) {
    return { allowsWake: true, keepsWarm: true, reason: "subresource-burst" };
  }

  // 9. Everything else: allowed to wake, not allowed to warm. If your logs show
  //    real users landing here, widen rules 6-8. Do not widen this one.
  return { allowsWake: true, keepsWarm: false, reason: "unclassified" };
}

Signals you can actually use, and how much to trust them

Every signal in that function has a different reliability, and it is worth being explicit about which ones are load-bearing:

• User-agent — weak but nearly free, and adequate here. Yes, it is trivially spoofed; it does not matter. A bot that spoofs a browser UA is not attacking you, it is at worst costing you an idle hour. This is a cost heuristic, not a security boundary. Never reuse this regex for anything that grants access.
• Request path — the strongest cheap signal. /healthz is a monitor. /wp-login.php on a Next.js app is a scanner, with certainty. Path rules are stable, explainable in a log line, and easy for a customer to audit when they disagree with you.
• Method — OPTIONS and HEAD are almost never a human reading something. Near-zero false positives, small but real effect: preflights alone kept one app in our sample awake.
• Session cookie or Authorization header — the best warm signal available. Someone authenticated. Even if it is a machine, it is a machine with credentials, which usually means a real integration doing real work: exactly the workload you want warm.
• Sec-Fetch-* and Accept — set by the browser rather than the page, so harder to get accidentally wrong than a UA string. A navigate-mode GET asking for text/html is a decent human proxy; API clients and scanners both fail it.
• Burst shape — the most reliable signal and the most expensive, because it needs short-window state per client. A browser loading a page pulls stylesheets, scripts and fonts immediately afterwards; a scanner takes one path and leaves. A ten-second counter keyed on client IP plus UA makes the difference stark.
• Your own probes, explicitly tagged — the only signal here with no false positives, because you control both ends. Set a header, match the header, done.

Note what is not on the list: IP reputation feeds, TLS fingerprints, behavioural scoring, anything you would buy. Those are bot-mitigation tools built for adversaries actively trying to look human. Your idle timer is not under attack; it is being nibbled by well-behaved robots that announce themselves in the user-agent string and obey robots.txt. Solve the problem you have.

Asymmetric errors deserve asymmetric defaults

The reason the two predicates get opposite defaults is that their mistakes cost wildly different amounts, and the difference is not a factor of two — it is a difference in kind.

A request that should have kept the app warm but did not costs exactly one cold start, once, for one visitor. It is bounded, it self-corrects (the next request finds the app awake), and it lands in your latency percentiles where you can see it. A request that should not have kept the app warm but did costs money for as long as that bot keeps polling — for an uptime monitor, forever. Unbounded, never self-correcting, visible nowhere except an invoice thirty days later. That is why keepsWarm fails closed.

The wake direction inverts every term. Waking unnecessarily costs one restore. Failing to wake means an error page to a real client — possibly a crawler, possibly a customer whose user-agent your regex mangled — with unbounded downside and no metric, because you never counted the request as served. That is why allowsWake fails open, with the scanner-wordlist rule as the single deliberate exception, safe only because those paths have no legitimate caller.

Written as a policy choice, the three options look like this:

• Any request warms — False-warm cost: severe and permanent; one 30-second monitor pins the app awake forever and scale-to-zero silently becomes always-on. False-sleep cost: zero, because it never sleeps. SEO risk: none. Effort: one line. This is the default almost everywhere, and it is why so many scale-to-zero bills look like always-on bills.
• Human-heuristic warms (the two-predicate classifier) — False-warm cost: low; a spoofing bot buys itself one idle window. False-sleep cost: one cold start per misclassified session, bounded and visible. SEO risk: none, because allowsWake stays open for crawlers by construction. Effort: a day to write, plus tuning driven by your own logs. This is what we shipped.
• Explicit-signal-only warms (nothing warms without a session or a documented header) — False-warm cost: near zero, the tightest possible. False-sleep cost: high; anonymous traffic cold-starts constantly, so it is only tolerable if your wake path is fast. SEO risk: still none if allowsWake stays open, but crawlers pay a cold start on nearly every page. Effort: low to build, high to live with. Fine for an internal tool, painful for a public site.

The second-order effect: cold starts get more frequent

Here is the part that surprises people, me included. The classifier works, the app sleeps, and your cold-start rate goes up sharply — because it was previously zero. All that robot traffic had been an accidental keep-warm service and you just cancelled it. You did not create a problem; you revealed one the bots were hiding, and your latency histogram will change shape the week you deploy this.

So the moment you fix the idle timer, wake latency becomes the priority, and how tolerable aggressive sleeping is depends on what "wake" means on your platform. If waking means pulling a container image and cold-starting a language runtime, sleeping hard is a bad trade and you should keep a warm floor. If waking means restoring a snapshot of a machine that was already running, the trade is easy.

On PandaStack every sandbox create is a snapshot restore rather than a boot — there is no warm idle pool, because there is nothing to keep warm. The restore step is around 49ms, and a full create runs at p50 179ms and p99 203ms including network setup and the readiness probe. The ~3s figure people sometimes quote is the first-ever cold boot of a template, which happens once when the snapshot is baked and never on the request path. Be honest about the boundary, though: those are platform-side numbers. What a visitor feels also includes DNS and edge routing and then your app actually being ready to serve — a framework that compiles on first request will dominate the microVM restore entirely.

Debugging your own: "what kept my app awake?"

The single highest-leverage thing you can do is log the classification decision alongside the request. One extra field — the reason string — turns an unanswerable billing argument into a two-line query. Log the path, method, user-agent, and the decision, then ask what has been holding the timer open.

-- "What kept my app awake?" -- every request that reset the idle timer
-- in the last 24h, grouped by who sent it.
SELECT
  reason,
  regexp_replace(path, '/[0-9a-f-]{8,}', '/:id') AS path_shape,
  left(user_agent, 48)                           AS ua,
  count(*)                                       AS hits,
  count(DISTINCT client_ip)                      AS ips,
  -- median gap between consecutive hits: humans are irregular, robots are not
  round(percentile_cont(0.5) WITHIN GROUP (
    ORDER BY EXTRACT(EPOCH FROM ts - lag(ts) OVER (PARTITION BY client_ip ORDER BY ts))
  )) AS p50_gap_s
FROM request_log
WHERE app_id = $1
  AND ts > now() - interval '24 hours'
  AND keeps_warm = true          -- only the requests that held the app open
GROUP BY 1, 2, 3
ORDER BY hits DESC
LIMIT 20;

-- The tell is p50_gap_s: a tight, low-variance gap (30, 60, 300) is a machine
-- on a cron. A human session has a messy gap and a diurnal shape.
-- Almost every app has three or four offenders and a long, harmless tail.

If your logs live in a flat file, the same thing falls out of a shell pipeline over JSON lines. Either way, expect a short answer. In every app we have looked at, three or four rows account for nearly all the false warms, and they are boring: one monitor, one scanner family, one crawler, and — until we fixed it — us.

Before you enforce a policy against production traffic, replay it. Take a day of access logs, run them through the classifier offline, and count what would have changed. A disposable sandbox is a convenient place to do that, because you can throw logs and a candidate ruleset at it and destroy the whole thing afterwards:

from pandastack import Sandbox

# Replay a day of real access logs against a candidate policy before enforcing
# it. Answers the only question that matters: how many idle minutes do we
# reclaim, and how many extra cold starts do we buy?
replay = """
import json, sys
from collections import Counter

IDLE_SECONDS = 900
warm_reasons, last_warm, cold_starts, awake_s = Counter(), None, 0, 0

for line in open("/workspace/access.jsonl"):
    r = json.loads(line)
    if r["keeps_warm"]:
        warm_reasons[r["reason"]] += 1
        if last_warm is None or r["ts"] - last_warm > IDLE_SECONDS:
            cold_starts += 1          # app had gone to sleep before this hit
        else:
            awake_s += r["ts"] - last_warm
        last_warm = r["ts"]

print("awake hours:", round(awake_s / 3600, 2))
print("cold starts:", cold_starts)
for reason, n in warm_reasons.most_common(5):
    print(f"  warmed by {reason}: {n}")
"""

with Sandbox.create(template="code-interpreter", ttl_seconds=600) as sbx:
    sbx.filesystem.write("/workspace/access.jsonl", open("access.jsonl").read())
    sbx.filesystem.write("/workspace/replay.py", replay)
    out = sbx.exec("python3 /workspace/replay.py", timeout_seconds=120)
    assert out.exit_code == 0, out.stderr
    print(out.stdout)
# sandbox is destroyed here -- the logs go with it

That is also why we could afford to sleep aggressively, and why we shipped this classifier at all: our scale-to-zero is literal — an idle app is snapshotted and its machine deleted, so a false warm lands on a customer's bill and a false sleep costs one restore. Managed Postgres has the same problem one layer down, where a monitoring connection running SELECT 1 keeps a database from ever auto-suspending. None of this is PandaStack-specific, though: the two predicates, the header on your own probes, and the reason field in your logs port to whatever runs your app. Please steal them.

When this is overkill

If your app has real, continuous human traffic, stop reading and go do something else. Scale-to-zero is not your lever: if there is a genuine request every few minutes during business hours, your app should be awake during business hours, and a classifier will spend its life agreeing with that. You would be optimising a cost that mostly is not there and paying for it in cold starts real users feel. For a busy app the levers are right-sizing and concurrency, not sleep policy.

A few other cases where this is the wrong project:

• Your idle bill is genuinely small. Do the multiplication before the engineering. If the app costs less per month than an hour of your time, the fix is a longer idle timeout and a shrug.
• Your wake path is slow and you cannot change it. A classifier that makes a slow app sleep more often is a downgrade. Fix wake latency first, or accept a warm floor — a deliberate always-on machine is a respectable answer.
• The app is private and unroutable. No public hostname means no CT-log scanners and no crawlers. Internal apps behind a VPN mostly do sleep correctly already, which is itself a hint about where the traffic was coming from.
• You need a hard guarantee that specific clients never wake the app. That is an authorization decision, not a heuristic. Put real authentication in front of the router; a user-agent regex is the wrong tool and will embarrass you.
• You have not measured yet. Log the reason field, wait a day, look. Maybe your top offender is one monitor you can point at a static endpoint on the edge instead, and the problem evaporates without a classifier at all. Cheaper than this post.

But if you configured scale-to-zero, believe nobody is using the app, and are billed as though somebody is: the app is telling the truth and so are you. Something is knocking. Go find out what, then decide separately whether it deserves an answer and whether it deserves your machine's attention for the next fifteen minutes. Those were always two questions.