What Free-Tier Abuse Actually Looks Like

The day you ship a free tier that runs arbitrary code, you stop being only a developer platform. You also become a small, unwilling hosting company whose entire customer base has no intention of ever paying you, whose support tickets you will never receive, and whose usage patterns you did not design for. This is not a hypothetical failure mode. It is a Tuesday.

I'm Ajay, I build PandaStack — a Firecracker microVM platform where anyone can create a sandbox and get a root shell inside it. That is the product, and it is also a vending machine for compute pointed at the open internet. This post is the field guide I wish I'd had: what abuse actually looks like from the operator's side, which signals survive contact with a motivated adversary, and the response ladder that lets you act without regularly wrecking a real customer's day.

One boundary up front. This is account-level abuse — signup rings, throwaway identities, farming, the economics of a tenant who costs you more than any plan you sell. The single-workload question, "is this specific process mining Monero," is a different problem with a different answer, and I wrote that one up separately. Here we are looking at the account graph, not the syscalls.

The taxonomy: five shapes, one of which is a customer

Abuse on a code-execution platform is not one thing, and treating it as one thing is why so many detection systems fire on the wrong population. In practice everything I have triaged falls into five buckets.

• Compute farming — mining is the famous case, but the more common and more boring version is somebody using your free tier as a build farm, a rendering queue, or free CI for a project that has nothing to do with you. Signature: sustained CPU, long-lived sandboxes, no interactive session, no HTTP traffic in.
• Residential-proxy and scraping fronts — your platform is not the target, it is the exit node. The workload is cheap on CPU and heavy on outbound connections to a wide, changing set of destinations. This one gets your egress IPs onto blocklists, which is a cost that outlives the account.
• Credential stuffing and spam egress — high connection counts to a small number of endpoints (an SMTP relay, one login form) with tiny payloads and a lot of failures. The economic damage is small; the reputational damage, when someone's abuse desk emails you, is not.
• Storage and hosting freeloading — using durable volumes, object storage, or a long-running app as free file hosting or a redirector for something you would not want your domain attached to. Low compute, low egress, indefinite duration. Easy to miss precisely because it is quiet.
• The enthusiastic user with an accidental infinite loop — a student, a hackathon team, somebody's first agent. Pegs a core, runs for hours, generated by a while-loop with a missing break. For the first ten minutes this is indistinguishable from bucket one, and it must NOT be handled like bucket one.

That last bucket is the whole difficulty of the job. Everything that makes a farm look like a farm — new account, no payment method, maximum resources, weird hours — also describes an excited person discovering that your free tier exists. The difference is almost never visible in the first ten minutes, and any system that acts within those ten minutes is a system that will regularly punish the exact user you built the free tier to attract.

The signals that generalise, at three layers

The signals worth building are the ones that do not depend on knowing what the customer's code is. Anything that requires you to correctly identify a binary, a library, or a string in a command line has a half-life measured in weeks. These three layers do not.

Layer 1: identity and signup

This is the cheapest layer and the one most likely to be wrong on its own. It is also where you catch rings rather than individuals, which is the only kind of win that lasts.

• Disposable and catch-all email domains. Worth checking, worth logging, not worth blocking alone. A blocklist is a list of domains somebody already found, and domain minting is cheaper than list maintenance — every ring I have investigated used domains my blocklist had never heard of.
• One payment instrument, device fingerprint, or browser profile across many accounts. Strong when present, absent by definition on the accounts that never reached a payment page.
• Signup bursts sharing an ASN, a subnet, or a naming pattern. The pattern is often almost comic: word plus four digits, ten accounts inside forty minutes, all from the same hosting ASN. Humans do not arrive in bursts of ten with matching name grammars.
• Signup to maximum concurrency in under a minute, with no interactive session. A real first-time user pokes around, reads a doc, creates one sandbox, breaks it. A script creates its quota's worth immediately and never opens a terminal.
• Correlated stop times. This one is underrated and nearly unfakeable by accident: four supposedly unrelated accounts that all go quiet within the same few seconds are one operator pressing one Ctrl-C.

Layer 2: workload shape

Shape, not magnitude. High CPU is what a build looks like. What farming looks like is high CPU that never comes down, with near-zero disk I/O and near-zero inbound bytes, for longer than any build takes. The load-bearing dimension is duration, and I want to be blunt about why: on our platform every first-party template bakes eight burstable vCPUs, so a 1 GiB guest can claim exactly as much CPU as a 4 GiB one. Memory quotas cannot bound CPU saturation. The only thing that reliably separates a mining loop from an all-core production build is that the build ends.

• Sustained CPU with a floor, not just an average. "Never dropped below 85% in six hours" is a far better feature than "averaged 95%", because averages are what bursty legitimate work also produces.
• Egress that looks like a proxy rather than a client — many destinations, short connections, roughly symmetric byte counts, and a destination set that keeps changing. A client talks to a few endpoints repeatedly; an exit node talks to the internet.
• Long-lived outbound sessions with a steady low-volume cadence and essentially nothing inbound. That is the mining and beaconing shape, and it is visible without inspecting a single process.
• Identical workload fingerprints across supposedly unrelated accounts. Same template, same region, same start command, same sandbox lifetime, same second of the hour. The fingerprint does not need to be interpretable to be a match.
• Zero interactive surface. No PTY sessions, no filesystem browsing, no dashboard logins — just API calls in a metronomic loop. Real users are messier than their scripts.

Layer 3: economics

The final layer is the one that works on abuse you never identified: a tenant whose cost-to-serve exceeds any plausible plan they could be on. You do not have to know what the workload is. You only have to know that it costs you more than the largest cheque this account could ever write.

Here is roughly the query I run. It is deliberately boring — it produces a ranked review list, it joins the cost back to the org and to identity context, and it bans nobody.

-- Top cost-to-serve tenants on a free plan, last 7 days.
-- Output is a REVIEW QUEUE ordered by what they cost us, with the
-- identity and shape context a human needs to make a judgement call.
WITH usage AS (
  SELECT
    org_slug,
    sum(cpu_seconds)                       AS cpu_secs,
    sum(mem_gib_seconds) / 3600.0          AS gib_hours,
    sum(egress_bytes) / 1e9                AS egress_gb,
    sum(cost_micros) / 1e6                 AS cost_usd,
    count(DISTINCT sandbox_id)             AS sandboxes,
    count(DISTINCT toDate(ts))             AS active_days
  FROM usage_events
  WHERE ts > now() - INTERVAL 7 DAY
  GROUP BY org_slug
),
shape AS (
  -- Duration is the discriminator, not magnitude. A build ends.
  SELECT
    org_slug,
    max(lifetime_seconds)                  AS longest_sandbox_s,
    avg(cpu_pct)                           AS cpu_avg,
    min(cpu_pct)                           AS cpu_floor,
    sum(pty_sessions)                      AS interactive_sessions,
    uniqExact(egress_dst_ip)               AS distinct_egress_dsts
  FROM sandbox_samples
  WHERE ts > now() - INTERVAL 7 DAY
  GROUP BY org_slug
)
SELECT
  o.slug,
  o.plan,
  o.created_at                             AS org_age,
  o.email_domain,
  o.has_payment_method,
  u.cost_usd,
  u.cpu_secs, u.gib_hours, u.egress_gb,
  u.sandboxes, u.active_days,
  s.longest_sandbox_s, s.cpu_avg, s.cpu_floor,
  s.interactive_sessions,
  s.distinct_egress_dsts,
  -- How many OTHER free orgs signed up on this domain. One is a person.
  -- Nine inside an hour is an operator.
  (SELECT count(*) FROM orgs x
    WHERE x.email_domain = o.email_domain AND x.plan = 'free') AS domain_siblings
FROM usage u
JOIN orgs o ON o.slug = u.org_slug
LEFT JOIN shape s ON s.org_slug = u.org_slug
WHERE o.plan = 'free'
  AND u.cost_usd > 5.00        -- more than any free allowance is worth
ORDER BY u.cost_usd DESC
LIMIT 100;

Score, don't gate: why per-signal thresholds fail

Every individual signal above has honest users behind it. Students use throwaway emails because they do not want marketing mail. CI is bursty and unattended by design. A bootcamp cohort signs up from one domain inside one hour and is structurally identical to a signup ring. Somebody's Monte Carlo simulation runs for four days at 100% CPU and they are delighted with you for allowing it.

So do not build gates on single signals. Build a score, require at least two independent signals before anything automated happens, and — this is the part people skip — make the score explainable. Every flagged tenant should come with the sentences that produced it: "CPU never dropped below 85% in six hours", "nine free orgs on this domain", "zero interactive sessions across 340 API calls". A reviewer who cannot argue with the reasoning cannot correct it either, and you will need them to correct it.

Independence is what makes correlation work. Two signals from the same layer are one signal wearing a hat: "new account" and "no payment method" describe the same fact about the same person.

The response ladder

The mistake is treating enforcement as binary — fine, or banned. In practice you want a ladder where the rungs are ordered by reversibility, and you only climb as far as your confidence justifies. Cheap and reversible at the bottom, expensive and permanent at the top.

• Rung 1, rate-limit — Reversibility: total, invisible, self-healing. Signal confidence needed: low. Effect on a real customer if wrong: they retry and notice nothing.
• Rung 2, throttle or tighten quota — Reversibility: total, one config change. Signal confidence needed: low-to-medium. Effect if wrong: a slower build and possibly a support ticket, which is a gift because it produces a human you can talk to.
• Rung 3, require verification — Reversibility: total, the customer clears it themselves. Signal confidence needed: medium. Effect if wrong: friction and annoyance. This rung is the highest-value one on the ladder: a card or SSO turns an anonymous attack into one with a cost and an identity attached.
• Rung 4, suspend workloads but PRESERVE data — Reversibility: high, resumable. Signal confidence needed: medium-to-high, ideally two independent signals. Effect if wrong: their app goes down, they are angry, and you can still fully restore them.
• Rung 5, ban the org — Reversibility: low in practice even when technically possible. Signal confidence needed: high, plus a human. Effect if wrong: you have destroyed a relationship and possibly data, and they will tell people, accurately.
• Rung 6, delete data — Reversibility: none. Signal confidence needed: absolute, after a retention window and a notice. Never automate this rung. There is no version of this that a scheduled job should be allowed to do at 3am.

Two rules make the ladder work. First, every automated action needs an appeal path and a human review queue attached, because you will false-positive a real customer and "an automated system flagged your account" with nobody behind it is how you turn a mistake into an unrecoverable one. Second, suspension must preserve state. Killing a workload and keeping its data means being wrong costs you an apology; deleting the data means being wrong costs you a customer and a public postmortem written by them.

The kill switch has to be org-scoped, and it has to actually reap

Sandboxes are disposable — that is the product. A create is a snapshot restore at roughly 179ms p50, so any enforcement aimed at a single sandbox is enforcement an abuser's retry loop routes around faster than your alert fires. The account is the durable object. Suspend the account.

And then there is the classic bug, which I have now written twice and will probably write again: "ban the org" that only writes a database row. The flag is set, the dashboard shows suspended, new creates are refused — and every VM the org already had is still running, still burning CPU, still egressing, because nothing ever told the fleet. The row is a policy statement, not an action. You need both.

// SuspendOrg: reversible, org-scoped, and it actually reaps.
// Order matters. Close the door FIRST, then clear the room -- reap
// before the flag lands and the retry loop just refills behind you.
func SuspendOrg(ctx context.Context, orgID string, reason string, actor string) error {
	// 1. Set the gate. Use the canonical org id, not a display slug --
	//    if callers can present two identity forms, resolve both here
	//    or the gate silently misses an entire tenant class.
	if _, err := db.ExecContext(ctx, `
		UPDATE orgs
		   SET suspended_at = now(), suspend_reason = $2, suspended_by = $3
		 WHERE id = $1`, orgID, reason, actor); err != nil {
		return fmt.Errorf("set suspend flag: %w", err)
	}

	// 2. Revoke credentials. A banned org with a live API token is not
	//    banned; it is inconvenienced. This is what actually stopped a
	//    real create-loop for us, well before any VM was touched.
	if _, err := db.ExecContext(ctx,
		`UPDATE api_tokens SET revoked_at = now() WHERE org_id = $1`, orgID); err != nil {
		return fmt.Errorf("revoke tokens: %w", err)
	}

	// 3. Enumerate live resources ACROSS THE FLEET. A sandbox you cannot
	//    trace to an org is untriageable, which is why org_id is a column
	//    on the row and not something we reconstruct from logs at 3am.
	res, err := fleet.ListResources(ctx, fleet.Filter{OrgID: orgID, State: "running"})
	if err != nil {
		return fmt.Errorf("enumerate org resources: %w", err)
	}

	// 4. Contain, don't destroy. Cut egress and hibernate: the workload
	//    stops earning and stops costing, memory and disk are preserved,
	//    and a wrong call is undone with a wake instead of an apology.
	var errs []error
	for _, r := range res {
		if err := fleet.DenyAllEgress(ctx, r.AgentID, r.ID); err != nil {
			errs = append(errs, err) // best effort: never block the hibernate
		}
		if err := fleet.Hibernate(ctx, r.AgentID, r.ID); err != nil {
			errs = append(errs, fmt.Errorf("hibernate %s on %s: %w", r.ID, r.AgentID, err))
		}
	}

	// 5. Audit, and file the appeal. Every automated suspension opens a
	//    ticket a human will read, whether or not the customer complains.
	audit.Record(ctx, "org.suspend", orgID, reason, actor, len(res))
	review.Enqueue(ctx, orgID, reason, res)

	return errors.Join(errs...)
}

Note what step four is not: a delete. Hibernate is the right rung here because it stops the cost immediately while keeping the state, which means the reversal is a wake call rather than a restoration from backup and a difficult email. If you only build one enforcement primitive, build this one.

Isolation is the backstop, not the control

It is tempting to think that strong isolation is an abuse control. It is not, and being clear-eyed about this saves you from a category error. Every abuser in the taxonomy above is using the platform exactly as designed. They are not escaping anything. A KVM boundary, a per-sandbox network namespace, a per-tenant egress policy — none of it stops somebody from running a loop you did not want to pay for.

What isolation does is decide what abuse costs. With a real boundary, abuse costs you money. Without one, abuse costs your other customers their data, and the thing you are handling stops being a trust-and-safety queue and becomes an incident with a disclosure obligation. Isolation does not stop abuse. It stops abuse from becoming a breach. That is worth a great deal, and it is a completely different sentence.

The instrumentation you need before any of this works

Everything above assumes three things exist. If they do not, none of it is buildable, and the order to build them in is not the order in which they are interesting.

1. Per-tenant attribution on every resource. Every sandbox, volume, database, app, and DNS record carries the owning org on the row. A resource you cannot trace to a tenant is untriageable, and you will discover this during the incident rather than before it.
2. Per-tenant cost, computed and stored. Not CPU seconds you could convert to dollars later — dollars, materialised, queryable, joined to the plan. "What does this org cost us" should be one query, because it is the question you will ask about every candidate.
3. An org-scoped kill switch that reaps. Enumerate across the fleet, act on every live resource, verify the count afterwards. Test it on your own internal org quarterly, because the failure mode is silent: the row updates, the API returns 200, and nothing stops.
4. An audit trail on enforcement itself. Who suspended whom, why, on what evidence, and when it was reversed. This is what makes the appeal path work and what stops your abuse tooling from becoming a thing nobody trusts to use.

There is one more piece of instrumentation that sounds like process rather than engineering, and matters as much: write down what you saw and why you acted. Six weeks later the same pattern comes back on a different domain with different account names, and that note is the difference between recognising it in ten minutes and rediscovering it over a weekend.

The honest summary

You cannot win this outright. Any control you build on identity is defeated by minting new identities; any control on workload identification is defeated by renaming things; any published quota becomes a config file for the person tuning their farm to sit just underneath it. If your goal is zero abuse, your real goal is zero free tier, and that is a legitimate choice — just make it deliberately rather than by accumulating controls until the funnel dies.

The achievable goal is narrower and better. Make abuse unprofitable: meter accurately, cap hard, require verification before sustained compute, and make egress default-deny so the exit-node and mining cases stop paying. Make enforcement cheap to reverse: contain before you delete, preserve data at every rung below the last, and keep a human in the loop with an appeal path they can actually action. Those two properties together produce the thing you actually need, which is a team willing to act. A control nobody dares use because the false-positive cost is catastrophic is not a control. It is a dashboard.