all posts

How to Run Cron Jobs Without Managing a Server

Ajay Kumar··8 min read

Almost every app grows scheduled work: expire stale sessions, roll up yesterday's usage, retry failed webhooks, email a weekly digest. The traditional answer is a VM with a crontab, which means you now own a machine whose only job is to be awake at 2am — plus its patching, its disk filling up, and the fact that when it reboots during a run, nothing tells you.

I'm Ajay, I build PandaStack. This is the practical version: how to run scheduled jobs without babysitting a server, and — more importantly — how to build them so they survive the failure modes that make scheduled work the most quietly broken part of most systems.

Step 1: Write the job as a plain program

The job should be an ordinary script with no scheduling logic inside it. No sleep loops, no 'is it Tuesday' checks, no internal timers. It does its work once and exits with a status code: zero for success, non-zero for failure. That's the whole contract, and keeping it means you can run the job by hand, test it, and move it between platforms without changing a line.

# jobs/rollup/handler.py -- runs once, exits with a meaningful code.
import os, sys
import psycopg

def main() -> int:
    with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
        with conn.cursor() as cur:
            # Idempotent by construction: re-running for the same day
            # overwrites rather than double-counting.
            cur.execute("""
                INSERT INTO usage_daily (day, org_id, seconds)
                SELECT date_trunc('day', ts), org_id, sum(seconds)
                  FROM usage_events
                 WHERE ts >= current_date - interval '1 day'
                   AND ts <  current_date
                 GROUP BY 1, 2
                ON CONFLICT (day, org_id) DO UPDATE
                   SET seconds = EXCLUDED.seconds
            """)
            print(f"rolled up {cur.rowcount} org-days", flush=True)
    return 0

if __name__ == "__main__":
    sys.exit(main())

Step 2: Make it safe to run twice

This is the rule that saves you, and it's worth designing for before you pick a platform. Your job will run twice. It will run twice because a retry fired, because someone triggered it manually while it was already running, because a deploy overlapped a schedule, or because the platform's at-least-once delivery did exactly what it promised. If a second run corrupts data, you don't have a scheduling problem — you have a correctness bug waiting for its moment.

  • Prefer upserts over blind inserts, keyed on the natural period (day, hour, invoice ID). The SQL above re-runs harmlessly.
  • Make state transitions conditional: `UPDATE ... WHERE status = 'pending'` rather than unconditional writes, so the second run affects zero rows instead of re-doing the work.
  • Give outbound side-effects an idempotency key. Every serious payment and email API supports one; use it, keyed on the period rather than a random UUID.
  • If none of that is possible, take a lock — see the next step.

Step 3: Decide what happens when a run overruns

A job scheduled every fifteen minutes that starts taking twenty is a completely ordinary consequence of your data growing. Most platforms — and plain crontab — will start the next run anyway, so you get two copies racing each other. Don't assume your platform prevents this; check, and if the answer is unclear, enforce it yourself. A Postgres advisory lock is three lines and removes the failure mode entirely.

# Exit quietly if a previous run is still going. The lock is released
# automatically when the connection closes -- including on a crash.
with conn.cursor() as cur:
    cur.execute("SELECT pg_try_advisory_lock(%s)", (hash("usage-rollup") % 2**31,))
    if not cur.fetchone()[0]:
        print("previous run still in progress; skipping", flush=True)
        sys.exit(0)

Step 4: Attach a schedule

With the job written as a standalone program, scheduling is configuration. On PandaStack you deploy the job as a function and attach a five-field cron expression; each run executes in its own Firecracker microVM, so a job processing customer data gets a real isolation boundary and a fresh filesystem rather than inheriting whatever the last run left behind.

from pandastack import Client

ps = Client()

fn = ps.functions.deploy(
    name="usage-rollup",
    runtime="python",
    path="./jobs/rollup",
    entrypoint="handler.py",
    env={"DATABASE_URL": DATABASE_URL},
)

ps.schedules.create(
    name="usage-rollup-nightly",
    function_id=fn["id"],
    cron="23 2 * * *",     # 02:23 UTC -- deliberately not on the hour
)
Two scheduling habits worth adopting permanently. Never schedule on the hour: every rate-limited API you call is at its busiest at :00 because everyone else picked it too. And write down which timezone the schedule uses — most platforms are UTC, so a job that must run at 9am local will silently drift by an hour twice a year.

Step 5: Alert on absence, not just on failure

This is the step that separates scheduled jobs that work from scheduled jobs that appear to work. Failure alerting catches a job that ran and crashed. It cannot catch the far more common and more expensive case: a job that stopped being scheduled at all — because a deploy dropped the config, because a workflow got auto-disabled, because a credential expired before the job even started, because someone paused it during an incident and never unpaused it.

The fix is a dead-man's switch. The job pings a monitor at the end of a successful run; the monitor alerts when the expected ping doesn't arrive. Healthchecks.io and Cronitor do this as a service, and a timestamp column plus an alerting rule does it in-house. Either way, the alert fires on silence.

import httpx

rc = main()
if rc == 0:
    # Only ping on success. A failed run should stay silent here so the
    # dead-man's switch fires -- a green ping on a broken run is worse
    # than no monitoring at all.
    httpx.get(os.environ["HEARTBEAT_URL"], timeout=10)
sys.exit(rc)

Step 6: Make 'did it run?' a query

You'll need to answer this at some point, usually during an incident, usually at an inconvenient hour. Grepping logs across a fleet is not an answer. Pick a platform that records each run's status, exit code, output, and duration, and you can check the history in one call — and, more usefully, spot the run that's been quietly exiting non-zero for three weeks.

runs = ps.schedules.runs(schedule_id)
recent = runs[:14]

failed = [r for r in recent if r["status"] != "succeeded"]
print(f"{len(failed)} failures in the last {len(recent)} runs")

# Duration creeping toward the interval is the early warning that
# overlap is coming. Watch it before it becomes an incident.
print("p95 duration:", sorted(r["duration_ms"] for r in recent)[-1])

The summary

Running a job on a schedule without a server is the easy part — every serious platform does it. The work that determines whether your scheduled jobs are trustworthy is in the job itself: make it a plain program that exits with a status, make it safe to run twice, decide explicitly what overlap does, avoid the top of the hour, and alert on silence rather than on errors. Do those five things and switching platforms later becomes a config change instead of a rewrite.

Frequently asked questions

What's the simplest way to run a scheduled job without maintaining a VM?

Write the job as a plain script that does its work once and exits with a status code, then attach it to a managed scheduler — your app platform's cron primitive, a functions-plus-schedule setup, or a CI scheduled workflow for non-critical housekeeping. Keeping all scheduling logic outside the job is what makes this easy: the same script runs by hand, in tests, and on any platform, so you're never rewriting the job to change where it runs.

How do I stop two copies of the same cron job running at once?

Don't rely on the platform to prevent it — many don't, and the documentation is often quiet on the point. Take a lock at the start of the job and exit cleanly if you can't get it. A Postgres advisory lock is the easiest version because it's released automatically when the connection closes, including when the process crashes, so you can't deadlock yourself with a stale lock file. Redis SETNX with a TTL works too if you already run Redis.

How do I know if a scheduled job silently stopped running?

Use a dead-man's switch: the job pings a monitoring endpoint after a successful run, and the monitor alerts when that ping fails to arrive inside the expected window. This is the only approach that catches a job that stopped being scheduled entirely, which is the most common serious failure — error alerting by definition only catches jobs that ran. Ping only on success, so a failing job triggers the absence alert rather than reporting itself healthy.

Should scheduled jobs share a database connection pool with my web app?

Generally no. A batch job doing bulk work has a very different connection profile from request handlers, and a job that opens a large pool or holds long transactions can starve the pool your users depend on. Give the job its own small pool — often a single connection is plenty — and be deliberate about long-running transactions, which hold locks and, on Postgres, block vacuum from cleaning up dead rows for as long as they're open.

What timezone do managed cron schedules use?

Almost always UTC unless you explicitly configure otherwise, and this catches people twice a year. A job scheduled at 09:00 UTC lands at 9am or 10am local depending on daylight saving, which matters enormously for anything with a business meaning — a report someone expects before a morning meeting, or a billing cutoff. If the wall-clock time matters, check whether your platform supports a timezone and set it; if it doesn't, either accept the drift deliberately or handle the local-time logic inside the job.

Keep reading

Run code in a microVM in one API call.

49ms p50 cold start. Fork, snapshot, and scale to zero.

Start free
Written by Ajay Kumar, Founder, PandaStack.