Almost every app grows scheduled work: expire stale sessions, roll up yesterday's usage, retry failed webhooks, email a weekly digest. The traditional answer is a VM with a crontab, which means you now own a machine whose only job is to be awake at 2am — plus its patching, its disk filling up, and the fact that when it reboots during a run, nothing tells you.
I'm Ajay, I build PandaStack. This is the practical version: how to run scheduled jobs without babysitting a server, and — more importantly — how to build them so they survive the failure modes that make scheduled work the most quietly broken part of most systems.
Step 1: Write the job as a plain program
The job should be an ordinary script with no scheduling logic inside it. No sleep loops, no 'is it Tuesday' checks, no internal timers. It does its work once and exits with a status code: zero for success, non-zero for failure. That's the whole contract, and keeping it means you can run the job by hand, test it, and move it between platforms without changing a line.
# jobs/rollup/handler.py -- runs once, exits with a meaningful code.
import os, sys
import psycopg
def main() -> int:
with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
with conn.cursor() as cur:
# Idempotent by construction: re-running for the same day
# overwrites rather than double-counting.
cur.execute("""
INSERT INTO usage_daily (day, org_id, seconds)
SELECT date_trunc('day', ts), org_id, sum(seconds)
FROM usage_events
WHERE ts >= current_date - interval '1 day'
AND ts < current_date
GROUP BY 1, 2
ON CONFLICT (day, org_id) DO UPDATE
SET seconds = EXCLUDED.seconds
""")
print(f"rolled up {cur.rowcount} org-days", flush=True)
return 0
if __name__ == "__main__":
sys.exit(main())Step 2: Make it safe to run twice
This is the rule that saves you, and it's worth designing for before you pick a platform. Your job will run twice. It will run twice because a retry fired, because someone triggered it manually while it was already running, because a deploy overlapped a schedule, or because the platform's at-least-once delivery did exactly what it promised. If a second run corrupts data, you don't have a scheduling problem — you have a correctness bug waiting for its moment.
- Prefer upserts over blind inserts, keyed on the natural period (day, hour, invoice ID). The SQL above re-runs harmlessly.
- Make state transitions conditional:
UPDATE ... WHERE status = 'pending'rather than unconditional writes, so the second run affects zero rows instead of re-doing the work. - Give outbound side-effects an idempotency key. Every serious payment and email API supports one; use it, keyed on the period rather than a random UUID.
- If none of that is possible, take a lock — see the next step.
Step 3: Decide what happens when a run overruns
A job scheduled every fifteen minutes that starts taking twenty is a completely ordinary consequence of your data growing. Most platforms — and plain crontab — will start the next run anyway, so you get two copies racing each other. Don't assume your platform prevents this; check, and if the answer is unclear, enforce it yourself. A Postgres advisory lock is three lines and removes the failure mode entirely.
# Exit quietly if a previous run is still going. The lock is released
# automatically when the connection closes -- including on a crash.
with conn.cursor() as cur:
cur.execute("SELECT pg_try_advisory_lock(%s)", (hash("usage-rollup") % 2**31,))
if not cur.fetchone()[0]:
print("previous run still in progress; skipping", flush=True)
sys.exit(0)Step 4: Attach a schedule
With the job written as a standalone program, scheduling is configuration. On PandaStack you deploy the job as a function and attach a five-field cron expression; each run executes in its own Firecracker microVM, so a job processing customer data gets a real isolation boundary and a fresh filesystem rather than inheriting whatever the last run left behind.
from pandastack import Client
ps = Client()
fn = ps.functions.deploy(
name="usage-rollup",
runtime="python",
path="./jobs/rollup",
entrypoint="handler.py",
env={"DATABASE_URL": DATABASE_URL},
)
ps.schedules.create(
name="usage-rollup-nightly",
function_id=fn["id"],
cron="23 2 * * *", # 02:23 UTC -- deliberately not on the hour
)Step 5: Alert on absence, not just on failure
This is the step that separates scheduled jobs that work from scheduled jobs that appear to work. Failure alerting catches a job that ran and crashed. It cannot catch the far more common and more expensive case: a job that stopped being scheduled at all — because a deploy dropped the config, because a workflow got auto-disabled, because a credential expired before the job even started, because someone paused it during an incident and never unpaused it.
The fix is a dead-man's switch. The job pings a monitor at the end of a successful run; the monitor alerts when the expected ping doesn't arrive. Healthchecks.io and Cronitor do this as a service, and a timestamp column plus an alerting rule does it in-house. Either way, the alert fires on silence.
import httpx
rc = main()
if rc == 0:
# Only ping on success. A failed run should stay silent here so the
# dead-man's switch fires -- a green ping on a broken run is worse
# than no monitoring at all.
httpx.get(os.environ["HEARTBEAT_URL"], timeout=10)
sys.exit(rc)Step 6: Make 'did it run?' a query
You'll need to answer this at some point, usually during an incident, usually at an inconvenient hour. Grepping logs across a fleet is not an answer. Pick a platform that records each run's status, exit code, output, and duration, and you can check the history in one call — and, more usefully, spot the run that's been quietly exiting non-zero for three weeks.
runs = ps.schedules.runs(schedule_id)
recent = runs[:14]
failed = [r for r in recent if r["status"] != "succeeded"]
print(f"{len(failed)} failures in the last {len(recent)} runs")
# Duration creeping toward the interval is the early warning that
# overlap is coming. Watch it before it becomes an incident.
print("p95 duration:", sorted(r["duration_ms"] for r in recent)[-1])The summary
Running a job on a schedule without a server is the easy part — every serious platform does it. The work that determines whether your scheduled jobs are trustworthy is in the job itself: make it a plain program that exits with a status, make it safe to run twice, decide explicitly what overlap does, avoid the top of the hour, and alert on silence rather than on errors. Do those five things and switching platforms later becomes a config change instead of a rewrite.