How to tail logs and debug a running app
When a deploy goes wrong, the useful first question isn't "what do the logs say" — it's "which logs". On any modern app platform there are at least three streams, they're produced by different things at different times, and picking the wrong one wastes the first ten minutes of an incident.
- Build logs — install and compile output, from the deploy. Read these when the deploy never went live.
- Runtime logs — your application's own stdout and stderr. Read these when the app is live and behaving badly.
- Host logs — the platform's view of the machine or VM. Read these when the app died and left nothing behind.
The rest of this is how to get at each of them, and what each one is good for. Examples use PandaStack because it's what I build; the shape of the problem is the same everywhere.
Build logs: the deploy never went live
If the app's status went to `failed` and the previous version is still serving, the failure happened during install, build, or the health check. That's build-log territory.
# Most recent deployments
curl -s https://api.pandastack.ai/v1/apps/$APP_ID/deploys \
-H "Authorization: Bearer $PANDASTACK_API_KEY" | jq '.deployments[0]'
# Stream that deployment's build log
curl -N https://api.pandastack.ai/v1/apps/$APP_ID/deploys/$DEPLOY_ID/logs \
-H "Authorization: Bearer $PANDASTACK_API_KEY"Three failures account for most of what you'll find here. A runtime version mismatch — the platform picked Node 22 and your package requires 20, or vice versa; pin it with a `.nvmrc` or `.tool-versions` in the repo. A missing build-time environment variable, where a framework needs a value at compile time and got `undefined`. And an out-of-memory kill during the build, which typically shows up as the build stopping mid-step with no error at all — the tell is a truncated log rather than a stack trace.
Runtime logs: the app is live and wrong
Once a deploy is live, what matters is what your process writes to stdout and stderr. Anything your framework logs, every unhandled rejection, every console.error lands here.
# Snapshot — the last ~1000 lines
curl -s https://api.pandastack.ai/v1/apps/$APP_ID/runtime-logs \
-H "Authorization: Bearer $PANDASTACK_API_KEY"
# Follow, as a stream
curl -N "https://api.pandastack.ai/v1/apps/$APP_ID/runtime-logs?follow=1" \
-H "Authorization: Bearer $PANDASTACK_API_KEY"One thing to know about the follow variant: it's a stream that stays open, so pipe it rather than letting it fill your terminal, and remember that a stream from a single instance shows you that instance. If you're debugging an intermittent error and the app has scaled out, you may be watching the wrong one.
Make the runtime logs worth reading
This is the part people skip and then regret at 2am. Two habits do most of the work.
First, log structured JSON, one object per line. Human-readable log lines are pleasant to read and impossible to filter, and every log tool in existence can query JSON.
import pino from "pino";
const log = pino({ level: process.env.LOG_LEVEL ?? "info" });
app.use((req, res, next) => {
const started = Date.now();
const requestId = req.headers["x-request-id"] ?? crypto.randomUUID();
res.on("finish", () => {
log.info({
requestId,
method: req.method,
path: req.path,
status: res.statusCode,
durationMs: Date.now() - started,
});
});
next();
});Second, put a request ID on every line. When a user reports an error, one identifier lets you pull every log line from that request instead of reading around a timestamp and hoping.
# With JSON lines, jq turns the log into a query interface
curl -s https://api.pandastack.ai/v1/apps/$APP_ID/runtime-logs \
-H "Authorization: Bearer $PANDASTACK_API_KEY" \
| jq -c 'select(.status >= 500)'
# Everything from one request
... | jq -c 'select(.requestId == "8f3c1e2a-...")'
# The slowest requests in the buffer
... | jq -s 'sort_by(-.durationMs) | .[0:10] | .[] | {path, durationMs}'The logs are empty and the app is definitely running
Nine times out of ten this is output buffering. When stdout is a pipe rather than a terminal — which it always is in production — many runtimes switch to block buffering and hold output until several kilobytes accumulate. Your app is logging; you just can't see it yet.
# Python: unbuffered
PYTHONUNBUFFERED=1 python app.py
# or: python -u app.py
# Node buffers far less, but explicit flushing still helps for crash paths.
# Gunicorn: don't capture output into a file you then can't read
gunicorn --access-logfile - --error-logfile - app:appThe other common cause: your framework is writing to a log file inside the container rather than to stdout. Point it at stdout — the platform captures that, and a log file inside an ephemeral filesystem disappears with the instance.
When logs aren't enough: get inside the running instance
Some problems don't show up in logs at all — a wrong environment variable, a file that isn't where the build put it, a port nothing is listening on. For those, a shell in the live instance answers in seconds what an hour of log reading won't.
# The app's sandbox id
SANDBOX_ID=$(curl -s https://api.pandastack.ai/v1/apps/$APP_ID \
-H "Authorization: Bearer $PANDASTACK_API_KEY" | jq -r .sandbox_id)
# Is anything actually listening?
curl -s -X POST https://api.pandastack.ai/v1/sandboxes/$SANDBOX_ID/exec \
-H "Authorization: Bearer $PANDASTACK_API_KEY" \
-d '{"cmd":"ss -tlnp"}' | jq -r .stdout
# Did the env var arrive? (names only — never print secret values)
curl -s -X POST https://api.pandastack.ai/v1/sandboxes/$SANDBOX_ID/exec \
-H "Authorization: Bearer $PANDASTACK_API_KEY" \
-d '{"cmd":"env | cut -d= -f1 | sort"}' | jq -r .stdoutThere's an interactive PTY endpoint too, which is what the dashboard terminal uses. The rule I'd apply: use the shell to diagnose, never to fix. A change you make by hand inside a running instance disappears on the next deploy and leaves the next person confused about why the fix stopped working.
Host logs: the app died and left nothing
If the process vanished without writing anything — no stack trace, no shutdown message — the kill came from outside it. Host-level logs are where that shows up: out-of-memory kills, health-check failures that triggered a restart, the platform stopping the instance.
curl -s https://api.pandastack.ai/v1/sandboxes/$SANDBOX_ID/logs \
-H "Authorization: Bearer $PANDASTACK_API_KEY"A silent death under load is a memory limit until proven otherwise. A silent death shortly after deploy is usually a health check the app failed while it was still warming up — the fix is a longer grace period or a health endpoint that doesn't touch the database.
The order I'd work in
- Did the deploy complete? If not, build logs — and check for a truncated log before reading the errors.
- Is the process running and serving? Runtime logs, filtered to errors.
- Does the app respond to some requests and not others? Runtime logs, filtered by request ID from a failing request.
- Did the process disappear? Host logs, looking for OOM or a restart.
- Do the logs say nothing at all? Exec in and check the port, the environment, and the working directory.
- Only then start changing code.
The meta-point: most debugging time is spent establishing which of those five states you're in. Structured logs with request IDs collapse that step from ten minutes to one, which is why they're worth adding before you need them rather than during the incident that made you want them.
Frequently asked questions
Why are my application logs empty even though the app is running?
Output buffering, in most cases. When stdout is a pipe rather than a terminal — which it always is in production — many runtimes switch to block buffering and hold output until a few kilobytes accumulate, so a low-traffic app can appear silent for minutes. Set PYTHONUNBUFFERED=1 for Python, or run with python -u. The other frequent cause is a framework configured to write to a log file inside the instance instead of stdout, where the platform never sees it and the file dies with the instance.
What's the difference between build logs and runtime logs?
Build logs come from the deploy pipeline — dependency installation, compilation, the initial health check — and they stop the moment the deploy succeeds or fails. Runtime logs are your process's own stdout and stderr while it serves traffic. If the deploy failed and the old version is still live, the answer is in the build logs. If the app is live and returning errors, it's in the runtime logs. Reading the wrong one is the most common way to waste the start of an incident.
How do I debug an app that crashes with no error message?
A process that vanishes without writing anything was almost certainly killed from outside — the OOM killer, or the platform restarting it after a failed health check. Check the host-level logs, which record that, rather than the application logs, which by definition contain nothing. A silent death under load points at memory; a silent death shortly after a deploy points at a health check failing while the app was still starting up.
Should I log in JSON or plain text?
JSON, one object per line, for anything running in production. Plain text is nicer to read directly and effectively impossible to query, so the moment you want "every 500 in the last hour for this user" you're writing fragile grep. With JSON lines, jq handles that locally and every log platform handles it centrally. Keep pretty-printed output for local development, where you're reading rather than searching.
Is it safe to open a shell in a production instance?
To diagnose, yes — checking which port is listening, whether an environment variable arrived, or where the build actually put your files answers in seconds what logs may never tell you. To fix, no. A change made by hand inside a running instance disappears at the next deploy, and in the meantime production no longer matches your repository, which is a genuinely dangerous state to leave behind. Diagnose in the shell, fix in the code, redeploy.
Keep reading
49ms p50 cold start. Fork, snapshot, and scale to zero.