Jobs and Queues

Every operation pgbr performs is a job on a BullMQ queue backed by Redis. The dashboard produces; the worker consumes.

The three queues

Queue	Payload	Produced by
`backup`	`BackupJobPayload`	The backup dialog, and cron schedulers
`restore`	`RestoreJobPayload`	The restore dialog
`migrate`	`MigrateJobPayload`	The migration page

Each queue gets its own Worker instance with its own concurrency, so a long migration never starves the backup queue. WORKER_CONCURRENCY (default 5) applies to each queue independently — one worker at the default can be running 15 jobs at once.

Job status

Job rows in Postgres move through four statuses:

Status	Meaning
`pending`	Defined in the schema; in practice rows are written as `running`.
`running`	The worker has claimed the job and written the row.
`completed`	The process exited 0 and, for backups, the artifact uploaded.
`failed`	Something failed. `error` holds the tool's stderr.

The row is written before the work starts and finalised in both the success and failure paths, so a job never silently vanishes. Any failure after the row exists — dump, non-zero exit, tar, upload — is caught and recorded.

Job IDs

Job IDs differ by origin, and the difference matters:

User-triggered jobs get a UUID generated by the dashboard, passed as both the BullMQ job ID and the database row ID. This is what lets the UI correlate a request with its row.
Scheduled jobs carry no dashboard-generated ID. The worker falls back to the BullMQ job ID, which for a scheduler looks like repeat:<scheduler-id>:<ts>.

That fallback is deliberate: the BullMQ ID is stable across a re-delivery, so a stalled scheduled job that another worker picks up lands on the same row rather than orphaning the first one in running.

Scheduler job IDs are sanitized before they become object keys — everything outside [A-Za-z0-9._-] becomes _, so repeat:abc:123 yields backups/repeat_abc_123.backup.

Interruption recovery

pgbr has no "mark every running row failed on boot" sweep. That would be unsafe the moment more than one worker exists — a booting replica would fail jobs another replica was actively running.

Instead it relies on BullMQ's stalled-job detection: if a worker dies mid-job, the job's lock expires and another worker reprocesses it. The processors' idempotent upserts are what make that safe.

A re-delivered backup re-runs pg_dump from scratch. Restores and migrations are re-delivered too — a re-run restore replays into the target database. With Single transaction on (the default), a partial first attempt rolled back, so the replay starts clean.

Retries

Jobs are enqueued with BullMQ's defaults, meaning no automatic retries on failure. A failed backup stays failed and visible. This is intentional: a dump that failed on a schema error will fail identically on retry, and retrying a restore is a decision with side effects. Re-run it yourself once you know why it failed.

Scheduled jobs in Redis

Repeating jobs are registered as BullMQ job schedulers — one per enabled backup schedule, keyed by the schedule's ID.

Postgres is the source of truth. The schedulers in Redis are derived state built by buildScheduleTemplate, which both the dashboard's actions and the worker's boot reconciliation call, so the two can't drift.

Scheduled jobs cap their Redis history at 100 completed and 100 failed entries. Postgres holds the real history; without the cap, repeat jobs would accumulate in Redis forever.

Reconciliation on boot

Every worker, on startup, tries to take a Redis lock (pgbr:reconcile-lock, 60s, set-if-not-exists). Whoever gets it:

Re-registers a scheduler for every enabled schedule with an owner
Removes any registered scheduler with no matching active schedule

Both directions are idempotent, so it heals drift from a Redis flush or a dashboard action that died between the database write and the Redis write.

Write ordering

When you create a schedule, the dashboard writes Postgres first and Redis second — and rolls the row back if registration fails. You never end up looking at a schedule that silently doesn't run. Reconciliation is the backstop, not the primary path.

Live updates

/api/events is an SSE stream carrying progress, completed, and failed from all three queues as { queue, event }. It emits a heartbeat comment every 25 seconds to keep proxies from closing the connection, and cleans up its listeners when the client disconnects.

BullMQ's queue events are instance-wide, so the route looks each event's job up by ID and forwards it only if job.data.userId matches the session. You're told when your own jobs move and nothing else — another account's activity isn't even visible as a timing signal.

progress — not active — marks a job's start, because the worker writes its row and then calls updateProgress. By the time the browser refreshes, there's something new to render.