Introduction
OutlabsTaskq (outlabs-taskq) is a Postgres-native task queue for Python fleets.
It is infrastructure extracted from production systems at OutLabs — the same class of problem as outlabs-auth: stop reinventing a foundational backend piece in every product.
Why it exists
Most teams either:
- Run Redis / RabbitMQ / Celery and accept another operational dependency, or
- Grow an ad-hoc
jobstable until claim races, lost settles, and silent duplicates show up at 2am.
OutlabsTaskq takes a third path: PostgreSQL is the broker, and the PL/pgSQL functions are the contract. The Python package is a typed client — not a second source of truth.
What the current release contains
SQL kernel
Schema taskq, migrations 0001–0048, SQL contract 0.6.12, capability roles, queue admission ownership, and plan/apply migration plus exact verification.
Typed client
Task + TaskRegistry + TaskQ enqueue/claim/settle — plus admission, follow-ups, workflows, schedules, and workflow continuations.
Worker + scheduler
Fenced claims, heartbeats, typed settles, soft stop, presence, trusted host effects, and a standalone database-attested scheduler—all operated through the complete taskq worker run and taskq scheduler run CLI surface.
Optional HTTP
Mount the FastAPI facade for credential-free workers. Auth is injected; outlabs-auth is optional.
Flow control (off by default)
Per-queue overload protection: a circuit breaker (streak / rate / latency triggers, single-flight half-open, slow-start recovery), in-flight caps, GCRA rate limits, key-space fairness, priority aging, schedule and redrive smear, queue counters with health verdicts, and an operator audit log.
Status
Latest alpha release: 0.1.0a41. The a40 runtime introduced the SQL kernel,
typed client/worker/CLI, consumer testing helpers, Protocol-v1 HTTP clients and
facade, authorization boundary, durable admission, atomic follow-ups, sealed
workflows/dependencies, database-time schedules, finite projections, bounded
worker presence, native workflow-member continuations, and the trusted
host-effect fence. It also includes the standalone scheduler, database target
attestation, source-owned YAML manifests, durable schedule decisions,
overlap/lateness policy, deterministic-definition auto-pause, and bounded
global job-event/workflow/schedule projections. The 0.4–0.6 flow-control plane
(migrations 0022–0042) adds per-queue, off-by-default overload protection:
queue counters and health verdicts, GCRA rate limits, in-flight caps, key-space
fairness, slow-start ramps, schedule and redrive smear, a three-trigger circuit
breaker with single-flight half-open recovery, priority aging, and an operator
audit log. Migration 0043 adds workflow-safe set-based admission and
counter-backed enqueue depth checks. Migrations 0045–0048 add the terminal
host-effect fence and an installation-bound, PostgreSQL-role queue admission
owner. The a40 correction covers replay paths, permits independently
credentialed housekeeping to fire a schedule with transaction-bound
provenance, stores owner identity by role OID, and adds paused/quiesced adoption
and rotation recovery. Protocol revision is 1.0.18.
Release a41 leaves that runtime and SQL contract unchanged. It adds one bounded PostgreSQL 18 dump/restore catalog-verifier allowance; existing a40 consumers can stay on a40. See restore verification before accepting a restored database.
This documentation source tracks the published a41 release. Verify the package index, release record, and artifact hashes before rollout. Deep contract specs and accepted reusable decisions live in the public GitHub repo.
Related OutLabs infrastructure
| Package | Role |
|---|---|
outlabs-auth | Identity / RBAC / API keys for FastAPI |
outlabs-taskq | Job queue / worker coordination on Postgres |