Runtime-first upgrades

TaskQ has two independent version axes: the installed Python package and the SQL contract recorded in the target database. An artifact is ready only when its runtime declares the observed SQL contract in its closed compatibility set. A numerically newer contract is not automatically safe.

The 0.1.0a40 release targets SQL contract 0.6.12 and declares explicit compatibility with the earlier contracts needed for a staged runtime-first rollout. Runtimes older than a40 do not recognize 0.6.12 and may not map its queue-owner denial correctly.

Required order

1. Publish or otherwise approve one immutable wheel and source distribution. Record their hashes and retain the previous artifact.
2. Regenerate every consumer lock from the published artifact and build each deployable API, worker, scheduler, operator, migrator, and maintenance artifact once. Confirm the installed package reports the expected version.
3. Pause affected queues and schedules. Deploy the compatible runtime while the database remains on its old, supported contract. Fence every older process so it cannot reconnect.
4. Run the release preflight against the exact target. Review package version, observed SQL contract, target environment, installation identity, pending migration fingerprints, queue state, and scheduler ownership.
5. Apply migrations only after the operator has independently established the runtime-first condition. With outlabs-release, an existing TaskQ database with pending migrations requires the explicit --runtime-first-confirmed gate. The guard runs before Auth, TaskQ, or host migrations mutate the database.
6. Require taskq db verify and post-migration attestation to report an exact match. Bind a new empty queue, or use adoption/rotation only under the paused, quiesced recovery rules.
7. Prove owner and non-owner LOGIN admission, separate-housekeeper schedule fire, worker claim and settlement, replay denial, and the host application's authorization boundary on the exact deployment artifacts.
8. Resume one queue and one bounded canary at a time. Record terminal counts, schedule occurrences, workflow identity, and application side effects before widening traffic.

outlabs-release preflight

# Only after every compatible runtime is deployed and every older runtime is fenced:
outlabs-release migrate \
  --database primary \
  --confirm \
  --runtime-first-confirmed

outlabs-release attest
taskq --context production db verify

--runtime-first-confirmed is an operator attestation, not an automatic discovery mechanism. Do not pass it because a deployment was requested; pass it only after process inventory and immutable-artifact evidence prove the condition.

Recovery boundary

Before the SQL migration, restoring the previous artifact is a valid rollback while the database still reports a contract that artifact accepts. After a forward-only migration, pause queues and schedules and recover forward with a compatible runtime. There is no supported schema downgrade or generic queue ownership clear.

If the bound producer role was dropped or must be replaced, keep the queue paused and quiesced, attest the exact target, rotate to a new dedicated taskq_producer member with a nonempty actor and reason, inspect the resulting queue_audit row, and repeat catalog and LOGIN-boundary verification before resuming.

Evidence packet

Keep the following values outside transient terminal scrollback and exclude credentials and payloads:

• source commits, package versions, artifact hashes, and consumer lock hashes;
• old and new SQL contracts, migration head, target environment, and installation ID;
• old-process fencing, API/scheduler/worker process inventory, and queue states;
• negative and positive database-role and host-authorization checks;
• canary workflow/job identities, terminal counts, committed side effects, and the forward-recovery command.