Skip to content

PgQueuer v1.0

v1.3.0 Breaking

This release includes 2 breaking changes for platform teams planning a safe upgrade.

✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

postgresql python queue

Summary

AI summary

Broad release touches v1.3.0 Features:, qm, schema, and queries.

Full changelog

v1.3.0

Features:

  • Web dashboard (pgq web): built-in FastAPI + htmx UI for queue insights and job management: backlog age, throughput, duration percentiles, failures with tracebacks, worker liveness, schedules, table health, plus requeue/cancel actions. Live updates ride the existing LISTEN/NOTIFY channel. Install with pgqueuer[web,asyncpg].
  • First-class Postgres schema support: --schema / PGQUEUER_SCHEMA installs PgQueuer's tables, triggers, and functions into a named schema instead of public.
  • Automatic log aggregation: QueueManager rolls pgqueuer_log into pgqueuer_statistics on a timer (log_aggregation_interval, default 30s) instead of requiring a cron job. Coordinated across workers via advisory lock; skipped when a worker did no work since the last tick.
  • pgq sql command group: emit install/uninstall/upgrade/durability/autovac SQL to stdout with no DB connection. Pipe into psql or capture as a migration file for Flyway, sqitch, Alembic.
  • Unified connection handling: ConnectionSettings plus shared connect_asyncpg/connect_psycopg factories bring consistent PGQUEUER_* env var support to the CLI, MCP server, and web dashboard.

Performance:

  • In-memory driver dequeue replaced a full-table scan/sort with per-entrypoint priority heaps. Drain throughput roughly 4x (373 → 1610 jobs/s at 50k jobs).

Fixes:

  • Several dequeue/shutdown edge cases: batch jobs no longer stranded on shutdown, lifecycle tasks torn down on run() failure, dequeue timeout now probes eligible queued work before deferred ETA, batch cap respects the tightest entrypoint concurrency limit, dequeue picks capped to remaining per-entrypoint/worker slots, and drain no longer shuts down on a stale queued-work count (which could strand a retried job).
  • Schema upgrade rebuilds the statistics aggregation index.

Deprecations:

  • --dry-run deprecated in favor of pgq sql install|uninstall|upgrade|durability.
  • add_prefix() deprecated; prefix is now a DBSettings field reading PGQUEUER_PREFIX directly.

Documentation:

  • New guides for the web dashboard and log-aggregation gating/backlog caveat; connection docs clarify the bring-your-own-connection model; general docs cleanup.

What's Changed

  • fix(qm): confirm empty queue with uncached count before drain shutdown by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/692
  • fix(qm): dispatch remaining batch jobs on shutdown instead of stranding picked rows by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/693
  • fix(qm): tear down lifecycle tasks on run() failure by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/694
  • fix(qm): probe eligible queued work before deferred eta in dequeue timeoutmeout by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/695
  • fix(queries): cap dequeue picks to remaining per-entrypoint and worker slots by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/696
  • fix(qm): drop batch cap to tightest entrypoint concurrency limit by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/697
  • build: check formatting in make lint by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/698
  • perf(inmemory): replace full-scan dequeue with priority heaps by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/699
  • docs: clean up dash-heavy phrasing across docs by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/700
  • feat: unify connection config across CLI, MCP, and pool factories by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/702
  • feat(web): add built-in web dashboard for queue insights and management by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/691
  • refactor(settings): make prefix a DBSettings field, deprecate add_prefix by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/704
  • feat(schema): add first-class Postgres schema support by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/703
  • feat(cli): add pgq sql command group, deprecate --dry-run by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/705
  • test: bump testcontainers to >=4.14.2 to drop deprecation warning by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/706
  • feat(stats): periodic log->statistics aggregation without cron by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/707
  • docs(agents): ban module-level globals outright by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/708
  • fix(schema): rebuild aggregation index on upgrade by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/670
  • fix(qm): skip log aggregation tick when worker did no work by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/713
  • docs(reliability): document log-aggregation gating and backlog caveat by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/714
  • fix(ci): drop pytest-xdist workers from 4 to 2 by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/715

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v1.2.0...v1.3.0

Breaking Changes

  • --dry-run deprecated in favor of pgq sql install|uninstall|upgrade|durability
  • add_prefix() deprecated; prefix now read from PGQUEUER_PREFIX via DBSettings

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track PgQueuer v1.0

Get notified when new releases ship.

Sign up free

About PgQueuer v1.0

All releases →

Related context

Earlier breaking changes

  • v1.0.0 QueueManager now requires Queries(driver) instead of driver directly; PgQueuer constructor unchanged
  • v1.0.0 RetryWithBackoffEntrypointExecutor renamed to DatabaseRetryEntrypointExecutor and no longer depends on async-timeout
  • v1.0.0 CLI flags --pg-host, --pg-user etc. removed; use libpq environment variables or --pg-dsn instead
  • v1.0.0 Synchronous entrypoint handlers are rejected; only async def is allowed
  • v1.0.0 Custom async Driver must implement notify(channel, payload) method

Beta — feedback welcome: [email protected]