pg_durable: Microsoft open sources in-database durable execution
Microsoft open-sources pg_durable, a PostgreSQL extension for durable execution of long-running SQL workflows with automatic checkpointing and recovery.
pg_durable is a PostgreSQL extension that enables long-running, fault-tolerant workflows defined entirely in SQL. Users define workflows as directed acyclic graphs of SQL steps using composable operators like `~>` and `|=>`, then start execution with `df.start()`. The runtime automatically checkpoints between steps and resumes from the last checkpoint after crashes, restarts, or step failures, eliminating the need for separate cron jobs, worker processes, or queue infrastructure. Built on Rust libraries duroxide and duroxide-pg, it runs as an in-process background worker with no external services required. Target use cases include vector embedding pipelines, data ingestion, scheduled maintenance, fan-out aggregation, and external API workflows. The project is in preview and supports PostgreSQL 17 and 18 on amd64.
The extension includes row-level security, multi-user support, and operational visibility through PostgreSQL tables. Microsoft recommends against using pg_durable for single-statement operations, sub-millisecond synchronous work, restricted PostgreSQL environments, or highly heterogeneous distributed systems.
What HN community is saying
Commenters broadly question the architectural choice of storing workflow definitions in the database rather than in application code. The primary concern is version control, testing, and debugging: multiple users note that workflow logic should live in Git alongside code, not migrate with schema changes. A contributor clarifies that pg_durable supports function versioning via the underlying duroxide framework and acknowledges the need for better tooling around lifecycle management.
Secondary discussion centers on comparisons to existing solutions. Commenters mention Apache Airflow, Temporal, and pgmq as established alternatives and question the specific advantage of pg_durable over external orchestrators. One contributor mentions Microsoft's own Durable Task Framework as prior art and notes potential benefits: reduced overhead, co-location with the database for better load awareness, and unified state tracking without cross-referencing logs and code.