Postgres Recursive CTEs for Hierarchies

PostgreSQLBackendDatabase
Share on LinkedIn Share on X Share on Reddit Share on HN Share on Bluesky

Hierarchies appear everywhere: reporting lines, product categories, nested comments, multi-level BOMs. Relational tables are flat; trees are a graph constraint you enforce with parent_id foreign keys and query with recursive common table expressions (CTEs).

The hard parts are cycle safety, performance on wide/deep trees, aggregating up and down the tree, and knowing when adjacency list + recursive CTE stops scaling and ltree or closure tables deserve a migration.

Adjacency list baseline

CREATE TABLE org_unit (
  id         int PRIMARY KEY,
  name       text NOT NULL,
  parent_id  int REFERENCES org_unit(id)
);

CREATE INDEX org_unit_parent_idx ON org_unit (parent_id);

Roots have parent_id IS NULL. Inserts are O(1).

Walking down: all descendants

WITH RECURSIVE subtree AS (
  SELECT id, name, parent_id, 1 AS depth,
         ARRAY[id] AS path
  FROM org_unit
  WHERE id = 4

  UNION ALL

  SELECT c.id, c.name, c.parent_id, s.depth + 1,
         s.path || c.id
  FROM org_unit c
  JOIN subtree s ON c.parent_id = s.id
  WHERE NOT c.id = ANY(s.path)
)
SELECT * FROM subtree ORDER BY depth, name;

Anchor member (non-recursive): starting node(s). Recursive member: joins working set to base table.

path array: prevents revisiting nodes if a cycle exists.

Postgres 14+ CYCLE syntax:

WITH RECURSIVE subtree AS (
  ...
  CYCLE id SET is_cycle TO 'yes' DEFAULT 'no' USING path
)
SELECT * FROM subtree WHERE is_cycle = 'no';

Walking up: ancestors to root

WITH RECURSIVE ancestors AS (
  SELECT id, name, parent_id, 0 AS depth
  FROM org_unit WHERE id = 42

  UNION ALL

  SELECT p.id, p.name, p.parent_id, a.depth + 1
  FROM org_unit p
  JOIN ancestors a ON p.id = a.parent_id
)
SELECT * FROM ancestors ORDER BY depth DESC;

Aggregates roll-up

Each node has headcount; executive wants total including reports—propagate leaf metrics upward via join direction carefully. Test on small hand-drawn trees first.

For BOM quantity explosion:

WITH RECURSIVE explode AS (
  SELECT parent_part_id AS root_id, child_part_id, qty::numeric AS cumulative_qty
  FROM bom_edge WHERE parent_part_id = $1

  UNION ALL

  SELECT e.root_id, b.child_part_id, e.cumulative_qty * b.qty
  FROM explode e
  JOIN bom_edge b ON b.parent_part_id = e.child_part_id
  WHERE e.cumulative_qty * b.qty < 1000000
)
SELECT root_id, child_part_id, cumulative_qty FROM explode;

Depth limits and pagination

"Only direct reports" is depth = 1—skip recursion. "Up to 3 levels" adds WHERE s.depth < 3 in recursive term.

Paginating recursive results is awkward—materialized path or closure table for "page 2 of subtree."

Indexing and EXPLAIN

Critical index: (parent_id) for downward walks.

Wide tree (CEO → 500 direct reports) explodes row count—breadth kills more than depth.

Mitigations: materialized view refreshed nightly, closure table, or ltree:

CREATE EXTENSION ltree;
ALTER TABLE org_unit ADD path ltree;
CREATE INDEX path_gist ON org_unit USING gist (path);
SELECT * FROM org_unit WHERE path <@ '1.4';

Closure table pattern

CREATE TABLE org_closure (
  ancestor_id   int NOT NULL,
  descendant_id int NOT NULL,
  depth         int NOT NULL,
  PRIMARY KEY (ancestor_id, descendant_id)
);

Subtree query without recursion:

SELECT o.* FROM org_unit o
JOIN org_closure c ON o.id = c.descendant_id
WHERE c.ancestor_id = 4;

Mutations: moving subtrees

Verify no cycle before reparent: ensure new parent is not in node's subtree. ltree/closure require bulk path or closure row updates.

Materialized hierarchy cache

When recursive CTE latency crosses SLO:

CREATE MATERIALIZED VIEW org_unit_enriched AS
WITH RECURSIVE tree AS ( ... ) SELECT * FROM tree;
REFRESH MATERIALIZED VIEW CONCURRENTLY org_unit_enriched;

Refresh after bulk HR imports—not every row update.

Graphs with multiple parents

BOM and dependency graphs are DAGs—model edge (from_id, to_id) with cycle guard, not single parent_id.

Common data quality failures

SELECT id FROM org_unit o
WHERE parent_id IS NOT NULL
  AND NOT EXISTS (SELECT 1 FROM org_unit p WHERE p.id = o.parent_id);

When to stop using recursive CTEs

Signals: P95 subtree query > 50ms at your scale, depth > 20, frequent ancestor/d descendant checks, or graph is DAG not tree.

Recursive CTEs are the right default for org charts and category trees—until measurement says otherwise.

work_mem and spilling on deep trees

Recursive CTEs materialize working tables; wide trees exceed work_mem and spill to disk—EXPLAIN ANALYZE shows workfile usage. Raise work_mem session-local for reporting queries only, not globally. Alternative: limit depth, paginate roots, or precompute closure. max_recursive_iterations (PG 14+) caps runaway graphs—set on APIs exposed to user-supplied graph traversal to prevent DoS.

Serializable isolation and hierarchy updates

Reparenting under SERIALIZABLE may abort when concurrent reads traverse old and new structure—retry logic required. For HR org changes during business hours, short transactions on metadata tables plus READ COMMITTED default often suffice; use SELECT ... FOR UPDATE on moved subtree root during reparent.

Comparing ltree operators

Beyond @> (ancestor) and <@ (descendant), ~ matches lquery patterns—useful for "all units matching *.engineering.*" without recursive CTE. ? checks label at level. Migration from adjacency to ltree: bulk UPDATE path FROM recursive compute, validate with nlevel(path) vs recursive depth, add GiST index before cutover.

Comment threading and visibility

Comment trees add visible and deleted_at predicates in recursive term—apply in anchor and recursive members identically or hidden replies reappear as orphan paths. Soft-delete patterns: stop recursion at deleted node but keep subtree counts for moderation dashboards via separate moderation_walk CTE.

Recursive CTE in writable CTEs (DML)

Postgres allows WITH RECURSIVE ... UPDATE/DELETE patterns for hierarchical bulk operations—use carefully with row locks. Example: deactivate entire subtree by recursive selection then update—prefer single UPDATE ... WHERE id IN (SELECT id FROM recursive_cte) for clarity and lock ordering.

Performance comparison table

Approach Query complexity Write complexity Best for
Adjacency + recursive CTE Low Low Small/medium trees
ltree Medium Medium (path recompute) Frequent subtree queries
Closure table Low reads High writes Read-heavy, moderate writes
Materialized view Lowest reads Refresh lag Analytics snapshots

Choose based on read/write ratio measured in production, not architecture blog preferences.

API design for hierarchy endpoints

Expose three endpoints—ancestors, descendants, subtree aggregate—backed by different SQL plans rather than one mega-recursive query with optional flags. Cache subtree roots with TTL for catalog trees changing hourly not per second. Return cycle_detected boolean in API when using CYCLE clause so clients can surface data quality errors to admins instead of silent truncation.

Debugging recursive performance in production

Log recursive depth and row count from application after EXPLAIN ANALYZE in staging derived templates. When production latency spikes, compare current depth distribution to baseline—org reorg flattening tree should reduce depth but reorg bugs often create temporary deep chains before cleanup jobs run.

Data import validation for hierarchies

After bulk CSV import of parent_id relationships, run cycle detection and orphan queries before enabling user traffic. One bad row creating cycle can DoS admin tree views—validate in transaction, reject file on failure.

Serializable subtree deletion

Deleting subtree root with recursive CTE selection then DELETE must lock rows in consistent order to avoid deadlock with concurrent inserts under parent—delete deepest nodes first via recursive ordering by depth DESC or use closure table batch delete pattern.

Export for downstream systems

Graph analytics tools prefer edge list CSV—export (parent_id, child_id) pairs rather than recursive query at export time; recursion stays for online API, batch export uses simple join.

Resources

Frequently asked questions

When should I use a recursive CTE versus storing a materialized path?

Recursive CTEs on adjacency lists are simplest for read-mostly trees under a few thousand nodes per query and moderate depth. Materialized paths or closure tables win when you need constant-time subtree checks, frequent ancestor lookups, or depth limits without recursive planning cost.

How do I prevent infinite loops in cyclic graphs?

Track visited node IDs in an array column in the recursive term and filter with WHERE NOT (id = ANY(path)). Postgres 14+ also offers CYCLE clause in WITH RECURSIVE for automatic cycle detection.

Are recursive CTEs efficient for deep org charts?

Each depth level is another iteration; depth 50 means 50 recursive steps. Index parent_id, consider ltree extension, or precompute paths. EXPLAIN shows Recursive Union nodes—watch work_mem and row counts on wide trees.

Can I use recursive CTEs for bill-of-materials quantity explosion?

Yes. Carry cumulative quantity in the recursive term: child_qty = parent_qty * edge_qty. Use numeric types, not float. Cap depth or detect cycles; BOM graphs are prone to cycles from data entry errors.

Hiring a senior Android / Flutter engineer?

I architect and ship production mobile software — Kotlin, Jetpack Compose, Flutter — for robotics, EV infrastructure, fintech, and real-time systems. Open to remote roles in Europe and the US.

Get in touch →