Module pruner 

Background pruner for the historical column families.

Pruning is a standalone Service rather than a framework pipeline: it does not consume checkpoints from ingestion, it reads the already-committed state and deletes data below a retention floor. The shape mirrors the validator’s perpetual-store pruner (a periodic background task) more than the indexer framework’s per-pipeline prune hook — the deletions are data-driven (we walk transaction effects to retract superseded object versions) and the floor is a single value shared across every historical CF.

What gets pruned

• Per-transaction CFs (transactions, effects, events, tx_metadata_by_seq) — range-deleted over the pruned tx_seq range; the keys are contiguous big-endian tx_seq, so one range tombstone per CF clears the chunk.
• Per-checkpoint CFs (checkpoint_summary, checkpoint_contents) — range-deleted over the pruned checkpoint range.
• Digest reverse indexes (tx_seq_by_digest, checkpoint_seq_by_digest) — point-deleted; their keys are digests, so we collect them from the data being pruned (tx digests from each effects row, checkpoint digests from each summary) before deleting.
• objects history — point-deleted, effects-driven: each pruned transaction’s modified_at_versions (superseded input versions) and all_tombstones (deleted / wrapped markers) are the exact (ObjectID, version) rows that are now dead. The latest live version is never an input to a pruned transaction, so it — and the greatest object_version_by_checkpoint entry that resolves to it — is preserved.
• object_version_by_checkpoint — retracted in lockstep with objects history: the same effects-driven walk records per-object retractions, then each prune batch coalesces them per object to the latest superseding checkpoint and point-deletes that object’s checkpoint-pinned entries below it (plus the tombstone entry itself when the object was removed there). Point deletes rather than a range delete because a hot object’s successive retractions share the id||0 start and would nest into O(K^2) range-tombstone fragments (see the Retractions collector). The retained set mirrors the objects versions kept, so the index never points at a pruned version.
• Ledger-history bitmaps (transaction_bitmap, event_bitmap) — not deleted directly; advancing the database-local pruning floor lets their compaction filters drop fully-pruned buckets. Merge operands can require one covering compaction to materialize and a later compaction to filter; the forced catch-up pass and periodic compaction provide those sweeps.

The live-set-bounded indexes (object_by_owner, object_by_type, balance, package_versions) and the tiny epochs CF are never pruned.

Floor, retention, and safety

Retention is epoch-based: the retention_epochs most-recent epochs are retained, and the target floor is the start checkpoint of the oldest retained epoch. The floor is then clamped so it never advances past the oldest in-memory snapshot’s checkpoint: point and range deletes are already invisible to a snapshot (RocksDB pins the data a live snapshot references), but the bitmap compaction filter physically removes buckets irrespective of snapshots, so the clamp keeps every live snapshot’s advertised available range valid even under an aggressively small retention.

Each tick advances the floor toward that target by at most max_checkpoints_per_tick checkpoints (in max_chunk_checkpoints atomic chunks), so a large backlog — for example when pruning is first enabled on an old database — drains across many ticks rather than one long blocking pass. The floor converges to the target over subsequent ticks.

Ordering and crash-safety

Each chunk stages all of its deletes plus the new PruningWatermarks row into one atomic batch, commits, and only then advances the in-memory bitmap floor. Because the watermark row lives in the same batch as the deletes, a crash either loses the whole chunk (re-pruned next run) or commits it wholesale; there is no partial-delete-without-watermark state. Range and point deletes are idempotent, so a re-run is harmless.

Structs

PrunerMetrics

Prometheus metrics for the pruner.

Functions

embedded_prunable_checkpoint

The highest checkpoint the embedded fullnode’s pruner may prune through (inclusive) without deleting source data the embedded indexer still needs: min(checkpoint_hi_inclusive) across every embedded-cohort pipeline (LIVE_COHORT and HISTORY_COHORT).

prune_history_cohort

Prune the embedded fullnode’s history cohort up to a floor supplied by the validator’s perpetual-store pruner.

start_pruner

Start the background pruner as a Service.