Struct IngestionService 

pub struct IngestionService { /* private fields */ }

Implementations

impl IngestionService

pub fn new( args: ClientArgs, config: IngestionConfig, metrics_prefix: Option<&str>, registry: &Registry, ) -> Result<Self>

Create a new instance of the ingestion service, responsible for fetching checkpoints and disseminating them to subscribers.

• args specifies where to fetch checkpoints from.
• config specifies the various sizes and time limits for ingestion.
• metrics_prefix and registry are used to set up metrics for the service.

After initialization, subscribers can be added using Self::subscribe_bounded, and the service is started with Self::run, given a range of checkpoints to fetch (potentially unbounded).

pub fn with_clients( ingestion_client: IngestionClient, streaming_client: Option<BoxedStreamingClient>, config: IngestionConfig, metrics: Arc<IngestionMetrics>, ) -> Self

Construct an IngestionService from pre-built clients, bypassing ClientArgs-driven construction.

Callers that supply their own IngestionClientTrait / CheckpointStreamingClient implementations — for example, when embedding the indexer in a fullnode that already has checkpoint data on hand — use this constructor instead of Self::new.

metrics is the shared IngestionMetrics handle. The caller is expected to have built it once and passed the same handle to IngestionClient::from_trait (or another IngestionClient constructor) so the service and the client report against a single set of registered metric vectors — building two sets against the same prometheus registry would double-register the metric names.

pub async fn latest_checkpoint_number(&mut self) -> Result<u64>

Return the latest checkpoint number known to the ingestion service, preferably via the streaming client, and failing that via the ingestion client.

pub fn config(&self) -> &IngestionConfig

The ingestion configuration this service was built with.

pub fn subscribe_bounded( &mut self, size: usize, ) -> Receiver<Arc<CheckpointEnvelope>>

Add a new subscription backed by a bounded mpsc channel of the given capacity. The channel itself is the backpressure signal: when this consumer falls behind, the channel fills and the adaptive ingestion controller cuts fetch concurrency. Send blocks when the channel is full.

Callers typically pass pipeline::IngestionConfig::subscriber_channel_size().

pub async fn run<R>(self, checkpoints: R) -> Result<Service>

where R: RangeBounds<u64> + Send + 'static,

Start the ingestion service as a background task, consuming it in the process.

Checkpoints are fetched concurrently from the checkpoints iterator and pushed to subscribers’ channels (potentially out-of-order). Each subscriber’s bounded channel acts as the backpressure signal: when it fills, the adaptive ingestion controller throttles fetch concurrency. The slowest subscriber gates ingestion for everyone.

If a subscriber closes its channel, the ingestion service shuts down as well.

If ingestion reaches the leading edge of the network, it will encounter checkpoints that do not exist yet. These are retried on a fixed retry_interval until they become available.