Struct CommitterConfig 

pub struct CommitterConfig {
    pub write_concurrency: usize,
    pub watermark_interval_ms: u64,
    pub watermark_interval_jitter_ms: u64,
    pub min_batch_rows: usize,
    pub max_batch_delay_ms: u64,
}

Fields

write_concurrency: usize

Number of concurrent writers per pipeline.

watermark_interval_ms: u64

Minimum time between two watermark writes of a concurrent pipeline, in milliseconds. A watermark that advances once this much time has passed since the previous write is written immediately; one that advances sooner is written when the gap has passed.

watermark_interval_jitter_ms: u64

Maximum random jitter to add to the watermark interval, in milliseconds.

min_batch_rows: usize

How many rows the collector waits for before it cuts them into a batch. With the default of 0, pending rows are cut into a batch as soon as the committer has room for one, so batches only grow while the committer is busy. Stores that prefer fewer, larger writes can raise this; rows then wait until this many are pending, or until the oldest has waited max_batch_delay_ms. A sequential pipeline also cuts a smaller batch once its reorder buffer holds max_pending_rows rows and the next checkpoint has arrived.

max_batch_delay_ms: u64

Longest time, in milliseconds, that pending rows wait for min_batch_rows to be reached before they are written anyway. Not used while min_batch_rows is 0.

Implementations

impl CommitterConfig

pub fn max_batch_delay(&self) -> Duration

pub fn watermark_interval(&self) -> Duration

pub fn watermark_interval_with_jitter(&self) -> Instant

Returns the earliest instant at which the next watermark write may start: now plus the watermark interval plus a random jitter between 0 and watermark_interval_jitter_ms.

Trait Implementations

impl Clone for CommitterConfig

fn clone(&self) -> CommitterConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CommitterConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CommitterConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for CommitterConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for CommitterConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.