Expand description
Analytics store implementation with TransactionalStore support.
This store supports two modes:
§Live Mode
Derives watermarks from file names via bucket iteration at startup, rather than storing them separately. File uploads inherently update the watermark since file names encode checkpoint ranges.
§Migration Mode
When migration_id is set, the store operates in migration mode:
- Existing file ranges are loaded at startup and updated in-place.
- Watermark is stored in a separate file:
_metadata/watermarks/{pipeline}@migration_{id}.json - Conditional PUT with etag is used to prevent concurrent modification of data files
Structs§
- Analytics
Connection - Connection to the analytics store.
- Analytics
Store - Analytics store wrapper that delegates to an inner store mode.
- Batch
- Rows accumulated across commits, waiting to be flushed to a file.
- File
Range Entry - Entry for a single file range in the index.
- File
Range Index - Index of existing file ranges for a pipeline.
- Live
Store - Live mode - derives watermarks from file names.
- Migration
Store - Migration mode - uses explicit watermark files and conditional PUT.
Enums§
- Store
Mode - The operational mode of the analytics store.
- Watermark
Update Error - Error type for watermark updates.