Skip to main content

Using Quilts with the Walrus Client

For efficiently storing large numbers of small blobs, Walrus provides the quilt feature. A quilt batches multiple blobs into a single storage unit, significantly reducing overhead and cost. Learn more about quilts.

You can interact with quilts using a dedicated set of walrus subcommands.

Blobs within a quilt are retrieved by a QuiltPatchId, not their standard BlobId. This ID is generated based on all blobs in the quilt, so a blob's QuiltPatchId changes if it is moved to a different quilt.

Standard blob operations like delete, extend, or share cannot target individual blobs inside a quilt. You must apply them to the entire quilt.

Store files as a quilt​

To store all files from one or more directories recursively, use the --paths flag. The filename of each file is used as its unique identifier within the quilt. Regular expressions are supported for uploading from multiple paths.

Like the regular store command, you can specify the storage duration using --epochs, --earliest-expiry-time, or --end-epoch.

$ walrus store-quilt --epochs <EPOCHS> --paths <PATH_TO_DIRECTORY_1> <PATH_TO_DIRECTORY_2> <PATH_TO_BLOB>

All identifiers must be unique within a quilt. The operation fails otherwise. Identifiers are the unique names used to retrieve individual blobs from within the quilt.

To specify a list of blobs as JSON objects, use the --blobs flag. This gives you more control, allowing you to set a custom identifier and tags for each file. If identifier is null or omitted, the file name is used instead.

$ walrus store-quilt \
    --blobs '{"path":"<PATH_TO_BLOB_1>","identifier":"walrus","tags":{"color":"grey","size":"medium"}}' \
            '{"path":"<PATH_TO_BLOB_2>","identifier":"seal","tags":{"color":"grey","size":"small"}}' \
    --epochs <EPOCHS>

Read blobs from a quilt​

You can retrieve individual blobs from a quilt without downloading the entire quilt. The read-quilt command allows you to query for specific blobs by their identifier, tags, or unique blob ID.

To read blobs by their identifiers, use the --identifiers flag:

$ walrus read-quilt --out <DOWNLOAD_DIR> \
    --quilt-id 057MX9PAaUIQLliItM_khR_cp5jPHzJWf-CuJr1z1ik --identifiers walrus.jpg another-walrus.jpg

You can access and filter blobs within a quilt based on their tags. If you have a collection of animal images stored in a quilt, each labeled with a species tag such as species=cat, you can download all images labeled as cats with the following command:

$ walrus read-quilt --out <DOWNLOAD_DIR> \
    --quilt-id 057MX9PAaUIQLliItM_khR_cp5jPHzJWf-CuJr1z1ik --tag species cat

You can also read a blob using its QuiltPatchId, which you can retrieve using walrus list-patches-in-quilt:

$ walrus read-quilt --out <DOWNLOAD_DIR> \
  --quilt-patch-ids GRSuRSQ_hLYR9nyo7mlBlS7MLQVSSXRrfPVOxF6n6XcBuQG8AQ \
  GRSuRSQ_hLYR9nyo7mlBlS7MLQVSSXRrfPVOxF6n6XcBwgHHAQ

To see all patches contained within a quilt along with their identifiers and QuiltPatchIds, use the list-patches-in-quilt command:

$ walrus list-patches-in-quilt 057MX9PAaUIQLliItM_khR_cp5jPHzJWf-CuJr1z1ik