Operate an Upload Relay

Walrus enables apps to store data from within end-user browsers that have low to moderate machine specifications (mobile devices, low-powered laptops, and so on). In practice, an in-browser process that directly communicates with the storage nodes cannot easily achieve this because of the high number of network connections required to upload all slivers to all shards.

The upload relay is a downloadable program that community members, Mysten Labs, and app developers can run on internet-facing hosts to facilitate storing blob slivers onto the storage nodes on behalf of end users. This mitigates browser resource consumption and enables web-based store operations.

tip

Mysten Labs runs 2 publicly available upload relays at the following addresses:

Testnet: https://upload-relay.testnet.walrus.space
Mainnet: https://upload-relay.mainnet.walrus.space

These endpoints are also listed in the Network Reference.

The upload relay offloads erasure coding and sliver fan-out from the browser. The client sends a blob and a tip to the relay; the relay distributes slivers to the storage node committee, collects confirmations, and returns an availability certificate. The client registers, certifies, and pays on Sui, and always retains ownership.

Understand the design

At a high level, a client stores a blob using an upload relay as follows:

The client locally encodes the blob and registers it on Sui.
The client sends the blob to the upload relay through an HTTP POST request to the blob-relay endpoint /v1/blob-upload-relay.
The upload relay encodes the blob, sends the slivers to the storage nodes, collects a storage confirmation certificate, and sends it back to the client.
The client uses the confirmation certificate to certify the blob on Sui.

The upload relay does not perform any onchain operation and only helps clients distribute the slivers of their blobs to storage nodes.

Blobs and quilts

The relay operates on a single blob per request. A blob is one immutable unit of data, whereas a quilt is a batch of many small blobs stored together to amortize per-blob overhead. Batch small files into a quilt on the client before you store, then relay the resulting blob. For the distinction and when to use each, see Batch Storage with Quilt.

The Walrus CLI, Rust SDK, and TypeScript SDK already implement the flow between clients and the upload relay. Developers do not need to implement it themselves. To compare upload paths, see Choose your upload path. For completeness, the following sections discuss how the service works, how clients pay for it, and how they use it.

Operation modes

You can operate the upload relay in two ways:

Free service: The relay accepts HTTP POST requests with blobs from clients and relays them to the storage nodes for free.
Paid service: In this configuration, the upload relay requires a tip to relay a blob. You can use the tip to cover the costs of running the infrastructure and earn revenue on the service. The relay currently supports a constant tip and a tip that scales linearly with the unencoded data size.

Upload relays expose a tip-configuration endpoint /v1/tip-config that returns the tipping configuration. For example:

{
  "send_tip": {
    "address": "0x1234...",
    "kind": {
    "const": 105
    }
  }
}

The configuration above specifies that every store operation requires a tip of 105 MIST (arbitrary value), paid to the set address 0x1234... Free upload relays also return this configuration, but with a value of "no_tip".

Pay the tip

This step is only necessary if the relay requires a tip.

To pay the tip, the client proceeds as follows:

Computes the blob_digest = SHA256(blob).
Generates a random nonce and hashes it: nonce_digest = SHA256(nonce).
Computes the unencoded_length = blob.len().

Then, the client creates a programmable transaction block (PTB), where the first input 0 is the bcs encoded representation of blob_digest || nonce_digest || unencoded_length. The relay later uses this to authenticate the sender of the store request. In the same PTB, the client transfers the appropriate tip amount to the wallet of the relay. You can also find this address through the /v1/tip-config endpoint. Usually, the client also registers the blob in this transaction. This is not mandatory, and you can register the blob in another transaction, but it is commonly convenient and cheaper to perform all these operations together.

After the transaction executes, the client keeps the transaction ID tx_id, the nonce, and the blob_id, which the next phase requires.

The relay enforces a freshness check on the transaction that paid the tip. The default threshold is 1 hour, but each relay can independently configure this value.

Send data to the upload relay

See the full OpenAPI specification for the upload relay for complete details.

The client sends a POST request to the /v1/blob-upload-relay API endpoint on the relay, containing the bytes of the blob to be stored in the body.

You must specify the following parameters in the query string of the URL:

blob_id: Required. The blob ID of the blob to be stored. Example: blob_id=E7_nNXvFU_3qZVu3OH1yycRG7LZlyn1-UxEDCDDqGGU
tx_id: Required if the relay requires a tip. The transaction ID (base58 encoded) of the transaction that transferred the tip to the relay. Example: tx_id=EmcFpdozbobqH61w76T4UehhC4UGaAv32uZpv6c4CNyg
nonce: Required if the relay requires a tip. The nonce (the pre-image of the hash added to the transaction inputs created above) as a base64 URL-encoded string without padding. Example: nonce=rw8xIuqxwMpdOcF_3jOprsD9TtPWfXK97tT_lWr1teQ
deletable_blob_object: Required if the blob is registered as deletable. The object ID of the blob as a hexadecimal string. If you store the blob as a permanent one, do not specify this parameter. Example: deletable_blob_object=0x56ae1c86e17db174ea002f8340e28880bc8a8587c56e8604a4fa6b1170b23a60
encoding_type: Optional. The encoding type to be used for the blob. The default value (and currently the only value) is RS2.

Receive the certificate

After the relay finishes storing the blob, it collects the confirmation certificate from the storage nodes.

The relay then sends a response to the client containing the blob_id of the stored blob along with the confirmation_certificate. The client can then use this certificate to certify the blob onchain.

Install the relay

To download a pre-built binary of walrus-upload-relay, go to the releases page. The walrus-upload-relay binary does not daemonize itself and requires a supervisor process to ensure boot at startup, restart on failure, and so on.

Use Docker

Docker Hub hosts the image for walrus-upload-relay as mysten/walrus-upload-relay.

$ docker run -it --rm mysten/walrus-upload-relay --help

Build from source

The sources for walrus-upload-relay live on GitHub in the crates/walrus-upload-relay subdirectory. Run the following from the root of the Walrus repository:

$ cargo build --release --bin walrus-upload-relay
./target/release/walrus-upload-relay --help

Configure the relay

The following example shows how to place the configuration so that the relay can reach it when you invoke Docker. This example assumes the following:

$HOME/.config/walrus/walrus_upload_relay_config.yaml exists on the host machine and contains the configuration for walrus-upload-relay, as described in Configure relay-specific settings.
$HOME/.config/walrus/client_config.yaml exists on the host machine and contains Walrus client configuration as specified in the CLI setup section.
The relay can bind to port 3000. Change this to whichever port you want to expose from your host.

$ docker run \
    -p 3000:3000 \
    -v $HOME/.config/walrus/walrus_upload_relay_config.yaml:/opt/walrus/walrus_upload_relay_config.yaml \
    -v $HOME/.config/walrus/client_config.yaml:/opt/walrus/client_config.yaml \
    mysten/walrus-upload-relay \
    --context testnet \
    --walrus-config /opt/walrus/client_config.yaml \
    --server-address 0.0.0.0:3000 \
    --relay-config /opt/walrus/walrus_upload_relay_config.yaml

Pointing the relay at a local network

The relay talks to whichever network its client configuration defines. The --context flag selects a context from the --walrus-config file, so there is no separate flag for a local network. To relay against a locally run Walrus network, add a context for it to your client_config.yaml, then pass that context name with --context. For how to run a local network, see Run a local Walrus network.

Configure relay-specific settings

The following is an example of the walrus-upload-relay configuration file:

crates/walrus-upload-relay/walrus_upload_relay_config_example.yaml
tip_config: !send_tip
  address: 0x2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a
  kind: !const 42
tx_freshness_threshold_secs: 36000
tx_max_future_threshold:
  secs: 30
  nanos: 0

The available options are:

tip_config: The configuration for the tip to be paid. Set this to !no_tip for the free service, or !send_tip to configure the requested tip. When using !send_tip, address contains the hex-encoded address of the upload relay owner where the tip should be sent, and kind specifies the type of tip. Set kind to !const for a constant tip for each store, or !linear for a tip that is linear in the unencoded blob size.
tx_freshness_threshold_secs: The maximum number of seconds the relay considers a transaction that pays the tip valid.
tx_max_future_threshold_secs: The maximum amount of time in the future for which the relay accepts a tip-paying transaction as valid. This accounts for some clock skew.

Understand the design​

Operation modes​

Pay the tip​

Send data to the upload relay​

Receive the certificate​

Install the relay​

Use Docker​

Build from source​

Configure the relay​

Configure relay-specific settings​