Introduction

Oyster is a Web2-friendly object storage service backed by Walrus (decentralized blob storage) and Sui (onchain state). It gives you familiar HTTP and S3 APIs while your data is stored on a decentralized network.

Core Concepts

Accounts

Every user has an account. Your Oyster administrator creates accounts and issues you an initial API key (Bearer token). With that token you can create additional API keys, manage buckets and blobs, and generate S3-compatible access keys.

API Keys

An API key is a Bearer token used to authenticate JSON API requests. You include it in the Authorization header:

Authorization: Bearer <your-api-key>

The plaintext key is shown exactly once, at creation time. Store it securely.

Buckets

Buckets are named containers for your blobs. Bucket names are globally unique and follow S3-style naming rules:

• 3–63 characters long
• Lowercase letters, digits, and hyphens only
• Must start and end with a letter or digit
• No consecutive hyphens
• Cannot look like an IP address (for example, 192.168.1.1)

Blobs

A blob is a binary object stored inside a bucket, identified by a user-chosen key (like a file path, for example images/photo.png). Blobs are content-addressed: identical content is stored once and can be referenced by multiple keys.

Key properties of blobs:

• Public reads: anyone can download a blob by bucket name and key, or by its content-addressed blob ID. No authentication is needed for reads.
• Authenticated writes: uploading, deleting, and listing blobs requires a valid API key or S3 credentials.
• Overwrite semantics: uploading to an existing key replaces the blob.
• Reference-counted deletion: deleting a key removes the reference; the underlying data is only removed when no other keys point to it.
• Expiration: blobs share their account's StoragePool lifetime rather than expiring individually. A background extension service renews each pool before it expires; see Blob Lifecycle for details.

Two API Surfaces

Oyster exposes two ways to interact with your data:

JSON API

A RESTful HTTP API under /api/v1/. Use it with curl, any HTTP client, or the oyster-cli command-line tool. Responses are JSON. This API covers everything: account management, bucket/blob CRUD, S3 access key management, wallet info, and more.

S3-Compatible API

An AWS S3-compatible interface that speaks the same protocol as Amazon S3. Use the AWS CLI, boto3, the AWS SDK for JavaScript, or any S3-compatible client. Authenticate with S3 access keys (created through the JSON API) using standard AWS Signature Version 4.

Both APIs share the same underlying storage and database. Changes made through one are immediately visible in the other.

What's Next

• Getting Started: set up credentials and make your first API calls.
• JSON API Reference: full endpoint documentation.
• S3 API Reference: S3-compatible operations and AWS CLI setup.
• Guides: CLI quick start, SDK examples, and deeper topics.

Getting Started

Follow these steps to complete your first interactions with Oyster. By the end, you have created a bucket, uploaded a blob, and downloaded it back.

Prerequisites

• curl: for making HTTP requests to the JSON API
• AWS CLI (optional): for using the S3-compatible API (install guide)

Obtaining Credentials

Oyster uses a two-tier auth model: operators manage accounts with long-lived per-app admin keys, and end users authenticate data operations with API keys. Both tiers use Authorization: Bearer <hex>; the route prefix selects which credential table is consulted.

For Operators

The server operator creates an app, gets back a first admin key, and provisions accounts. See the Admin API docs for full details.

# 1. Create an app (server operator runs this once). `app new` auto-issues
#    a first admin key by default; pass --no-key to opt out.
oysterd app new --name my-app --contact_email admin@example.com
# Prints: 550e8400-e29b-41d4-a716-446655440000   <- app id
# Prints: <64-char hex admin key>                 <- save this

export ADMIN_KEY="<64-char hex admin key from above>"

# 2. Create an account using the admin key
export OYSTER_URL="http://localhost:3000"
curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-app-user"}' \
  "$OYSTER_URL/api/v1/accounts" | jq

The response includes the account ID and an initial API key:

{
  "account_id": "550e8400-e29b-41d4-a716-446655440000",
  "api_key": {
    "id": "b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e",
    "prefix": "a1b2c3d4",
    "bearer_token": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
    "created_at": "2025-01-15T10:30:00Z"
  }
}

Save the account_id and bearer_token. The token is only shown once.

export ACCOUNT_ID="550e8400-e29b-41d4-a716-446655440000"
export API_KEY="a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2"

For End Users

Your operator provides you with an API key. Store it in an environment variable for the rest of this guide:

export OYSTER_URL="http://localhost:3000"
export API_KEY="your-api-key-here"

Create Your First Bucket

Buckets are named containers for your blobs. Create one called my-bucket:

curl -s -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-bucket"}' \
  "$OYSTER_URL/api/v1/buckets" | jq

Response:

{
  "name": "my-bucket",
  "account_id": "550e8400-e29b-41d4-a716-446655440000",
  "created_at": "2025-01-15T10:30:00Z"
}

Upload a Blob

Upload a text file to your bucket with the key hello.txt:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  --data-binary "Hello, Oyster!" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt" | jq

Response:

{
  "key": "hello.txt",
  "blob_id": "2cf24dba5fb0a30e...",
  "size": 14,
  "md5": "9a0364b9e99bb480...",
  "sui_object_id": null,
  "created_at": "2025-01-15T10:31:00Z"
}

You can also upload a file from disk:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  --data-binary @photo.png \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/images/photo.png" | jq

Download a Blob

Blob reads are public (no authentication needed):

curl -s "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Output:

Hello, Oyster!

List Blobs in a Bucket

curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs" | jq

Response:

{
  "data": [
    {
      "key": "hello.txt",
      "blob_id": "2cf24dba5fb0a30e...",
      "bucket_name": "my-bucket",
      "content_type": "text/plain",
      "size": 14,
      "md5": "9a0364b9e99bb480...",
      "created_at": "2025-01-15T10:31:00Z"
    }
  ],
  "next_cursor": null
}

Delete a Blob

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Returns HTTP 204 (no content) on success.

Create Additional API Keys (Operator)

Additional API keys are created by the operator through the Admin API using admin-key authentication. This requires the $ADMIN_KEY and $ACCOUNT_ID variables from the Obtaining Credentials section.

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/$ACCOUNT_ID/api-keys" | jq

Response:

{
  "id": "b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e",
  "prefix": "a1b2c3d4",
  "bearer_token": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
  "created_at": "2025-01-15T10:32:00Z"
}

Important: The bearer_token field is only shown once. Save it immediately.

See the Admin API docs for full details including error handling and key revocation.

Set Up S3 Access Keys (Operator)

To use the AWS CLI or any S3-compatible SDK, the operator creates S3 access keys through the Admin API. This requires the $ADMIN_KEY and $ACCOUNT_ID variables from the Obtaining Credentials section.

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/$ACCOUNT_ID/access-keys" | jq

Response:

{
  "access_key_id": "OYAK1234567890ABCDEF",
  "secret_access_key": "abcdef1234567890abcdef1234567890abcdef12",
  "created_at": "2025-01-15T10:33:00Z"
}

Important: The secret_access_key is only shown once. Save it immediately. You can have up to 3 active S3 access keys per account.

Then configure the AWS CLI:

aws configure set aws_access_key_id "OYAK1234567890ABCDEF" --profile oyster
aws configure set aws_secret_access_key "abcdef1234567890..." --profile oyster
aws configure set region "us-east-1" --profile oyster
aws configure set endpoint_url "$OYSTER_URL" --profile oyster

Now you can use standard S3 commands:

# Create a bucket
aws --profile oyster s3api create-bucket --bucket my-s3-bucket

# Upload a file
aws --profile oyster s3api put-object \
  --bucket my-s3-bucket --key hello.txt --body hello.txt

# Download a file
aws --profile oyster s3api get-object \
  --bucket my-s3-bucket --key hello.txt downloaded.txt

For the full S3 API reference, see S3 API Reference.

What's Next

• JSON API Reference: detailed documentation of every endpoint.
• S3 API Reference: complete S3-compatible operations and setup.
• Guides: CLI quick start, SDK examples, and advanced topics.

JSON API Reference

The Oyster JSON API is served under /api/v1/. All requests and responses use JSON (except blob content, which is raw binary). Authenticated endpoints require a Bearer token in the Authorization header.

Base URL

All API endpoints are prefixed with /api/v1:

$OYSTER_URL/api/v1/

Throughout this reference, $OYSTER_URL is set to your Oyster server address (for example, http://localhost:3000).

Authentication

Most endpoints require a Bearer token:

Authorization: Bearer <your-api-key>

Endpoints that do not require authentication:

• Reading blobs by key or blob ID
• Health, readiness, and metrics probes
• OpenAPI documentation

Error Responses

All errors return a JSON body with a single error field:

{
  "error": "human-readable error message"
}

Status Codes

501 is currently produced only by three account stubs that are wired up but not yet implemented: PUT /account/billing, GET /account/report, and POST /account/transfer. These are intentionally omitted from the per-endpoint reference until they are functional.

Cross-Cutting Error Contracts

A few error bodies are shared across multiple routes and carry a structured block alongside the standard error string. Document once here; per-route docs link back.

• InsufficientBalance (402): the Pearl-derived wallet doesn't hold enough WAL or SUI to fund the onchain action. Body carries a funding_required: { wal_frost, sui_mist } block (both decimal strings). Currently fires on PUT /buckets/{bucket}/blobs/{key} (see Store Blob) and DELETE /buckets/{bucket}/blobs/{key} (see Delete Blob). When the lookup itself fails, funding_required is null.
• CapExceeded (400): the upload would push the account past its per-account max_unencoded_bytes cap. Body carries a cap_exceeded block pointing at the admin endpoint that can raise the cap. Currently fires on PUT /buckets/{bucket}/blobs/{key} (see Store Blob). The cap is an upper bound by default; a per-account avg_blob_size turns it into a lower bound on storable capacity for blobs of that size.

The admin-side onchain shrink endpoint (PUT /accounts/{account_id}/max-storage) has its own 400 variants (would_orphan, shrink_aborted) documented in the admin reference.

Pagination

List endpoints use cursor-based pagination:

Query parameters:

• cursor (optional): opaque string from a previous response's next_cursor
• limit (optional): number of items per page (default: 20, max: 100)

Response format:

{
  "data": [ ... ],
  "next_cursor": "opaque-cursor-string"
}

When next_cursor is null, there are no more results. To fetch the next page, pass the next_cursor value as the cursor query parameter.

Example: paginating through buckets:

# First page
curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets?limit=10" | jq

# Next page (using next_cursor from previous response)
curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets?limit=10&cursor=eyJjcmVhdGVk..." | jq

Interactive Documentation

Oyster serves an interactive OpenAPI UI at:

$OYSTER_URL/api/docs

Explore and test all endpoints directly from your browser.

Authentication

Oyster uses Bearer tokens for authenticated routes plus public access for blob reads and infrastructure probes. There are two tiers of Bearer token, distinguished by which routes they unlock; both share the same Authorization: Bearer <hex> wire format.

Authentication Modes at a Glance

How Oyster tells them apart: the URL prefix selects the credential table. Admin routes look up the Bearer token in the app_admin_keys table (one app per key); data routes look it up in the api_keys table (one account per key). Both tokens are 64-char hex; the hash check happens on whichever table the route is statically wired to.

Bearer Token (API Key) Authentication

Include your API key in the Authorization header:

Authorization: Bearer <api-key>

Example:

curl -s \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets"

Key properties

API keys are provisioned through the Admin API (see Admin). The full secret is shown only once at creation time. A lost key cannot be recovered.

Errors

Admin-Key Authentication (for Apps)

Admin endpoints require a per-app admin key issued by the server operator:

Authorization: Bearer <admin-key>

Admin keys are generated server-side with oysterd app issue-admin-key <app_id>. They are not available through a public API. Multiple admin keys per app are supported (AWS-style two-key rotation, no cap).

Key properties

Account ownership enforcement

An app can only manage accounts it created. Attempting to access another app's accounts returns 403 Forbidden:

{ "error": "forbidden: account does not belong to this app" }

Rotation

Admin keys do not expire. The recommended pattern is AWS-style two-key overlap:

1. Operator issues a new key alongside the old one (oysterd app issue-admin-key <APP_ID>).
2. Callers swap to the new key.
3. After confirming nothing still uses the old key, the operator revokes it (oysterd app revoke-admin-key <OLD_KEY_ID>). Revocation takes effect immediately, with no caching.

oysterd app list-admin-keys <APP_ID> shows all keys (active and revoked) so an operator can audit before revoking.

Errors

Public Endpoints (No Authentication)

The following routes require no authentication:

• Blob reads: GET /api/v1/buckets/{bucket_name}/blobs/{key} and GET /api/v1/blobs/by-blob-id/{blob_id}
• Infrastructure: /health, /ready, /metrics, /api/docs

Example:

curl -s "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Security Notes

• API keys and admin keys: Only the BLAKE2s-256 hash is stored. A lost key cannot be recovered; issue a new one instead.
• Admin-key compromise: A leaked admin key gives full app-admin access (account creation, key issuance, S3 access keys) until revoked. Treat it like a long-lived service credential and rotate it periodically and on personnel changes through oysterd app issue-admin-key + oysterd app revoke-admin-key.
• TLS: Always terminate TLS in front of Oyster in production.

Admin API

The Admin API lets app operators manage accounts, API keys, and S3 access keys. All admin endpoints require admin-key authentication (long-lived per-app Bearer tokens issued through oysterd app issue-admin-key; see Authentication).

An app can only manage accounts it created. Attempting to access another app's accounts returns 403 Forbidden.

Accounts

Create Account

POST /api/v1/accounts

Creates a new account owned by the authenticated app. An initial API key is generated automatically.

Request body (optional):

{
  "name": "my-app-user",
  "max_unencoded_bytes": 5000000000,
  "avg_blob_size": 10000000
}

Example:

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-app-user"}' \
  "$OYSTER_URL/api/v1/accounts" | jq

Response (201 Created):

{
  "account_id": "550e8400-e29b-41d4-a716-446655440000",
  "api_key": {
    "id": "b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e",
    "prefix": "a1b2c3d4",
    "bearer_token": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
    "created_at": "2025-01-15T10:30:00Z"
  }
}

The bearer_token is returned only at creation time. A lost key cannot be recovered; create a new one instead.

Errors:

Update Storage Cap

PUT /api/v1/accounts/{account_id}/max-storage

Raises or lowers the per-account max_unencoded_bytes cap. Lowering the cap below the account's current onchain encoded usage is rejected; lowering between current usage and current pool capacity submits an onchain shrink transaction to release the freed reserve back to the Pearl-derived wallet.

Path parameters:

Request body:

{ "max_unencoded_bytes": 10000000000, "avg_blob_size": 10000000 }

Example:

curl -s -X PUT \
  -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"max_unencoded_bytes": 10000000000}' \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/max-storage" | jq

Response (200 OK):

{
  "account_id": "550e8400-e29b-41d4-a716-446655440000",
  "max_unencoded_bytes": 10000000000,
  "avg_blob_size": 10000000,
  "pool": {
    "reserved_encoded_bytes": 8000000000,
    "used_encoded_bytes": 4123456789
  },
  "shrink_tx_digest": "5jK...digest"
}

When the account has no pool yet (no upload has lazy-created one), the cap is updated in the DB only (pool and shrink_tx_digest are both null). When the new cap covers the existing pool's reserved bytes, the same DB-only path runs (shrink_tx_digest is null, pool is populated from the onchain read).

Lower-bound semantics (avg_blob_size)

max_unencoded_bytes is stated in unencoded bytes, but Walrus tracks usage in encoded bytes, and the encoding f carries a large fixed per-blob metadata overhead. By default (avg_blob_size = 0) the cap therefore acts as an upper bound: an account storing many small blobs pays that overhead per blob and hits the cap well below its stated unencoded budget.

Setting avg_blob_size = s flips this into a lower bound for blobs averaging s: Oyster inflates the encoded admission ceiling by the per-blob expansion factor f(s)/s, guaranteeing that at least max_unencoded_bytes unencoded bytes are storable when the account's blobs average ≥ s. The expansion factor shrinks as blobs grow: roughly 66034× at 1 KB, 70× at 1 MB, 11× at 10 MB, 5.1× at 100 MB, asymptoting to ~4.5×. A 10 MB avg_blob_size (the default) sets the ceiling at about 11 × the cap's bare encoded value. Blobs smaller than s carry more overhead and so reach the ceiling before the unencoded total reaches max_unencoded_bytes.

This only raises the admission ceiling; it does not pre-reserve or pre-pay capacity. Onchain pool capacity still grows incrementally per upload (pay-as-you-go for actual encoded usage), so a non-zero avg_blob_size costs nothing for normal workloads. Setting avg_blob_size = 0 reproduces the historical upper-bound behavior byte-for-byte; accounts created before this feature default to 0.

Onchain shrink semantics. When the new cap is lower than the pool's current reserved capacity and at least one encoded byte can be freed without orphaning data, Oyster submits a Pearl-signed system::decrease_storage_pool_capacity_by_size PTB. The contract extracts a Storage object covering the freed bytes and transfers it back to the pool's owner (the Pearl-managed sender). Over time these extracted Storage objects accumulate in the wallet¹.

Errors:

would_orphan body: emitted when lowering would drop the cap below current onchain usage:

{
  "error": "max-storage update would orphan stored data: ...",
  "would_orphan": {
    "max_unencoded_bytes": 1000000000,
    "used_encoded_bytes": 4123456789,
    "threshold_encoded": 1500000000
  }
}

shrink_aborted body: emitted when the shrink PTB aborted because another replica's upload raced and re-consumed the capacity Oyster was about to extract:

{
  "error": "max-storage shrink aborted: ...",
  "shrink_aborted": {
    "move_abort_description": "EInsufficientCapacity in storage_pool::extract_storage",
    "max_unencoded_bytes": 2000000000,
    "extract_size": 1234567890
  }
}

Both would_orphan and shrink_aborted are safe to retry after the underlying state has settled (for example, delete some blobs to lower used_encoded_bytes, or wait for the concurrent upload to finish).

A successful cap change writes an account.max_storage_updated audit event.

API Keys

Create API Key

POST /api/v1/accounts/{account_id}/api-keys

Creates a new API key for an existing account.

Path parameters:

Example:

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/api-keys" | jq

Response (201 Created):

{
  "id": "b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e",
  "prefix": "a1b2c3d4",
  "bearer_token": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
  "created_at": "2025-01-15T10:30:00Z"
}

Errors:

Revoke API Key

DELETE /api/v1/accounts/{account_id}/api-keys/{key_id}

Revokes an API key. The key immediately stops working for authentication.

Path parameters:

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/api-keys/b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e"

Response: 204 No Content

Errors:

S3 Access Keys

These endpoints manage S3-compatible access keys for accounts. See S3 Access Keys for key format details and limits.

Create Access Key

POST /api/v1/accounts/{account_id}/access-keys

Creates a new S3 access key pair. The secret is returned only once, so save it immediately. Each account can have up to 3 active access keys.

Path parameters:

Example:

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/access-keys" | jq

Response (201 Created):

{
  "access_key_id": "OYAK1234567890ABCDEF",
  "secret_access_key": "abcdef1234567890abcdef1234567890abcdef12",
  "created_at": "2025-01-15T10:30:00Z"
}

Errors:

List Access Keys

GET /api/v1/accounts/{account_id}/access-keys

Returns all S3 access keys for the account, including revoked ones. Secrets are never included in list responses.

Path parameters:

Example:

curl -s \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/access-keys" | jq

Response (200 OK):

[
  {
    "access_key_id": "OYAK1234567890ABCDEF",
    "created_at": "2025-01-15T10:30:00Z",
    "revoked_at": null
  },
  {
    "access_key_id": "OYAKFEDCBA0987654321",
    "created_at": "2025-01-10T08:00:00Z",
    "revoked_at": "2025-01-14T12:00:00Z"
  }
]

Errors:

Revoke Access Key

DELETE /api/v1/accounts/{account_id}/access-keys/{access_key_id}

Revokes an S3 access key. Any S3 requests using this key stop working immediately. Revoked keys no longer count toward the 3-key active limit.

Path parameters:

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/550e8400-e29b-41d4-a716-446655440000/access-keys/OYAK1234567890ABCDEF"

Response: 204 No Content

Errors:

App

Get App

GET /api/v1/admin/app

Returns the authenticated app, including the current webhook URL and the base64-encoded Ed25519 public key paired with it. Use this endpoint when you need to retrieve the response from PUT /admin/app/webhook.

Response (200 OK):

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "my-app",
  "contact_email": "admin@example.com",
  "webhook_url": "https://example.com/oyster/webhook",
  "webhook_public_key": "base64-encoded-32-byte-key",
  "created_at": "2025-01-15T10:30:00Z"
}

Errors:

Set Webhook URL

PUT /api/v1/admin/app/webhook

Registers or rotates the webhook URL for the authenticated app. Each call generates a fresh Ed25519 keypair; the response is the only opportunity to capture the public key for verification. Subsequent deliveries are signed with the corresponding private key. See Webhooks for the signature format.

Request body:

{ "webhook_url": "https://example.com/oyster/webhook" }

Response (200 OK): same shape as GET /admin/app above.

Errors:

Clear Webhook URL

DELETE /api/v1/admin/app/webhook

Clears the webhook URL and discards the keypair. Subsequent extension failures do not deliver a webhook.

Response (200 OK): the updated app row with all three webhook fields nulled.

Errors:

Server Commands

oysterd is the server binary. Besides the oysterd app subcommands below (for managing apps and admin keys), it runs the service itself.

Running the Server

oysterd serve     # run the HTTP + S3 server (default when no subcommand given)
oysterd extend     # run the background blob-extension service only

oysterd serve is the default; running oysterd with no subcommand is equivalent. oysterd extend runs only the background task that renews expiring storage pools (see Blob Lifecycle), without serving the API.

The global --pearl-service-secret-file <PATH> flag reads the Pearl service secret from a file instead of the PEARL_SERVICE_SECRET environment variable, which is useful for mounting the secret as a file (for example, a Kubernetes secret). It applies to any oysterd invocation.

Both services are otherwise configured through environment variables; see the README for the full env-var reference.

Create App

oysterd app new --name <NAME> --contact_email <EMAIL> [--no-key]

Creates a new app, prints its UUID, and (by default) auto-issues a first admin key alongside. Webhook URLs are configured by the app builder using the self-service PUT /admin/app/webhook endpoint above (or oyster app webhook set <URL>).

Example:

oysterd app new --name "my-app" --contact_email "admin@example.com"
# 550e8400-e29b-41d4-a716-446655440000
# 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Issue Admin Key

oysterd app issue-admin-key <app_id>

Generates a fresh admin key for the given app. Multiple admin keys per app are supported with no cap; use this for AWS-style two-key rotation.

stdout carries the raw admin key as a single line; this is the only machine-readable output, suitable for capturing in a variable or piping. The key id and 8-char prefix are not printed to stdout; they appear in a tracing::info! structured log line (fields app_id, key_id, prefix, message issued admin key) written to stderr. That line is emitted at the info level, so it shows with the default log filter but is suppressed if RUST_LOG raises the threshold above info. It is a human-readable log line, not a stable machine-readable string. To recover a key id reliably, use List Admin Keys.

Example:

oysterd app issue-admin-key 550e8400-e29b-41d4-a716-446655440000
# stdout: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
# stderr (info log): ... issued admin key app_id=550e8400-... key_id=<key_id> prefix=01234567

The printed key can be used directly in the Authorization header:

curl -H "Authorization: Bearer $(oysterd app issue-admin-key $APP_ID)" ...

List Admin Keys

oysterd app list-admin-keys <app_id>

Lists all admin keys for the given app in tab-separated format, including revoked ones (so an operator can confirm what is currently live). There is no header row; columns are id, prefix, created_at, revoked_at. The revoked_at column is the empty string for active keys (so a line for an active key ends with a trailing tab).

Example output (→ marks a tab):

b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e→01234567→2026-04-15T10:30:00Z→
a1b2c3d4-e5f6-7890-abcd-ef0123456789→89abcdef→2026-03-01T08:00:00Z→2026-04-15T10:31:00Z

Revoke Admin Key

oysterd app revoke-admin-key <key_id>

Marks an admin key as revoked. Subsequent requests using that key are rejected with 401. Revocation is by key_id (globally unique), not by the raw key value.

Example:

oysterd app revoke-admin-key b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e

List Apps

oysterd app list

Lists all registered apps in tab-separated format.

Example output:

ID	NAME	CONTACT_EMAIL
550e8400-e29b-41d4-a716-446655440000	my-app	admin@example.com

Buckets

Buckets are named containers that hold your blobs. Bucket names are globally unique: no two accounts can have a bucket with the same name.

Bucket Naming Rules

Bucket names must follow these rules:

• 3–63 characters long
• Only lowercase letters, digits, and hyphens (-)
• Must start and end with a letter or digit
• No consecutive hyphens (--)
• Cannot look like an IP address (for example, 192.168.1.1)
• Cannot use reserved names: health, ready, metrics, api

Valid examples: my-bucket, data-2025, images

Invalid examples: My-Bucket (uppercase), a (too short), -bucket (starts with hyphen), my--bucket (consecutive hyphens)

Create Bucket

POST /api/v1/buckets

Request body:

{
  "name": "my-bucket"
}

Example:

curl -s -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-bucket"}' \
  "$OYSTER_URL/api/v1/buckets" | jq

Response (201 Created):

{
  "name": "my-bucket",
  "account_id": "550e8400-e29b-41d4-a716-446655440000",
  "created_at": "2025-01-15T10:30:00Z"
}

Errors:

List Buckets

GET /api/v1/buckets

Returns a paginated list of buckets owned by your account.

Query parameters:

Example:

curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets?limit=10" | jq

Response (200 OK):

{
  "data": [
    {
      "name": "my-bucket",
      "account_id": "550e8400-e29b-41d4-a716-446655440000",
      "created_at": "2025-01-15T10:30:00Z"
    },
    {
      "name": "logs-2025",
      "account_id": "550e8400-e29b-41d4-a716-446655440000",
      "created_at": "2025-01-16T08:00:00Z"
    }
  ],
  "next_cursor": null
}

When next_cursor is not null, pass it as the cursor query parameter to fetch the next page.

Delete Bucket

DELETE /api/v1/buckets/{bucket_name}

Deletes a bucket. The bucket must be empty; delete all blobs first.

Path parameters:

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket"

Response: 204 No Content

Errors:

Blobs

Blobs are binary objects stored inside buckets. Each blob is identified by a user-chosen key (like a file path) and has a content-addressed blob ID computed from its contents.

Key properties:

• Reads are public: no authentication needed to download a blob
• Writes require auth: uploading, updating, and deleting need a Bearer token
• Overwrite semantics: uploading to an existing key replaces the blob
• Content-addressed: identical content is stored only once
• Reference-counted deletion: underlying data is removed only when no keys reference it
• Pool-scoped expiration: blobs share their account's StoragePool lifetime; a background extension service renews each pool before it expires (see Blob Lifecycle)

Store Blob

PUT /api/v1/buckets/{bucket_name}/blobs/{key}

Uploads binary data to the specified bucket and key. If a blob already exists at that key, it is replaced.

Path parameters:

Request headers:

Request body: Raw binary data (max 1 GB)

Example: upload a string:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  --data-binary "Hello, Oyster!" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt" | jq

Example: upload a file:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: image/png" \
  --data-binary @photo.png \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/images/photo.png" | jq

Example: create only (fail if key exists):

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  -H "If-None-Match: *" \
  --data-binary "Hello, Oyster!" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt" | jq

Example: safe overwrite (only if ETag matches):

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  -H 'If-Match: "9a0364b9e99bb480dd25e1f0284c8555"' \
  --data-binary "Updated content" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt" | jq

Example: upload with tags:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  -H "x-oyster-tag: env=prod" \
  -H "x-oyster-tag: team=platform" \
  --data-binary "Hello, Oyster!" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt" | jq

Each x-oyster-tag header carries exactly one key=value pair (percent-decoded; no &-joined pairs). Tags are replaced on every PUT, so re-uploading a key without any x-oyster-tag headers clears its tags. The same caps as the Blob Tags endpoints apply. See Tag rules.

Response (201 Created):

{
  "key": "hello.txt",
  "blob_id": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824",
  "size": 14,
  "md5": "9a0364b9e99bb480dd25e1f0284c8555",
  "sui_object_id": null,
  "created_at": "2025-01-15T10:31:00Z"
}

The response includes an ETag header containing the quoted MD5 digest (for example, "9a0364b9e99bb480dd25e1f0284c8555").

Errors:

When the cap is exceeded the response body looks like:

{
  "error": "storage cap exceeded: ...",
  "cap_exceeded": {
    "max_unencoded_bytes": 5000000000,
    "used_encoded_bytes": 4998123456,
    "new_unencoded_bytes": 16384,
    "admin_endpoint": "PUT /api/v1/accounts/{account_id}/max-storage"
  }
}

The cap is enforced in unencoded bytes against onchain encoded usage through the same f = encoded_blob_length_for_n_shards that the upload path uses to project the post-upload encoded total, so the short-circuit fires before any onchain work. Raise (or lower) the cap through the admin Update Storage Cap endpoint.

Read Blob by Key

GET /api/v1/buckets/{bucket_name}/blobs/{key}

Downloads a blob's contents. No authentication required.

Path parameters:

Example:

curl -s "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Conditional headers:

Example: cache validation:

curl -s -o /dev/null -w "%{http_code}" \
  -H 'If-None-Match: "9a0364b9e99bb480dd25e1f0284c8555"' \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"
# Returns 304 if unchanged, 200 with body if changed

Response (200 OK):

• Body: Raw binary blob data
• Content-Type: The MIME type set during upload
• ETag: Quoted MD5 digest (for example, "9a0364b9e99bb480dd25e1f0284c8555")
• Content-Disposition: attachment. Because reads are public and serve a caller-supplied Content-Type, blobs are returned as downloads so a text/html/SVG payload can't execute as a page on the Oyster origin. The response also carries X-Content-Type-Options: nosniff and Content-Security-Policy: default-src 'none'; sandbox. Embedding a blob as a subresource (<img>, <video>, <script src>, fetch) is unaffected; only direct top-level navigation downloads instead of rendering.

Errors:

Read Blob by Blob ID

GET /api/v1/blobs/by-blob-id/{blob_id}

Downloads a blob by its content-addressed hash. Useful when you know the blob ID but not which bucket or key it's stored under. No authentication required.

Path parameters:

Example:

curl -s "$OYSTER_URL/api/v1/blobs/by-blob-id/2cf24dba5fb0a30e..."

Response (200 OK):

• Body: Raw binary blob data
• Content-Type: application/octet-stream

Errors:

List Blobs

GET /api/v1/buckets/{bucket_name}/blobs

Returns a paginated list of blobs in a bucket.

Path parameters:

Query parameters:

Example:

curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs?limit=50" | jq

Response (200 OK):

{
  "data": [
    {
      "key": "hello.txt",
      "blob_id": "2cf24dba5fb0a30e...",
      "bucket_name": "my-bucket",
      "account_id": "550e8400-e29b-41d4-a716-446655440000",
      "content_type": "text/plain",
      "size": 14,
      "md5": "9a0364b9e99bb480...",
      "sui_object_id": null,
      "created_at": "2025-01-15T10:31:00Z"
    }
  ],
  "next_cursor": null
}

Update Blob Metadata

PATCH /api/v1/buckets/{bucket_name}/blobs/{key}/metadata

Updates metadata for an existing blob. Currently only content_type can be changed.

Path parameters:

Request body:

{
  "content_type": "image/png"
}

Example:

curl -s -X PATCH \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"content_type": "image/png"}' \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/photo.png/metadata" | jq

Response (200 OK): Full blob metadata (same shape as items in List Blobs).

Errors:

Delete Blob

DELETE /api/v1/buckets/{bucket_name}/blobs/{key}

Deletes a blob by key. The underlying data is only removed from storage when no other keys reference the same content (reference-counted deletion).

Path parameters:

Conditional headers:

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Example: delete only if ETag matches:

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  -H 'If-Match: "9a0364b9e99bb480dd25e1f0284c8555"' \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt"

Response: 204 No Content

Errors:

A 402 carries the same funding_required block as the upload path:

{
  "error": "insufficient balance: ...",
  "funding_required": {
    "wal_frost": "1234567890",
    "sui_mist": "98765432"
  }
}

wal_frost and sui_mist are decimal strings (Pearl-derived wallet's owed top-up); inspect your wallet through Get Wallet Address. When delete_blob returns 402, the DB row is left intact on purpose so the client can fund the Pearl-derived wallet and retry the same DELETE. Other onchain delete errors are still swallowed to preserve idempotent-delete semantics.

Blob Tags

Each blob can carry a small set of arbitrary key=value tags, stored in Oyster's database (independent of the underlying blob content). Tags set through this JSON API and tags set through the S3 Object Tagging operations share the same backing store. A tag written through one API is visible through the other.

All tag endpoints live under a blob's /tags path, require Bearer auth, and return 404 if the blob does not exist or is not owned by the authenticated account.

Tag rules

Allowed characters in keys and values: ASCII alphanumerics plus space and + - = . _ : / @. Keys must be non-empty; values might be empty. Duplicate keys are rejected. Any request whose resulting tag set violates these rules returns 400.

Get Tags

GET /api/v1/buckets/{bucket_name}/blobs/{key}/tags

Returns all tags on the blob.

Example:

curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags" | jq

Response (200 OK):

{
  "tags": {
    "env": "prod",
    "team": "platform"
  }
}

Replace Tags

PUT /api/v1/buckets/{bucket_name}/blobs/{key}/tags

Replaces the blob's entire tag set with the supplied map. Tags not present in the request are removed.

Request body:

{ "tags": { "env": "prod", "team": "platform" } }

Example:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tags": {"env": "prod", "team": "platform"}}' \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags"

Response: 204 No Content

Errors: 400 if the tag set violates tag rules; 401 if unauthenticated; 404 if the blob is not found.

Merge Tags

PATCH /api/v1/buckets/{bucket_name}/blobs/{key}/tags

Merges the supplied map into the blob's existing tags (upsert per key). Keys not mentioned in the request are left untouched.

Request body:

{ "tags": { "team": "storage" } }

Example:

curl -s -X PATCH \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tags": {"team": "storage"}}' \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags"

Response: 204 No Content

Errors: 400 if the merged set would violate tag rules (for example, exceed the 10-tag cap); 401 if unauthenticated; 404 if the blob is not found.

Delete All Tags

DELETE /api/v1/buckets/{bucket_name}/blobs/{key}/tags

Clears every tag on the blob.

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags"

Response: 204 No Content

Set a Single Tag

PUT /api/v1/buckets/{bucket_name}/blobs/{key}/tags/{tag_key}

Upserts a single tag. The request body is the raw tag value as text/plain (not JSON).

Example:

curl -s -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: text/plain" \
  --data-binary "prod" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags/env"

Response: 204 No Content

Errors: 400 if adding the tag would exceed the 10-tag cap or the value/key violates tag rules; 401 if unauthenticated; 404 if the blob is not found.

Delete a Single Tag

DELETE /api/v1/buckets/{bucket_name}/blobs/{key}/tags/{tag_key}

Deletes a single tag by key. Idempotent: deleting a tag that doesn't exist still returns 204.

Example:

curl -s -X DELETE \
  -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/my-bucket/blobs/hello.txt/tags/env"

Response: 204 No Content

S3 Access Keys

S3 access keys let you authenticate with Oyster's S3-compatible API using standard AWS Signature Version 4. Each access key consists of an access key ID (20 characters, prefixed with OYAK) and a secret access key (40 hex characters).

You can have up to 3 active access keys per account.

Managing Access Keys

Access keys are provisioned through the Admin API. An app operator uses admin-key authentication to create, list, and revoke keys for accounts they manage.

Key Format

Each account can have at most 3 active access keys. Revoked keys do not count toward this limit.

Wallet

Each Oyster account has an associated Sui wallet address, derived from the account's identity by the Pearl custodial wallet service. This wallet is used for onchain operations when blobs are stored on Walrus.

Get Wallet Address

GET /api/v1/account/wallet

Returns the Sui wallet address associated with your account.

Example:

curl -s -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/account/wallet" | jq

Response (200 OK):

{
  "address": "0x1a2b3c4d5e6f7890abcdef1234567890abcdef1234567890abcdef1234567890"
}

Errors:

Infrastructure

These endpoints are used for health monitoring and observability. They do not require authentication and are not under the /api/v1/ prefix.

Health Check (Liveness)

GET /health

Always returns 200 OK. Use this as a Kubernetes liveness probe or a simple "is the server running?" check.

Example:

curl -s "$OYSTER_URL/health" | jq

Response (200 OK):

{
  "status": "ok"
}

Readiness Check

GET /ready

Returns 200 OK when all dependencies are healthy, or 503 Service Unavailable if any dependency is unreachable. Use this as a Kubernetes readiness probe to gate traffic until the server is fully operational.

Example:

curl -s "$OYSTER_URL/ready" | jq

Response (200 OK, all healthy):

{
  "ready": true
}

Response (503 Service Unavailable, degraded):

{
  "ready": false,
  "database": "unreachable",
  "pearl": "unreachable"
}

The database and pearl fields are only present when the corresponding service is unhealthy.

Prometheus Metrics

GET /metrics

Returns metrics in Prometheus text exposition format. Scrape this endpoint with Prometheus, Grafana Agent, or any compatible collector.

Example:

curl -s "$OYSTER_URL/metrics"

Response (200 OK):

# HELP oyster_active_accounts Number of active accounts
# TYPE oyster_active_accounts gauge
oyster_active_accounts 42

# HELP oyster_active_blobs Number of active blobs
# TYPE oyster_active_blobs gauge
oyster_active_blobs 1337

# HELP oyster_blob_store_operations_total Blob store operations
# TYPE oyster_blob_store_operations_total counter
oyster_blob_store_operations_total{operation="store",result="success"} 500
oyster_blob_store_operations_total{operation="read",result="success"} 2000
oyster_blob_store_operations_total{operation="delete",result="success"} 100

Available metrics:

OpenAPI Documentation

GET /api/docs

Serves an interactive API documentation UI (powered by Scalar). Open this URL in your browser to explore and test all endpoints interactively.

open "$OYSTER_URL/api/docs"

S3 API Reference

Oyster provides an S3-compatible API that works with the AWS CLI, boto3, the AWS SDK for JavaScript, and any other S3-compatible client. It uses standard AWS Signature Version 4 (SigV4) authentication.

How It Works

The S3 API runs on the same HTTP port as the JSON API. Any request that doesn't match /api/v1/, /health, /ready, /metrics, or /api/docs is routed to the S3-compatible handler.

This means you point your S3 client at the same $OYSTER_URL, with no separate port or endpoint needed.

Supported Operations

See Limitations for what's not yet supported compared to full AWS S3.

Authentication

S3 requests are authenticated using S3 access keys (created through the JSON API). The AWS SDK and CLI handle SigV4 signing automatically, so you just provide your access key ID and secret.

Both APIs share the same database, so buckets and objects created through S3 are visible in the JSON API and vice versa.

Getting Started

Head to S3 Setup to configure your AWS CLI or SDK.

S3 Setup

Configure the AWS CLI and SDKs to work with Oyster's S3-compatible API using the steps below.

Prerequisites

• An Oyster API key (Bearer token); see Getting Started
• AWS CLI v2 (for CLI usage)

Step 1: Create S3 Access Keys

Access keys are created through the Admin API using admin-key authentication:

curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_KEY" \
  "$OYSTER_URL/api/v1/accounts/$ACCOUNT_ID/access-keys" | jq

Save the access_key_id and secret_access_key from the response, because the secret is only shown once.

See S3 Access Keys for more on key format and limits.

Step 2: Configure the AWS CLI

Set up a named profile pointing at your Oyster instance:

aws configure set aws_access_key_id "OYAK1234567890ABCDEF" --profile oyster
aws configure set aws_secret_access_key "abcdef1234567890abcdef1234567890abcdef12" --profile oyster
aws configure set region "us-east-1" --profile oyster
aws configure set endpoint_url "$OYSTER_URL" --profile oyster

Region: Oyster ignores the region value, but AWS SigV4 requires one. Use any valid region string, for example us-east-1.

Step 3: Verify Connectivity

aws --profile oyster s3api list-buckets

You should see a JSON response with your buckets (or an empty list if you haven't created any yet):

{
    "Buckets": []
}

Quick Test

Try a full round-trip:

# Create a bucket
aws --profile oyster s3api create-bucket --bucket test-bucket

# Upload a file
echo "Hello from S3!" > /tmp/hello.txt
aws --profile oyster s3api put-object \
  --bucket test-bucket --key hello.txt --body /tmp/hello.txt

# Download it back
aws --profile oyster s3api get-object \
  --bucket test-bucket --key hello.txt /tmp/downloaded.txt
cat /tmp/downloaded.txt

SDK Configuration

When using AWS SDKs programmatically, you need to set path-style addressing and a custom endpoint. Here are examples for common SDKs:

Python (boto3)

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:3000",
    aws_access_key_id="OYAK1234567890ABCDEF",
    aws_secret_access_key="abcdef1234567890...",
    region_name="us-east-1",
)

# Path-style is the default for custom endpoints in boto3
s3.list_buckets()

JavaScript / TypeScript (AWS SDK v3)

import { S3Client, ListBucketsCommand } from "@aws-sdk/client-s3";

const client = new S3Client({
  endpoint: "http://localhost:3000",
  region: "us-east-1",
  credentials: {
    accessKeyId: "OYAK1234567890ABCDEF",
    secretAccessKey: "abcdef1234567890...",
  },
  forcePathStyle: true,
});

const response = await client.send(new ListBucketsCommand({}));

Rust (aws-sdk-s3)

#![allow(unused)]
fn main() {
use aws_sdk_s3::config::{Credentials, Region};

let creds = Credentials::new(
    "OYAK1234567890ABCDEF",
    "abcdef1234567890...",
    None, None, "oyster",
);

let config = aws_sdk_s3::Config::builder()
    .behavior_version_latest()
    .region(Region::new("us-east-1"))
    .endpoint_url("http://localhost:3000")
    .credentials_provider(creds)
    .force_path_style(true)
    .build();

let client = aws_sdk_s3::Client::from_conf(config);
}

Important: Always set force_path_style: true (or equivalent). Oyster uses path-style URLs (endpoint/bucket/key), not virtual-hosted-style (bucket.endpoint/key).

Bucket Operations

All bucket operations require S3 authentication (SigV4). See S3 Setup for configuration.

CreateBucket

Creates a new bucket.

aws --profile oyster s3api create-bucket --bucket my-bucket

Response:

{
    "Location": "/my-bucket"
}

Bucket names follow the same naming rules as the JSON API: 3–63 characters, lowercase letters/digits/hyphens only, no consecutive hyphens, no IP address format, and no reserved names (health, ready, metrics, api).

Errors:

HeadBucket

Checks if a bucket exists and you have access to it. Returns no body — only HTTP status.

aws --profile oyster s3api head-bucket --bucket my-bucket

Returns exit code 0 on success (HTTP 200). If the bucket doesn't exist or isn't owned by your account, the AWS CLI prints an error (HTTP 404).

Errors:

ListBuckets

Lists all buckets in your account.

aws --profile oyster s3api list-buckets

Response:

{
    "Buckets": [
        {
            "Name": "my-bucket",
            "CreationDate": "2025-01-15T10:30:00Z"
        },
        {
            "Name": "logs-2025",
            "CreationDate": "2025-01-16T08:00:00Z"
        }
    ]
}

Returns up to 1000 buckets. No pagination is supported for this operation.

DeleteBucket

Deletes a bucket. The bucket must be empty first.

aws --profile oyster s3api delete-bucket --bucket my-bucket

Returns no output on success (HTTP 204).

Errors:

Object Operations

All S3 operations require authentication (SigV4), including reads. See S3 Setup for configuration.

Authenticated reads: Unlike the JSON API, where blob reads are public and unauthenticated, all S3 reads require authentication.

PutObject

Uploads an object to a bucket. If an object with the same key already exists, it is replaced.

aws --profile oyster s3api put-object \
  --bucket my-bucket \
  --key hello.txt \
  --body hello.txt

Response:

{
    "ETag": "\"9a0364b9e99bb480dd25e1f0284c8555\""
}

The ETag is the MD5 digest of the uploaded content.

Setting Content-Type

aws --profile oyster s3api put-object \
  --bucket my-bucket \
  --key image.png \
  --body photo.png \
  --content-type "image/png"

If --content-type is omitted, it defaults to application/octet-stream.

Setting Tags on Upload

Attach tags at upload time with --tagging, a URL-encoded query string of key=value pairs:

aws --profile oyster s3api put-object \
  --bucket my-bucket \
  --key hello.txt \
  --body hello.txt \
  --tagging "env=prod&team=platform"

Tags set this way share the same store as the JSON API and the Object Tagging operations below, and are subject to the same tag rules.

Key Behavior

• Overwrite: Uploading to an existing key replaces the object
• Expiration: Objects share the owning account's StoragePool lifetime; the background extension service renews the pool before it expires (see Blob Lifecycle)
• Content-addressed: Identical content produces the same blob ID internally, enabling deduplication

Conditional Headers

PutObject supports If-Match and If-None-Match headers for safe writes:

• If-None-Match: *: upload only if the key doesn't already exist (create-only semantics). Returns 412 PreconditionFailed if the key exists.
• If-Match: "<etag>": overwrite only if the current object's ETag matches. Returns 412 PreconditionFailed on mismatch.

# Create-only: fail if the key already exists
aws --profile oyster s3api put-object \
  --bucket my-bucket \
  --key hello.txt \
  --body hello.txt \
  --if-none-match "*"

Errors:

GetObject

Downloads an object's contents.

aws --profile oyster s3api get-object \
  --bucket my-bucket \
  --key hello.txt \
  downloaded.txt

Response metadata:

{
    "ContentLength": 14,
    "ContentType": "text/plain",
    "ETag": "\"9a0364b9e99bb480dd25e1f0284c8555\"",
    "LastModified": "2025-01-15T10:31:00Z"
}

The file contents are written to the output path (downloaded.txt in this example).

Conditional Headers

GetObject supports If-Match and If-None-Match for cache validation:

• If-Match: "<etag>": return the object only if its ETag matches. Returns 412 PreconditionFailed on mismatch.
• If-None-Match: "<etag>": return the object only if its ETag differs. Returns 304 NotModified if the ETag matches (useful for cache validation).

# Only download if the object has changed
aws --profile oyster s3api get-object \
  --bucket my-bucket \
  --key hello.txt \
  --if-none-match '"9a0364b9e99bb480dd25e1f0284c8555"' \
  downloaded.txt

Errors:

HeadObject

Retrieves object metadata without downloading the contents. Useful for checking if an object exists or reading its size and content type.

aws --profile oyster s3api head-object \
  --bucket my-bucket \
  --key hello.txt

Response:

{
    "ContentLength": 14,
    "ContentType": "text/plain",
    "ETag": "\"9a0364b9e99bb480dd25e1f0284c8555\"",
    "LastModified": "2025-01-15T10:31:00Z"
}

HeadObject supports the same If-Match and If-None-Match conditional headers as GetObject. Returns 412 PreconditionFailed or 304 NotModified as appropriate.

Errors:

DeleteObject

Deletes an object from a bucket.

aws --profile oyster s3api delete-object \
  --bucket my-bucket \
  --key hello.txt

Returns no output on success.

This operation is idempotent: deleting a key that doesn't exist still returns success, matching standard S3 behavior.

Deletion is reference-counted: the underlying blob data is only removed from storage when no other keys reference the same content.

Conditional Headers

DeleteObject supports If-Match for safe deletion:

• If-Match: "<etag>": delete only if the object's ETag matches. Returns 412 PreconditionFailed on mismatch.

Errors:

ListObjectsV2

Lists objects in a bucket with optional filtering and pagination.

Basic Listing

aws --profile oyster s3api list-objects-v2 --bucket my-bucket

Response:

{
    "Name": "my-bucket",
    "Contents": [
        {
            "Key": "hello.txt",
            "Size": 14,
            "ETag": "\"9a0364b9e99bb480...\"",
            "LastModified": "2025-01-15T10:31:00Z",
            "StorageClass": "STANDARD"
        },
        {
            "Key": "images/photo.png",
            "Size": 204800,
            "ETag": "\"d41d8cd98f00b204...\"",
            "LastModified": "2025-01-15T11:00:00Z",
            "StorageClass": "STANDARD"
        }
    ],
    "KeyCount": 2,
    "MaxKeys": 1000,
    "IsTruncated": false
}

Filtering by Prefix

List only objects under a specific "folder":

aws --profile oyster s3api list-objects-v2 \
  --bucket my-bucket \
  --prefix "images/"

Simulating Folders with Delimiter

Use --delimiter "/" to group objects into virtual folders:

aws --profile oyster s3api list-objects-v2 \
  --bucket my-bucket \
  --delimiter "/"

Response:

{
    "Name": "my-bucket",
    "Contents": [
        {
            "Key": "hello.txt",
            "Size": 14,
            "ETag": "\"9a0364b9e99bb480...\"",
            "LastModified": "2025-01-15T10:31:00Z",
            "StorageClass": "STANDARD"
        }
    ],
    "CommonPrefixes": [
        {
            "Prefix": "images/"
        }
    ],
    "KeyCount": 2,
    "MaxKeys": 1000,
    "Delimiter": "/",
    "IsTruncated": false
}

Objects directly at the root level appear in Contents, while "folders" (key prefixes before the delimiter) appear in CommonPrefixes.

Combining Prefix and Delimiter

List the contents of a specific "folder":

aws --profile oyster s3api list-objects-v2 \
  --bucket my-bucket \
  --prefix "images/" \
  --delimiter "/"

Pagination

Limit results and paginate through large listings:

# First page
aws --profile oyster s3api list-objects-v2 \
  --bucket my-bucket \
  --max-keys 10

# Next page (using NextContinuationToken from previous response)
aws --profile oyster s3api list-objects-v2 \
  --bucket my-bucket \
  --max-keys 10 \
  --starting-token "last-key-from-previous-page"

Supported Parameters

Errors:

Object Tagging

Oyster implements the three S3 object-tagging operations. Tags are stored in Oyster's database in the same blob_tags table used by the JSON API tag endpoints; a tag set through S3 is visible through the JSON API and vice versa. The same tag rules apply (max 10 tags; key ≤ 128 B; value ≤ 256 B; set ≤ 2048 B; restricted charset).

PutObjectTagging

Replaces the object's entire tag set.

aws --profile oyster s3api put-object-tagging \
  --bucket my-bucket \
  --key hello.txt \
  --tagging 'TagSet=[{Key=env,Value=prod},{Key=team,Value=platform}]'

GetObjectTagging

Returns the object's current tags.

aws --profile oyster s3api get-object-tagging \
  --bucket my-bucket \
  --key hello.txt

Response:

{
    "TagSet": [
        { "Key": "env", "Value": "prod" },
        { "Key": "team", "Value": "platform" }
    ]
}

DeleteObjectTagging

Removes all tags from the object.

aws --profile oyster s3api delete-object-tagging \
  --bucket my-bucket \
  --key hello.txt

Errors:

Limitations

Oyster implements the most commonly used S3 operations. The sections below document what's different from a full AWS S3 deployment.

Supported vs. Not Supported

Behavioral Differences

Object Expiration

All objects in a bucket share the owning account's StoragePool lifetime, with no per-object expiration to set. Oyster runs a background extension service that renews the pool before it expires, so objects persist indefinitely as long as the service is running and the account's wallet stays funded. See Blob Lifecycle for the model.

Bucket Naming

Oyster's bucket naming rules are slightly stricter than AWS S3:

ListBuckets Limit

ListBuckets returns a maximum of 1000 buckets with no pagination support.

Path-Style URLs Only

Oyster only supports path-style S3 URLs:

http://endpoint/bucket-name/key

Virtual-hosted-style URLs (bucket-name.endpoint/key) are not supported. Always set force_path_style: true in your SDK configuration.

No Region Semantics

Oyster ignores the region in S3 requests. All data is stored in the same location. You still need to specify a region for SigV4 signing to work, so use any valid region string, for example us-east-1.

ETag Format

ETags are always the MD5 digest of the object content, even for large objects. There is no multipart ETag format, because multipart upload is not supported.

Conditional Request Headers

If-Match and If-None-Match headers are supported on PutObject, GetObject, HeadObject, and DeleteObject (If-Match only). These enable cache validation (If-None-Match returns 304 on GET/HEAD) and safe concurrent writes (If-Match for optimistic locking, If-None-Match: * for create-only semantics). Time-based conditionals (If-Modified-Since, If-Unmodified-Since) are not supported.

Guides

Practical guides for working with Oyster beyond the API reference.

• oyster-cli Quick Start: command-line tool for managing buckets, blobs, and keys
• AWS SDK Examples: full workflows in Python (boto3) and JavaScript (AWS SDK v3)
• Content Addressing: how blob IDs, deduplication, and reference-counted deletion work
• Blob Lifecycle: expiration, automatic renewal, and the extension service
• Pearl Master-Seed Rotation: operator runbook for moving accounts onto a new master seed (oysterd keys)

oyster-cli Quick Start

oyster-cli is a command-line tool for interacting with Oyster. It wraps the JSON API and handles authentication, content-type detection, and pagination for you.

Configuration

The CLI looks for a config file in this order:

1. Path specified with --config
2. ./client.yaml (current directory)
3. $XDG_CONFIG_HOME/oyster/client.yaml
4. $HOME/.config/oyster/client.yaml

Contexts

client.yaml holds a map of named contexts, each pointing at a different Oyster deployment. The top-level active_context selects which context is used by default.

Example client.yaml:

active_context: testnet
contexts:
  testnet:
    url: "https://oyster.testnet.example/api/v1"
    api_key: "your-api-key-here"
    apps:
      my-app-1:
        admin_key: "<64-char hex admin key>"
      my-app-2:
        admin_key: "<64-char hex admin key>"
  devnet:
    url: "http://localhost:3000/api/v1"
    api_key: "dev-key"

Important: The URL must include the /api/v1 path. The CLI appends endpoint paths (such as /buckets) directly to this URL.

Precedence for the active-context name (highest first):

1. --context <name> flag
2. OYSTER_CONTEXT environment variable
3. active_context field in client.yaml

If none of the three is set and the file has exactly one context, that context is used automatically. Ad-hoc --url ... --api-key ... invocations without any context still work for one-time commands.

You can also override individual fields with flags:

oyster --url http://localhost:3000/api/v1 --api-key "your-key" list-buckets

Global flags

Bucket management

Create a bucket

oyster create-bucket my-bucket

List buckets

oyster list-buckets

Limit results:

oyster list-buckets --limit 10

Delete a bucket

oyster delete-bucket my-bucket

The bucket must be empty. Delete all blobs first, or the server returns an error.

Storing and reading blobs

Upload a file

oyster store photo.png --bucket my-bucket

The key defaults to the filename (photo.png). Override it with --key:

oyster store photo.png --bucket my-bucket --key images/vacation/photo.png

Set a specific content type:

oyster store data.bin --bucket my-bucket --content-type application/x-custom

If --content-type is omitted, the CLI auto-detects it from the file extension (see Content-Type Detection below).

Attach tags at upload time with --tag key=value (repeatable):

oyster store photo.png --bucket my-bucket --tag env=prod --tag team=platform

Tags are replaced on every upload to a key. See Blob tags for the limits and for managing tags after upload.

Download a file

oyster read hello.txt --bucket my-bucket

This prints the blob contents to stdout. Save to a file with -o:

oyster read hello.txt --bucket my-bucket -o downloaded.txt

Reading blobs does not require an API key. Reads are public.

List blobs

oyster list-blobs --bucket my-bucket

Output (human-readable):

KEY            CONTENT_TYPE    SIZE    CREATED
hello.txt      text/plain      14      2025-01-15T10:31:00Z
images/cat.png image/png       204800  2025-01-15T11:00:00Z

Delete a blob

oyster delete hello.txt --bucket my-bucket

Blob tags

The oyster tags command group manages the key=value tags on a blob. Tags are stored in Oyster's database and shared with the S3 object-tagging operations. Limits: max 10 tags, key ≤ 128 bytes, value ≤ 256 bytes, set ≤ 2048 bytes, and a restricted charset (ASCII alphanumerics plus space and + - = . _ : / @).

List tags

oyster tags list --bucket my-bucket --key hello.txt

Set a single tag

Upserts one tag (key=value):

oyster tags set --bucket my-bucket --key hello.txt env=prod

Remove a single tag

oyster tags rm --bucket my-bucket --key hello.txt env

Clear all tags

oyster tags clear --bucket my-bucket --key hello.txt

Replace vs. merge

replace sets the entire tag set, dropping any tags not listed. merge upserts the supplied tags, leaving other existing tags untouched. Both take repeatable --tag key=value flags:

# Full replace — the blob ends up with exactly these two tags
oyster tags replace --bucket my-bucket --key hello.txt \
  --tag env=prod --tag team=platform

# Merge — adds/updates these tags, keeps the rest
oyster tags merge --bucket my-bucket --key hello.txt --tag team=storage

API key and access key management

API keys and S3 access keys are managed by operators through the Admin API, not through the CLI. See the Admin API docs for details on creating, listing, and revoking keys.

Other commands

View wallet address

oyster wallet

View resolved configuration

oyster info

Shows which config file is loaded, the server URL, and the API key prefix.

App admin-key management

Apps are first-class principals that authenticate admin app-management calls (creating accounts, issuing API keys and S3 access keys) independently of end-user API keys. They need a way to store the per-app admin key without leaking it through shell history. The CLI persists admin keys under contexts.<ctx>.apps.<app_name>.admin_key.

Import an admin key

oyster app import my-app

Prompts for the admin key without echoing it (when stdin is a tty), then writes it to the active context's apps.my-app entry. If stdin is a pipe, the key is read as a line instead — useful for scripts. Requires that client.yaml already exists; the CLI does not auto-create it.

Rotation

Admin keys do not expire. Rotation is operator-driven with AWS-style two-key overlap:

# operator
oysterd app issue-admin-key <APP_ID>
# stdout: <new admin_key>   (the raw bearer — the only machine-readable output)
# stderr: a `tracing` log line with the key id + prefix (needed later to
#         revoke). It's an `info`-level log, so it appears with the default
#         log filter but is suppressed if RUST_LOG raises the threshold above
#         `info`. Capture the key id from `oysterd app list-admin-keys`.

# user — replace the local entry with the new key
oyster app import my-app

# operator — after confirming nothing still uses the old key
oysterd app revoke-admin-key <OLD_KEY_ID>

oysterd app list-admin-keys <APP_ID> shows all keys (active and revoked), so an operator can confirm what is live. Multiple admin keys per app are supported with no cap.

Webhook management

oyster app webhook drives the self-service webhook endpoints (Set Webhook URL) using the active context's admin key. When the context defines more than one app, pass --app <name> to choose which one.

# Show the current webhook URL and public key
oyster app webhook show

# Register or rotate the webhook URL (each call mints a fresh Ed25519 keypair;
# the printed public key is needed to verify subsequent deliveries)
oyster app webhook set https://example.com/oyster/webhook

# Clear the webhook URL and discard the keypair
oyster app webhook clear

See Webhooks for the delivery signature format.

Account management

Once a context has at least one app with an admin_key, the oyster app account subcommand tree manages the accounts that app owns. Use it to mint accounts, rotate which account the CLI's api_key points at, and inspect the API keys an account has issued.

--app <name> selector

Every oyster app account subcommand resolves which app to act through using crates/oyster-cli/src/config.rs::resolve_admin:

• If the active context defines exactly one app, it is auto-selected.
• If the active context defines multiple apps, you must pass --app <name> (or the command errors and lists the known apps).
• If the active context defines zero apps, the command errors. Import an admin key first with oyster app import.

oyster app account list                  # active context has 1 app
oyster --app my-app account list         # multiple apps; pick one

Subcommands

The following subcommands are available under oyster app account.

list

Tabular view of the accounts owned by the selected app. Each row is id, name, created_at, active_api_key_count.

oyster app account list

create [--name NAME] [--note NOTE] [--activate]

Mints a fresh account plus an initial API key for it.

• --name NAME: human-readable label stored on the account.
• --note NOTE: note attached to the issued API key (defaults to "api" server-side).
• --activate: atomically saves the new bearer to context.api_key in client.yaml. Without this flag, the bearer is printed once and you can wire it up yourself.

oyster app account create --name alice --activate

use <id-or-name> [--revoke <key_id> | --revoke-oldest]

Pivots the active context's api_key onto a different account. It mints a fresh API key on the target account (note oyster-cli: activate <id-or-name>), atomically writes it to context.api_key, and saves client.yaml.

There is a server-side cap of 3 active API keys per account (MAX_API_KEYS_PER_ACCOUNT in crates/oyster/src/routes/admin.rs). If use would exceed that cap, the server returns 409 Conflict with "limit" in the message and the CLI behavior depends on whether stdout is a TTY:

• TTY: the CLI uses inquire::Select (inline, never alt-screen) to show the account's existing keys and ask which to revoke, then retries the mint.
• Non-TTY (CI, scripts, --json): the call fails unless you pre-select the key to revoke. Pass either:
  • --revoke <KEY_ID> to revoke a specific key, or
  • --revoke-oldest to revoke the oldest active key (sorted by created_at).

The two flags are mutually exclusive.

oyster app account use alice
oyster app account use alice --revoke-oldest
oyster app account use alice --revoke 0123abcd...

select

TTY-only inquire picker over the app's accounts. Dispatches to use with the chosen account. Errors in non-TTY contexts. Scripts should use use <id-or-name> directly.

oyster app account select

keys <id-or-name>

Lists API key metadata for the named account: id, note, created_at, and revoked_at. Bearer secrets are never returned.

oyster app account keys alice

JSON output

Add --json to any command for machine-readable output:

oyster --json list-blobs --bucket my-bucket

{
  "data": [
    {
      "key": "hello.txt",
      "blob_id": "2cf24dba5fb0a30e...",
      "content_type": "text/plain",
      "size": 14,
      "created_at": "2025-01-15T10:31:00Z"
    }
  ],
  "next_cursor": null
}

Content-type detection

When uploading without --content-type, the CLI guesses the MIME type from the file extension:

AWS SDK Examples

These examples show complete workflows using AWS SDKs with Oyster's S3-compatible API. For initial SDK setup, see S3 setup.

Python (boto3)

Setup

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:3000",
    aws_access_key_id="OYAK1234567890ABCDEF",
    aws_secret_access_key="abcdef1234567890abcdef1234567890abcdef12",
    region_name="us-east-1",
)

Create a bucket

s3.create_bucket(Bucket="my-bucket")

Upload a file

# From a file on disk
s3.upload_file("photo.png", "my-bucket", "images/photo.png")

# From a string
s3.put_object(
    Bucket="my-bucket",
    Key="hello.txt",
    Body=b"Hello, Oyster!",
    ContentType="text/plain",
)

Download a file

# To a file on disk
s3.download_file("my-bucket", "hello.txt", "downloaded.txt")

# To memory
response = s3.get_object(Bucket="my-bucket", Key="hello.txt")
content = response["Body"].read()
print(content.decode())  # "Hello, Oyster!"

List objects

# List all objects
response = s3.list_objects_v2(Bucket="my-bucket")
for obj in response.get("Contents", []):
    print(f"{obj['Key']}  {obj['Size']} bytes")

# List with prefix (simulate folder listing)
response = s3.list_objects_v2(
    Bucket="my-bucket",
    Prefix="images/",
    Delimiter="/",
)

# Files directly in "images/"
for obj in response.get("Contents", []):
    print(f"  File: {obj['Key']}")

# "Subfolders" in "images/"
for prefix in response.get("CommonPrefixes", []):
    print(f"  Folder: {prefix['Prefix']}")

Paginate through large listings

paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", MaxKeys=100):
    for obj in page.get("Contents", []):
        print(obj["Key"])

Delete an object

s3.delete_object(Bucket="my-bucket", Key="hello.txt")

Delete a bucket

s3.delete_bucket(Bucket="my-bucket")

Check if an object exists

try:
    s3.head_object(Bucket="my-bucket", Key="hello.txt")
    print("Object exists")
except s3.exceptions.ClientError as e:
    if e.response["Error"]["Code"] == "404":
        print("Object not found")
    else:
        raise

Conditional requests

Use If-Match and If-None-Match headers for safe writes and cache validation:

# Create-only: fail if the key already exists
try:
    s3.put_object(
        Bucket="my-bucket",
        Key="config.json",
        Body=b'{"version": 1}',
        IfNoneMatch="*",
    )
except s3.exceptions.ClientError as e:
    if e.response["Error"]["Code"] == "PreconditionFailed":
        print("Key already exists — not overwritten")
    else:
        raise

# Safe overwrite: only update if the ETag matches
response = s3.head_object(Bucket="my-bucket", Key="config.json")
current_etag = response["ETag"]

s3.put_object(
    Bucket="my-bucket",
    Key="config.json",
    Body=b'{"version": 2}',
    IfMatch=current_etag,
)

# Cache validation: skip download if unchanged
try:
    s3.get_object(
        Bucket="my-bucket",
        Key="config.json",
        IfNoneMatch=current_etag,
    )
except s3.exceptions.ClientError as e:
    if e.response["Error"]["Code"] == "304":
        print("Not modified — use cached copy")
    else:
        raise

Full workflow

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:3000",
    aws_access_key_id="OYAK1234567890ABCDEF",
    aws_secret_access_key="abcdef1234567890...",
    region_name="us-east-1",
)

# Create bucket
s3.create_bucket(Bucket="demo")

# Upload
s3.put_object(Bucket="demo", Key="doc.txt", Body=b"Hello!")
s3.put_object(Bucket="demo", Key="images/a.png", Body=b"\x89PNG...")
s3.put_object(Bucket="demo", Key="images/b.png", Body=b"\x89PNG...")

# List with folder simulation
resp = s3.list_objects_v2(Bucket="demo", Delimiter="/")
print("Root files:", [o["Key"] for o in resp.get("Contents", [])])
print("Folders:", [p["Prefix"] for p in resp.get("CommonPrefixes", [])])
# Root files: ['doc.txt']
# Folders: ['images/']

# Download
obj = s3.get_object(Bucket="demo", Key="doc.txt")
print(obj["Body"].read().decode())  # "Hello!"

# Clean up
s3.delete_object(Bucket="demo", Key="doc.txt")
s3.delete_object(Bucket="demo", Key="images/a.png")
s3.delete_object(Bucket="demo", Key="images/b.png")
s3.delete_bucket(Bucket="demo")

JavaScript / TypeScript (AWS SDK v3)

Setup

import {
  S3Client,
  CreateBucketCommand,
  PutObjectCommand,
  GetObjectCommand,
  HeadObjectCommand,
  ListObjectsV2Command,
  DeleteObjectCommand,
  DeleteBucketCommand,
} from "@aws-sdk/client-s3";

const client = new S3Client({
  endpoint: "http://localhost:3000",
  region: "us-east-1",
  credentials: {
    accessKeyId: "OYAK1234567890ABCDEF",
    secretAccessKey: "abcdef1234567890abcdef1234567890abcdef12",
  },
  forcePathStyle: true,
});

Create a bucket

await client.send(new CreateBucketCommand({ Bucket: "my-bucket" }));

Upload an object

await client.send(
  new PutObjectCommand({
    Bucket: "my-bucket",
    Key: "hello.txt",
    Body: "Hello, Oyster!",
    ContentType: "text/plain",
  })
);

Upload a file from disk (Node.js)

import { createReadStream } from "fs";

await client.send(
  new PutObjectCommand({
    Bucket: "my-bucket",
    Key: "images/photo.png",
    Body: createReadStream("photo.png"),
    ContentType: "image/png",
  })
);

Download an object

const response = await client.send(
  new GetObjectCommand({
    Bucket: "my-bucket",
    Key: "hello.txt",
  })
);

const body = await response.Body.transformToString();
console.log(body); // "Hello, Oyster!"

List objects

const response = await client.send(
  new ListObjectsV2Command({
    Bucket: "my-bucket",
    Prefix: "images/",
    Delimiter: "/",
  })
);

for (const obj of response.Contents ?? []) {
  console.log(`File: ${obj.Key} (${obj.Size} bytes)`);
}

for (const prefix of response.CommonPrefixes ?? []) {
  console.log(`Folder: ${prefix.Prefix}`);
}

Delete an object

await client.send(
  new DeleteObjectCommand({
    Bucket: "my-bucket",
    Key: "hello.txt",
  })
);

Check if an object exists

try {
  await client.send(
    new HeadObjectCommand({
      Bucket: "my-bucket",
      Key: "hello.txt",
    })
  );
  console.log("Object exists");
} catch (err) {
  if (err.name === "NotFound") {
    console.log("Object not found");
  } else {
    throw err;
  }
}

Conditional requests

Use IfMatch and IfNoneMatch parameters for safe writes and cache validation:

// Create-only: fail if the key already exists
try {
  await client.send(
    new PutObjectCommand({
      Bucket: "my-bucket",
      Key: "config.json",
      Body: JSON.stringify({ version: 1 }),
      IfNoneMatch: "*",
    })
  );
} catch (err) {
  if (err.name === "PreconditionFailed") {
    console.log("Key already exists — not overwritten");
  } else {
    throw err;
  }
}

// Safe overwrite: only update if the ETag matches
const head = await client.send(
  new HeadObjectCommand({ Bucket: "my-bucket", Key: "config.json" })
);

await client.send(
  new PutObjectCommand({
    Bucket: "my-bucket",
    Key: "config.json",
    Body: JSON.stringify({ version: 2 }),
    IfMatch: head.ETag,
  })
);

Full workflow

import { S3Client, CreateBucketCommand, PutObjectCommand,
  GetObjectCommand, ListObjectsV2Command, DeleteObjectCommand,
  DeleteBucketCommand } from "@aws-sdk/client-s3";

const client = new S3Client({
  endpoint: "http://localhost:3000",
  region: "us-east-1",
  credentials: {
    accessKeyId: "OYAK1234567890ABCDEF",
    secretAccessKey: "abcdef1234567890...",
  },
  forcePathStyle: true,
});

// Create bucket
await client.send(new CreateBucketCommand({ Bucket: "demo" }));

// Upload objects
await client.send(new PutObjectCommand({
  Bucket: "demo", Key: "doc.txt", Body: "Hello!",
}));
await client.send(new PutObjectCommand({
  Bucket: "demo", Key: "images/a.png", Body: Buffer.from([0x89, 0x50]),
}));

// List with folder simulation
const list = await client.send(new ListObjectsV2Command({
  Bucket: "demo", Delimiter: "/",
}));
console.log("Files:", list.Contents?.map(o => o.Key));
console.log("Folders:", list.CommonPrefixes?.map(p => p.Prefix));

// Download
const obj = await client.send(new GetObjectCommand({
  Bucket: "demo", Key: "doc.txt",
}));
console.log(await obj.Body.transformToString()); // "Hello!"

// Clean up
await client.send(new DeleteObjectCommand({ Bucket: "demo", Key: "doc.txt" }));
await client.send(new DeleteObjectCommand({ Bucket: "demo", Key: "images/a.png" }));
await client.send(new DeleteBucketCommand({ Bucket: "demo" }));

Content Addressing

Oyster uses content addressing to identify blobs. A blob's identity is derived from its contents, not from where it is stored. This enables deduplication, integrity verification, and content-based retrieval.

What is a blob ID?

When you upload data to Oyster, the server computes a BLAKE2s-256 hash of the raw bytes. This produces a 64-character hex string called the blob ID:

2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824

This blob ID is returned in the upload response and stored alongside the object metadata. The same data always produces the same blob ID, regardless of which bucket or key it's stored under.

Deduplication

If you upload the same file to two different keys or two different buckets, Oyster recognizes that the content is identical and stores the data only once. For example:

# Upload the same file to two different keys
curl -s -X PUT -H "Authorization: Bearer $API_KEY" \
  --data-binary @photo.png \
  "$OYSTER_URL/api/v1/buckets/bucket-a/blobs/photo.png" | jq .blob_id

curl -s -X PUT -H "Authorization: Bearer $API_KEY" \
  --data-binary @photo.png \
  "$OYSTER_URL/api/v1/buckets/bucket-b/blobs/copy.png" | jq .blob_id

Both uploads return the same blob_id because the content is identical. The underlying storage holds only one copy.

Reference counting

Each key-to-blob mapping is a reference. Oyster tracks how many references point to each blob ID:

bucket-a/photo.png  →  blob_id: abc123...  (ref count: 2)
bucket-b/copy.png   →  blob_id: abc123...

When you delete a key, Oyster removes that reference. The physical blob data is only deleted from storage when all references are removed:

# Delete one reference — blob data still exists
curl -s -X DELETE -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/bucket-a/blobs/photo.png"
# ref count: 1 — data preserved

# Delete the last reference — blob data is removed
curl -s -X DELETE -H "Authorization: Bearer $API_KEY" \
  "$OYSTER_URL/api/v1/buckets/bucket-b/blobs/copy.png"
# ref count: 0 — data deleted from storage

This means deleting a key in one bucket never affects the same content stored under a different key or bucket.

Reading by blob ID

You can retrieve any blob by its content-addressed ID, without knowing which bucket or key it belongs to:

curl -s "$OYSTER_URL/api/v1/blobs/by-blob-id/2cf24dba5fb0a30e..."

This is a public endpoint with no authentication required. It always returns the content with Content-Type: application/octet-stream.

This is useful when you have stored a blob ID externally (for example, in a database or onchain) and want to retrieve the data directly.

Practical implications

• Storage efficiency: identical files across buckets cost no extra storage.
• Safe deletion: deleting a key never destroys data that other keys depend on.
• Integrity: the blob ID serves as a checksum; if the data is corrupted, the hash does not match.
• Immutable content: a given blob ID always maps to the same data. Overwriting a key creates a new blob with a new blob ID.

Blob Lifecycle

Blobs progress through a well-defined lifecycle from upload to expiration. Oyster's automatic extension service keeps your data alive.

Upload and expiration

Walrus storage is epoch-scoped, not time-scoped. When you upload a blob, Oyster registers it under your account's StoragePool, a single onchain object whose end_epoch defines the lifetime of every blob it holds. The first upload from an account lazily creates the pool with POOL_INITIAL_EPOCHS_AHEAD of runway (default 5). Subsequent uploads share that same expiration.

The pool's end_epoch is surfaced on the account as pool_end_epoch. To inspect remaining runway, compare it against the network's current epoch. Blob responses do not carry a per-blob expiration. Every blob in the account shares the pool's lifetime.

Automatic extension

Oyster runs a background extension worker (oysterd extend) that keeps every account's StoragePool ahead of expiration. As long as the worker is running and the account's Pearl-derived wallet has WAL and SUI to spend, your blobs persist indefinitely.

What the worker guarantees

While the worker is running:

• Any account whose pool_end_epoch falls within current_epoch + POOL_EXTEND_LOOKAHEAD_EPOCHS is picked up and its pool extended by POOL_EXTEND_EPOCHS Walrus epochs, provided the Pearl-derived wallet has the WAL and SUI to cover it.
• Each pool is extended at most once per EXTENSION_CLAIM_COOLDOWN_SECS window, so retries and account.funding_required webhook deliveries are naturally rate- limited per account. The same cooldown doubles as webhook-spam suppression. A row that just emitted account.funding_required cannot re-emit until the cooldown expires.
• Latency between an account becoming eligible and its pool being extended is bounded above by EXTENSION_IDLE_SLEEP_SECS plus the RPC time of one extension.
• Before every extension the worker reads the pool's on-chain end_epoch and only submits the PTB if the chain still shows the pool inside the lookahead window. Walrus extensions are additive, so this is what keeps a retry idempotent: if an earlier attempt landed on-chain but Oyster's DB update was lost (write failure, timeout after execution, crash), the retry repairs pool_end_epoch from the chain instead of extending — and paying — a second time. The one remaining window is multi-replica: an attempt whose sign/submit outlives EXTENSION_CLAIM_COOLDOWN_SECS can overlap another replica's claim, so keep the cooldown well above the checkpoint-wait timeout.

Horizontal scaling

The worker is safe to run as multiple replicas against the same database. Each pool is claimed by exactly one replica per cycle, so two extenders never double-extend the same pool. The public Oyster Testnet runs 2 extender replicas behind a shared DB.

Metrics

The worker exposes Prometheus metrics on OYSTER_EXTENSION_METRICS_BIND_ADDR. The ones worth alerting on:

Configuration

Insufficient funds

If the Pearl-derived wallet for an account is short on WAL or SUI, the extend_storage_pool PTB fails with an insufficient-funds error. Oyster then takes these steps:

1. Logs the failure.
2. POSTs an account.funding_required webhook to the owning app's configured receiver URL (if any).
3. Leaves the cooldown TTL stamped on the row so the same account does not re-trigger the webhook for EXTENSION_CLAIM_COOLDOWN_SECS.

The next cycle re-claims the row once the cooldown expires. If the wallet is still underfunded, another webhook fires. See Webhooks for the full payload schema, retry policy, circuit-breaker behavior, and receiver examples.

Auto-grow

The pool's encoded-bytes reservation grows on demand. The first upload that does not fit in the current reservation submits a register_pooled_blobs PTB whose grow_by reserves the missing capacity in the same transaction.

When auto-grow retries

In a horizontally scaled Oyster deployment, two replicas can each compute grow_by against the same onchain snapshot, then race to submit their register PTBs. The replica that lands second sees its storage_pool::add_blob Move call abort with EInsufficientCapacity (code 6) because the first replica already consumed the reserve.

Oyster handles this by:

1. Refreshing the onchain StoragePoolInnerV1 through the Sui RPC's gRPC StateService.ListDynamicFields.
2. Reconciling the DB's pool_reserved_encoded_bytes and pool_used_encoded_bytes counters against onchain truth (the chain is authoritative, and a stale DB counter is overwritten).
3. Recomputing grow_by from the reconciled state.
4. Resubmitting the register PTB exactly once.

If the resubmit also aborts with EInsufficientCapacity, the error is surfaced to the caller. There is no second self-heal. A steady-state of cross-replica thrash is not a normal failure mode and the operator should investigate.

Interaction with the per-account cap

Auto-grow runs after the per-account max_unencoded_bytes cap is checked. The cap pre-check uses the same forward encoder (f = encoded_blob_length_for_n_shards) the upload path uses to project the post-upload encoded total, so a successful cap check already accounts for the would-be grow_by. Auto-grow can never push the account's encoded-bytes usage past the threshold the cap implies. (On small-shard / large-cap testbeds where the forward encoder would overflow i64, the pre-check falls back to a saturating comparison rather than a 500.)

By default the cap is an upper bound on storable unencoded bytes (each blob's fixed metadata overhead is paid per blob, so many small blobs hit the cap early). Setting a non-zero per-account avg_blob_size inflates the admission ceiling by the per-blob expansion factor f(s)/s, turning the cap into a lower bound: at least max_unencoded_bytes unencoded bytes are guaranteed storable when the account's blobs average ≥ s. New accounts default to a 10 MB avg_blob_size; avg_blob_size = 0 preserves the upper-bound behavior.

Why there is no per-process lock

Oyster scales horizontally: a Mutex inside one replica cannot coordinate with another replica's process, and chain-side state would still drift under concurrent uploads. The onchain StoragePoolInnerV1 is the source of truth. The one-shot reconcile-and-retry above absorbs the inevitable drift without serializing uploads.

Blob states

A blob's lifetime is bound to its account's StoragePool:

Upload → Active → Pool Approaching Expiry → Pool Extended → Active → ...
                                         ↘ (if wallet underfunded)
                                           Funding Required webhook

Deletion

Blobs can be explicitly deleted at any time through the API:

• JSON API: DELETE /api/v1/buckets/{bucket}/blobs/{key}
• S3 API: DeleteObject
• CLI: oyster delete <key> --bucket <bucket>

Deletion is reference-counted at the content-addressed level. The onchain delete_pooled_blob PTB fires only when the last reference to a given blob_id is removed from the account (see Content Addressing).

Local vs. onchain storage

Webhooks

Oyster posts a single webhook event: account.funding_required. It tells the owning app that an account's Pearl-derived wallet cannot cover the next extend_storage_pool PTB. Top up the wallet and the next extension cycle succeeds.

This page covers the trigger condition, payload schema, retry behavior, circuit-breaker semantics, and how to write a receiver.

Overview

When the extension worker tries to extend an account's StoragePool and Sui rejects the transaction with an insufficient-funds error, Oyster POSTs a JSON event to the receiver URL configured for the owning app. The receiver is expected to credit the wallet (or alert a human to do so) and acknowledge with a 2xx status.

Only account.funding_required is emitted currently. Future events share the same envelope shape. Receivers should switch on the type field rather than assuming a single schema.

Trigger condition

The webhook fires when all of the following hold during an extension cycle:

• The extension worker claims an account row whose pool_end_epoch < current_epoch + POOL_EXTEND_LOOKAHEAD_EPOCHS.
• The extend_storage_pool PTB submission fails with an error whose lowercased message contains insufficientgas, insufficientcoinbalance, or insufficient (case-insensitive substring match; see is_insufficient_funds_error in crates/oyster/src/webhook.rs).
• The owning app has a webhook receiver URL configured.

Any other class of failure (Sui RPC down, network timeout, or signing error) is logged and metered but does not fire a webhook.

Payload schema

The following JSON object is sent with every account.funding_required delivery.

{
  "event_id": "8f2c5e1a-...-uuid-v4",
  "type": "account.funding_required",
  "account_id": "acc_...",
  "pearl_address": "0x...",
  "amount": {
    "wal_frost": "12345678900",
    "sui_mist": "100000000"
  },
  "timestamp": "2026-05-05T10:31:00Z"
}

amount.* are decimal strings, not numbers, to avoid u64 precision loss in JSON. The SUI amount is currently a fixed 100_000_000 MIST (≈0.1 SUI) buffer. Oyster does not dry-run gas. The WAL amount is computed from the planned extension's encoded capacity × POOL_EXTEND_EPOCHS × the Walrus per-unit storage price.

Authentication

Every delivery is signed with a per-app Ed25519 keypair generated by the server when the webhook URL is registered. Two headers carry the signature:

The public key is returned in the response from PUT /api/v1/admin/app/webhook (or GET /api/v1/admin/app) as a base64-encoded 32-byte string. Receivers MUST verify the signature and reject deliveries whose fingerprint does not match the currently configured public key.

Retry policy

Oyster retries the same delivery up to MAX_RETRIES = 3 times with exponential backoff:

• Attempt 1: immediate.
• Attempt 2: after 100 ms.
• Attempt 3: after 200 ms.

(Backoff doubles each retry and is capped at 5 s, so the third sleep would be 400 ms, well under the cap.)

Retry semantics by response:

The same delivery might re-emerge later. See Idempotency.

Circuit breaker

To prevent a misbehaving receiver from monopolizing the extension worker, the webhook client wraps deliveries in a per-client circuit breaker:

• Closed (normal): every event attempts delivery.
• Opens after 5 consecutive failed deliveries.
• Stays open for 60 seconds. While open, new events are silently dropped (logged, counted on oyster_webhook_circuit_open_total, but not queued for later delivery).
• Half-open after the 60 s cooldown: the next event is allowed through as a probe. On success the circuit closes; on failure it re-arms for another 60 s.

Because dropped events are not queued, recovery from a long receiver outage relies on the next extension cycle re-claiming the account once its EXTENSION_CLAIM_COOLDOWN_SECS elapses. In practice, if your receiver is down, you miss notifications for the duration of the outage, but a healthy receiver starts receiving events again on the next cycle after recovery.

Idempotency

event_id is a fresh UUID v4 generated once per delivery in extension_task.rs, then reused across every retry attempt the webhook client makes for that delivery. Receivers MUST dedupe by event_id.

A separate delivery for the same account in a later cycle has a fresh event_id, so dedup is per-delivery, not per-account. If you want to suppress repeated notifications for the same underfunded account, do so in your receiver based on account_id and your own state.

Setup

Webhook receiver URLs are self-service. Use the oyster CLI (or the Admin API directly) with your app admin key:

# Register or rotate. Each call generates a fresh Ed25519 keypair;
# the response includes the new public key.
oyster app webhook set https://example.com/oyster/webhook

# Show the current URL and public key.
oyster app webhook show

# Stop deliveries.
oyster app webhook clear

set always rotates: the old public key is discarded and a fresh one is generated, so already-deployed receivers must be updated with the new key after each call. The corresponding HTTP endpoints are PUT /api/v1/admin/app/webhook, DELETE /api/v1/admin/app/webhook, and GET /api/v1/admin/app. See Admin API.

Verifying signatures

Verify each delivery against the public key returned at registration. Compute the verification over the exact request body bytes. Do not re-serialize the JSON.

Node.js (tweetnacl)

import nacl from "tweetnacl";

// Set this from the response body of `PUT /api/v1/admin/app/webhook`.
const PUBLIC_KEY = Buffer.from("<base64-public-key>", "base64");

function verifyOysterSignature(rawBody, headers) {
  const sig = headers["x-oyster-signature"] || "";
  const fp = headers["x-oyster-public-key-fingerprint"] || "";
  const expectedFp = PUBLIC_KEY.subarray(0, 8).toString("hex");
  if (fp !== expectedFp) return false;
  const sigPrefix = "ed25519=";
  if (!sig.startsWith(sigPrefix)) return false;
  const sigBytes = Buffer.from(sig.slice(sigPrefix.length), "base64");
  if (sigBytes.length !== 64) return false;
  return nacl.sign.detached.verify(rawBody, sigBytes, PUBLIC_KEY);
}

Python (pynacl)

import base64
import hmac
import nacl.signing
import nacl.exceptions

PUBLIC_KEY_B64 = "<base64-public-key>"  # from the PUT response
_PUBLIC_KEY_BYTES = base64.b64decode(PUBLIC_KEY_B64)
_VERIFY_KEY = nacl.signing.VerifyKey(_PUBLIC_KEY_BYTES)
_EXPECTED_FP = _PUBLIC_KEY_BYTES[:8].hex()

def verify_oyster_signature(raw_body: bytes, headers) -> bool:
    sig_header = headers.get("X-Oyster-Signature", "")
    fp_header = headers.get("X-Oyster-Public-Key-Fingerprint", "")
    if not hmac.compare_digest(fp_header, _EXPECTED_FP):
        return False
    if not sig_header.startswith("ed25519="):
        return False
    sig_bytes = base64.b64decode(sig_header[len("ed25519=") :])
    try:
        _VERIFY_KEY.verify(raw_body, sig_bytes)
        return True
    except nacl.exceptions.BadSignatureError:
        return False

Receiver examples

Both examples show the minimum viable receiver: dedupe by event_id, acknowledge promptly with 200, and return 5xx on processing failure so Oyster retries.

Node.js / Express

import express from "express";
import nacl from "tweetnacl";

const PUBLIC_KEY = Buffer.from(process.env.OYSTER_WEBHOOK_PUBKEY, "base64");
const EXPECTED_FP = PUBLIC_KEY.subarray(0, 8).toString("hex");

const app = express();
// We need the raw body to verify the signature; parse JSON ourselves.
app.use(express.raw({ type: "application/json" }));

const seenEventIds = new Set();

app.post("/oyster/webhook", async (req, res) => {
  const sig = req.header("x-oyster-signature") || "";
  const fp = req.header("x-oyster-public-key-fingerprint") || "";
  if (
    fp.length !== EXPECTED_FP.length ||
    !nacl.verify(Buffer.from(fp), Buffer.from(EXPECTED_FP))
  ) {
    return res.status(401).send();
  }
  if (!sig.startsWith("ed25519=")) return res.status(401).send();
  const sigBytes = Buffer.from(sig.slice("ed25519=".length), "base64");
  if (sigBytes.length !== 64) return res.status(401).send();
  if (!nacl.sign.detached.verify(req.body, sigBytes, PUBLIC_KEY)) {
    return res.status(401).send();
  }

  const { event_id, type, account_id, pearl_address, amount } = JSON.parse(
    req.body.toString("utf8"),
  );

  if (seenEventIds.has(event_id)) {
    return res.status(200).send();
  }
  seenEventIds.add(event_id);

  if (type !== "account.funding_required") {
    return res.status(200).send();
  }

  try {
    await topUpWallet(pearl_address, amount.wal_frost, amount.sui_mist);
    return res.status(200).send();
  } catch (err) {
    console.error("top-up failed for", account_id, err);
    seenEventIds.delete(event_id);
    return res.status(503).send();
  }
});

app.listen(8080);

Python / Flask

import base64
import hmac
import json
import os
import nacl.signing
import nacl.exceptions
from flask import Flask, request

PUBLIC_KEY_BYTES = base64.b64decode(os.environ["OYSTER_WEBHOOK_PUBKEY"])
VERIFY_KEY = nacl.signing.VerifyKey(PUBLIC_KEY_BYTES)
EXPECTED_FP = PUBLIC_KEY_BYTES[:8].hex()

app = Flask(__name__)
seen_event_ids = set()


def _verify(raw_body: bytes) -> bool:
    fp = request.headers.get("X-Oyster-Public-Key-Fingerprint", "")
    if not hmac.compare_digest(fp, EXPECTED_FP):
        return False
    sig_header = request.headers.get("X-Oyster-Signature", "")
    if not sig_header.startswith("ed25519="):
        return False
    try:
        sig_bytes = base64.b64decode(sig_header[len("ed25519=") :])
        VERIFY_KEY.verify(raw_body, sig_bytes)
        return True
    except (ValueError, nacl.exceptions.BadSignatureError):
        return False


@app.post("/oyster/webhook")
def funding_required():
    raw = request.get_data(cache=False)
    if not _verify(raw):
        return "", 401
    payload = json.loads(raw)
    event_id = payload["event_id"]

    if event_id in seen_event_ids:
        return "", 200
    seen_event_ids.add(event_id)

    if payload["type"] != "account.funding_required":
        return "", 200

    try:
        top_up_wallet(
            payload["pearl_address"],
            int(payload["amount"]["wal_frost"]),
            int(payload["amount"]["sui_mist"]),
        )
        return "", 200
    except Exception:
        app.logger.exception("top-up failed for %s", payload["account_id"])
        seen_event_ids.discard(event_id)
        return "", 503

In production, persist the dedup set (for example, Redis with a TTL of a few hours) so receiver restarts do not re-process events.

Error semantics

Metrics

The Oyster server's Prometheus endpoint exposes four webhook counters:

Pair these with the extension worker counters (oyster_extension_pools_extended_total, oyster_extension_errors_total{stage}) to alert on chronic under-funding without alerting on transient receiver failures.

Web Signup

Oyster can serve a self-serve signup page at /signup. A user signs in with Google, passes a Cloudflare Turnstile anti-bot check, and receives an app plus its first admin key, the same kind of key an operator would otherwise issue with oysterd app issue-admin-key. A small dashboard at /signup/keys lets them issue (capped), revoke, and rotate keys.

The feature is entirely opt-in. When its configuration is absent, the routes are not mounted at all.

Prerequisites

Two external accounts, both free:

1. Cloudflare Turnstile: create a widget in the Cloudflare dashboard (Turnstile → Add widget), listing your hostnames (plus localhost for dev). This yields the sitekey and secret key. Your site does not need to be behind Cloudflare.
2. Google OAuth: in a Google Cloud project, configure the OAuth consent screen (External; only non-sensitive scopes are used, so no verification review is needed) and create an OAuth 2.0 Client ID of type Web application with the redirect URI <OYSTER_PUBLIC_BASE_URL>/signup/callback. Publish the consent screen when you are ready for users outside your test list.

Configuration

Signup is enabled only when all five of the following are set. Setting only some is a startup error.

Behavior knobs (optional):

See .env.example at the repo root for a copy-paste template.

How it maps to the data model

Google account → users row (keyed by the stable sub claim in user_identities) → owns one apps row → holds app_admin_keys. Admin keys issued through the web are indistinguishable from CLI-issued ones. Google only authenticates the management of keys, never API calls themselves. Raw keys are shown exactly once and stored only as Blake2s-256 hashes. A lost key cannot be recovered, only replaced.

Waitlist review

In waitlist mode, a new user's first sign-in files a pending request and shows a "request received" page. Review requests with the server CLI (direct database access, the same as oysterd app):

oysterd signup list             # pending requests (TSV)
oysterd signup list --all       # include decided ones
oysterd signup approve <id-or-email>
oysterd signup reject <id-or-email>

No email is sent on approval. The user simply signs in again, and that sign-in completes signup and shows their admin key.

oysterd signup approve only flips an existing request, one filed when the person first signed in, because approval is matched on the Google sub, which you do not have until they authenticate. To pre-authorize someone who has not signed in yet, add their address to OYSTER_SIGNUP_ALLOWED_EMAILS (or their domain to OYSTER_SIGNUP_ALLOWED_DOMAINS). The gate matches the Google-verified email at sign-in time, so no prior request row or sub is needed. These lists are read from the environment, so changes take effect on restart.

Local development

Cloudflare publishes dummy Turnstile keys that require no account and skip real challenges:

TURNSTILE_SITE_KEY=1x00000000000000000000AA
TURNSTILE_SECRET_KEY=1x0000000000000000000000000000000AA

Google has no equivalent dummy mode. For a fully offline flow, use the signup testbed script, which boots Oyster with a mock Google OAuth server (scripts/signup-testbed.sh). To manually exercise real Google OAuth, register http://localhost:3000/signup/callback as a redirect URI on a dev OAuth client and set OYSTER_PUBLIC_BASE_URL=http://localhost:3000.

Operational notes

• Browser sessions live 8 hours in the web_sessions table (hashed tokens); an hourly sweep prunes expired rows.
• Web key issuance and revocation is recorded in audit_events (admin_key.issued_via_web and admin_key.revoked_via_web).
• closed mode is the abuse kill-switch: it stops new signups without affecting existing users' sign-in or their keys.

Pearl Master-Seed Rotation

Every Oyster account's wallet is an Ed25519 key that Pearl derives from a master seed with HKDF-SHA256. Pearl can hold several seeds at once, each with a version number, and every account records which version its wallet derives from (accounts.key_version). This guide is the operator runbook for moving accounts from one seed version to the next, which is what makes a leaked seed remediable.

Rotation is an on-chain move. The address an account funds and stores under is a function of the seed, so a new seed means a new address, and the assets at the old address have to be carried across. What a wallet owns is small and well defined:

• one StoragePool object (all of the account's PooledBlobs live inside it and travel with it),
• SUI coins (gas),
• WAL coins (storage payment),
• occasionally a Walrus Storage or Blob object left by an admin shrink.

oysterd keys migrate transfers all of these to the new-version address in one or a few transactions signed with the old key, verifies the pool arrived, and re-stamps the account. Nothing else references the address: the funding webhook and GET /account/wallet derive it on the fly, so after the flip they report the new one.

Prerequisites

• Oyster ≥ the release that ships oysterd keys (see the changelog) and Pearl ≥ 0.14.1 (versioned seeds).
• oysterd keys migrate|sweep needs the same environment as oysterd serve: DATABASE_URL, PEARL_GRPC_URL, PEARL_SERVICE_SECRET (or --pearl-service-secret-file), SUI_RPC_URL, WALRUS_SYSTEM_OBJECT, WALRUS_STAKING_OBJECT. Run it from a host with the production database and Pearl reachable; it does not need to be the serving host.
• Each account's old address must hold a little SUI. The move is paid by the old wallet. Accounts with nothing on-chain need no gas; accounts with a pool but no SUI are reported as NeedsGas and skipped until funded (a gas sponsor is not supported yet).

Procedure

1. Generate the new seed (≥ 32 random bytes, hex) and store it in the secret manager alongside the current one. Never reuse a seed.

2. Deploy Pearl with both seeds. The existing seed stays version 1 (PEARL_MASTER_SEED); the new one is PEARL_MASTER_SEED_V2 (or --pearl-master-seed-version-file 2:PATH). Leave PEARL_ACTIVE_KEY_VERSION=1 for now. Pearl refuses to start if the active version has no seed, and refuses to sign for a version it does not hold, so a typo fails closed.

3. Check the fleet.

   oysterd keys status
   

   Prints one row per key version with the account count and any rows currently holding a rotation lock. On a healthy fleet before the first rotation this is a single row for version 1 with zero locked.

4. Dry run.

   oysterd keys migrate --to-version 2 --dry-run
   

   Lists, per account, the old and new addresses and every object that would move, plus anything of a type the tool does not move. Nothing is locked or submitted. Review the skipped list: it should be empty.

5. Migrate.

   oysterd keys migrate --to-version 2
   

   Per account the tool takes a lock (accounts.key_migrating_since), moves the objects, confirms the pool is owned by the new address, sets key_version = 2, and releases the lock. While an account is locked, uploads, deletes and admin cap shrinks answer 503 and the extension worker skips its pool; the lock is held for the duration of one or two Sui transactions per account. Reads are unaffected throughout.

   The command prints one TSV line per account and exits non-zero if any account needs attention. It is idempotent: re-run it until it reports no problems. Accounts already at version 2 are skipped, an address that turns out to hold nothing just has its version flipped, and a run interrupted after the transfer but before the flip is repaired by the next run.

   Use --account <id> to migrate one account first, and --break-lock only for a lock that keys status shows was left behind by a crashed run and that nothing else is operating on.

6. Flip the active version so new accounts land on the new seed: PEARL_ACTIVE_KEY_VERSION=2 on Pearl, then restart Oyster (it reads the active version from Pearl at startup).

7. Sweep window. Integrators that copied a funding address rather than reading it from the funding_required webhook will keep paying the old address. Periodically move whatever lands there:

   oysterd keys sweep --from-version 1
   

   This takes no lock and changes no versions; it only drains the version-1 address of every account that is already past version 1 into that account's current address. Keep running it until several consecutive sweeps report nothing-on-chain for every account.

8. Retire the old seed. Remove PEARL_MASTER_SEED (version 1) from Pearl's configuration and the secret manager. From this point anything still sent to a version-1 address is unrecoverable, which is why step 7 comes first. Pearl will now refuse to derive version 1, so any account still on it would fail loudly; keys status must show zero accounts on version 1 before this step.

If the current seed has leaked

Treat it as a race: whoever holds the seed can drain the old addresses until the assets have moved. Do steps 1, 2 and 5 immediately, with no dry run, all accounts at once, and only then step 6 and the rest. The tool processes accounts sequentially with a short pause; on a large fleet run several instances with disjoint --account sets if speed matters. Rehearse the whole procedure on testnet before it is needed.

Verifying a rotation

For each migrated account, all of the following hold:

• oysterd keys status shows it on the new version with no lock.
• The StoragePool object is owned by the new address (Sui explorer, or the pool owner check the tool performs).
• The old address owns no objects.
• GET /api/v1/account/wallet returns the new address.
• A fresh upload, a read of a pre-rotation blob, and an extension cycle all succeed.

The key_rotation_e2e test in crates/oyster-e2e-tests runs exactly this sequence against an account created under version 1 on an in-process Sui + Walrus cluster.

Per-user key isolation

All accounts derive from one seed per version, so a seed leak exposes every wallet of that version at once. True isolation would give each account independent key material (for example a random per-account secret under envelope encryption in a KMS), so that one compromise is one wallet. Rotation as described here does not provide that, but the migration primitive is what a later move to per-account keys would use: derive the new address, drain the old one, re-stamp the row. The decision to defer per-user isolation, and the conditions for revisiting it, are recorded in docs/security/SEC-F3b-key-rotation-decision.md.