Skip to main content

Hashi Node Backups

Hashi automatically backs up the data a node needs to rejoin the committee after data loss. Every epoch, Hashi writes the node config, referenced key files, and database snapshot to a tar archive, then compresses and encrypts it as an ASCII-armored PGP message. The backup files use the suffix .tar.asc.

Encryption only needs the public PGP certificate, so the backup key does not need to be present on the server during normal operation. Restore needs access to the matching private key, either as a local PGP secret-key file or through gpg-agent for YubiKey-backed keys. Because gpg-agent can be forwarded over SSH, you can restore on a remote server using a YubiKey plugged into your laptop instead of plugging the YubiKey into the server itself.

Create a backup Key​

We recommend generating a PGP private key entirely on a YubiKey, and using its public key to encrypt the node backup. This makes it practically impossible for anyone to decrypt the backup without physical access to the YubiKey. The risk is that you will also be unable to decrypt the backup if you lose access to the YubiKey. To minimize this risk, we recommend obtaining at least two YubiKeys and repeating the procedure for each.

For the initial YubiKey setup, you must use a machine with a physical USB port. After initial setup, you can encrypt the share to the public key of the YubiKey on any machine, including a cloud server without a USB port.

  1. Obtain a new YubiKey 5 Series. There are six models: the YubiKey 5C NFC, 5 NFC, 5C, 5 Nano, 5C Nano, and 5CI. Any of these models works, though avoid the Nano models because they are smaller and easier to lose.

  2. Install tools on the USB-capable machine:

    • oct, OpenPGP card tools.

  3. Insert a YubiKey and confirm it is visible. Make sure no other YubiKeys or smartcards are plugged into the machine.

$ oct list

You will see something like this:

Available OpenPGP cards:
0006:26883270
  1. Save the card id:
CARD=$(oct list --idents-only | head -n1)
echo "$CARD"
  1. Change default PINs

The User PIN is used for daily cryptographic operations like decryption; the Admin PIN is used for card configuration such as generating/importing keys.

The default PINs for new YubiKeys are: User PIN: 123456 Admin PIN: 12345678

PINs can contain numbers, letters, and symbols (despite the name), and must be 6, 7, or 8 characters long.

Determine a new user and admin key for the YubiKey. These PINs are essential. If you lose them, you cannot decrypt the node backups. Back up the PINs in a safe place, such as a company password manager. Do not use the default PINs.

Change the PINs with the following commands. You will first be prompted to enter the current (default) PIN, then prompted to enter the new PIN twice.

oct pin --card "$CARD" set-user
oct pin --card "$CARD" set-admin
  1. Generate the PGP key on the YubiKey. You will be prompted to enter both the new user and new admin PINs. The public key will be saved to the output file.
oct admin --card "$CARD" generate \
  --userid "Your Name <you@example.com>" \
  --output "yubikey-public-$CARD.pgp" \
  curve25519

  1. Repeat the above steps for any additional YubiKeys.

  2. Store the YubiKeys in a secure location, such as a company safe. Also save the public key files in a secure location.

Option 2: Create a local private key file​

If you do not want to use a YubiKey, you can generate a PGP private key locally and encrypt with it. Because the private key is stored in a normal file instead of on a YubiKey, there is a greater risk of unauthorized use of the private key. Store the file carefully.

  1. Install tools:

  2. Generate a new key pair:

sq key generate \
  --own-key \
  --name "Your Name" \
  --email you@example.com \
  --expiration never \
  --without-password \
  --output private-key.asc \
  --rev-cert revocation-cert.asc
  1. Export the public key to a file. Note that despite the name, this doesn't actually delete the private key, it just extracts the public key.
sq key delete \
  --cert-file private-key.asc \
  --output public-key.asc
  1. Save the revocation, public key, and private key files in a secure location.

Configuring hashi​

To enable automatic backups using the new PGP public key, add the public cert to the node config file under the field backup-pgp-cert.

You can also configure the location where Hashi saves backups with the field backup-dir.

Cloud backup​

In the future, we will likely add support for automatically uploading backups to cloud storage such as s3.