Chalk Signing Key Backup Service

Chalk can make use of the Crash Override Signing Key Backup Service to securely backup encrypted forms of the values used by chalk for a variety of signing and attestation operations.

All secrets and keying material are locally generated on the system running chalk, with the secret itself being encrypted locally prior to being sent to the service.

This document provides an overview of the Signing Key Backup Service, how data is stored securely, and how chalk interacts with the API as a client.

Using the Chalk Signing Key Backup Service

Use of the Signing Key Backup Service is simple and chalk provides builtin command to help get things setup, and backed up, securely.

For chalk to be able to use the Signing Key Backup Service it needs to be configured with an API token. Running the following command will download a pre-configured chalk profile:

chalk load

The user will then be prompted to supply an API token:

info:  Attempting to load module from:

Configuring Component:
Configuring variable: auth_config.crashoverride.token -- CrashOverride API token
Get your API token at CrashOverride:

Default is:
Press [enter] to accept default, or enter a value:

The API token to provide to chalk can be found in the Crash Override console application by navigating to unchalked Repositories. It will have instructions how to setup chalk repositories which will show the API token to be copy-pasted into the terminal.

Upon pressing enter chalk will validate the supplied token and confirm that the configuration process has completed. Chalk will then validate the configuration parameters as a whole and embed them into the chalk binary.

Now chalk has been configured with a valid API token the backup process can be initialised and will generate a new keypair alongside a passphrase that is used encrypt the private key prior to sending to the backup service:

chalk setup

You will then see a number of lines of output in the terminal similar to the below:

info:  Ensuring cosign is present to setup attestation.
info:  Successfully stored encrypted signing key.
warn:  Please Note: Encrypted signing keys that have not been READ in the previous 30 days will be deleted!
info:  The ID of the backed up key is: 1234567890123456789012345678901234567890

To test the retrieval of the encrypted secret you can insert chalk on any file:

echo '#!/bin/sh' >
chalk insert

Chalk will detect that keys have already been generated and will retrieve the encrypted secret and use it to perform test sign and verify operations. When this is done successfully you will see terminal output similar to the below:

info:  Retrieved attestation key password from Signin Key Backup Service
info:  Ensuring cosign is present to setup attestation.
info:  Test sign successful.
info:  Test verify successful.

The encrypted secret will be automatically retrieved by chalk as needed as should not be something you have to worry about further.

If you chose to not use the API and self-manage your secrets please ensure make a copy of the randomly generated password that will be displayed and save it somewhere securely.


What exactly is saved to the Chalk Signing Key Backup Service?

Chalk saves the minimum required to backup your signing key. All key materials are generated and handled locally, with only an encrypted form of the secret ever leaving your system.

Specifically the Chalk Signing Key Backup Service stores the following data for each secret that is backed up to it:

  • secret_id - This is the unique identifier of each backed up secret. It is generated by chalk locally using the following scheme:

    sha256(16_bytes_random + private_key)
  • encrypted_secret - This is the encrypted form of the secret used to secure the private key. The value is constructed using a Luby-Rackoff cipher with a random nonce. (More details here)

  • owner - The owner of the backed-up encrypted secret. This value is constructed using the following scheme that avoids any plaintext identities being saved but still allowing the authenticated identity to be used for access control to the data:

    sha256(salt + authenticated_user_id)
  • creation_ts - This is the timestamp at which the encrypted secret was created

  • accessed_ts - This is the timestamp at which the encrypted secret was last accessed

How long will stored data be saved for?

Backed up encrypted signing keys will be saved for 30 days from the last time they were accessed. Simply invoking chalk after you have backed up keys will cause the backed up keys to be read and ensure they continue to be fresh.

How does the Chalk Signing Key Backup Service backup its own data?

The Chalk Signing Key Backup Service makes use of the AWS Backup service to periodically snapshot all application state. Backups are encrypted at rest and are kept for 1 year.

Are there more specifics on the encryption scheme used?

There sure are! In addition to the details provided below you can review the code that is used for the encryption of the secret in prp.nim that is part of the nimutils project that Crash Override has released open source.

The encryption scheme makes use of a PRP using the Luby-Rackoff construction. The easiest thing for us to do is to break the input into two 'halves',one being 128 bits (the width of AES, which we will call the 'left half'), and the other the rest of the remaining width of the input (the 'right half').

The nonce is random.

We first take the key passed in and, use it to generate pre-round keys.

The pre-round keys work by running a PRF in counter mode, where the starting value of the counter is the nonce. We use this to get the actual round keys (the nonce is not necessary in the luby-rackoff proof, but helps prevent dictionary attacks if they're viable in the threat model).

Each round of luby-rackoff requires a PRF. One is AES-CTR mode. We key with the round key, and use the left half as the starting position of the counter. We use that to generate a key stream, that we XOR into the right half. The other PRF is HMAC-3. We take the round key, HMAC the right side, truncate the result to 128 bits, then XOR into the left half.

The PRFs are used in a Feistel cipher, so we alternate PRFs through our four Feistel rounds.

While three-round Luby-Rackoff is secure against some use cases, we go through the full four rounds.

PRPs are reversible, and with Feistel construction, it's by running the rounds backward.

Once constructed it is this encrypted value that is sent to the API for secure backup.

I have more questions that are not answered here, where should I go?

We'd love to hear from you, please reach out to us at