Skip to main content

Strongbox: Secrets management

Introduction​

When compute and applications are distributed across hundreds or thousands of sites, it is critical that data is protected from security breaches, both when in storage and when transported over the network. Physical theft is a real possibility.

Secrets such as crypto keys, certificates, and third-party access credentials should not be bundled into application images. Secrets should instead be accessed through fully authenticated and audited APIs that allow secrets to be updated and crypto keys to remain hidden without updating software images.

An intrusion at one site must not compromise data at other sites, and when a security breach occurs, it must be easily isolated and mediated.

Strongbox is a system-wide distributed service in Avassa for managing secrets and policies. Secrets are automatically shared to sites where they are required, and policies are applied across the entire system. Sharing only occurs in one direction, from the management point and outward. Local secrets are not propagated upward to avoid poisoning from a potential breach.

Key Strongbox features include:

  • Cryptographic isolation of secrets between tenants and sites
  • One-step operation to block a tenant, a site, and a host
  • Fine-grained control of how secrets are distributed to sites
  • Local secrets storage is sealed until remotely unsealed
  • Fully audit-logged access to secrets
  • Centralized key management: key rotation, revocation, access control
  • Encrypt and decrypt services that allow use without access to actual crypto keys
  • Format-preserving encryption and masking before logging, storage, and transport
  • Fully encrypted and authenticated communication both between sites and between hosts at a site
  • Mount secrets as files or environment variables in containers
  • ACME protocol support for automated certificate provisioning
  • Kubernetes secrets provider for injecting secrets into Kubernetes workloads

Authentication and authorization​

All access to Strongbox is authenticated and audit-logged. There is no unauthenticated path to secrets.

Application authentication​

Applications authenticate using Approles, a split-credential scheme where one part (the role-id) is embedded in the application image and the other part (the secret-id) is injected by the scheduler at startup. Neither part alone is sufficient to authenticate. This protects against rogue containers and stolen credentials.

For workloads that need a stronger identity guarantee, Strongbox supports SPIFFE/SVID-based authentication, which issues verifiable workload identity credentials without requiring static secrets. See SPIFFE support.

User authentication​

Users authenticate through identity services: built-in username/password, OIDC, JWT, or SPIFFE. Multi-factor authentication is available via TOTP (Google Authenticator-compatible) and passkeys (WebAuthn/FIDO2). See Authentication.

Policies and tokens​

Every authenticated identity receives an access token with associated policies that define what it can read, write, or invoke. Policies are managed centrally and distributed to all sites. All access is audit-logged. See Authorization and the policy how-to.

The seal​

The Strongbox application consists of a protected core process that handles plain-text data. No data is allowed to leave the core process unencrypted. The state of the Strongbox application is encrypted using AES 256 GCM when it leaves the core process.

The key to this cipher (the sealkey) is not stored locally -- it must be provided by an external entity. Until this happens, Strongbox is sealed and cannot access its internal state. The sealkey is split into five parts using the Shamir Secret Sharing algorithm, and at least three parts must be combined to recover it. Parts should be stored separately.

Automatic vs. manual unseal​

Each site in the Avassa hierarchy can be configured to unseal automatically by retrieving its sealkey from its parent site. The sealkey is not stored in plain text at the parent -- it is protected by a key pair, optionally stored in a TPM. Individual controller nodes at a site hold separate shares of the access credential, requiring a majority quorum to assemble, which prevents a single stolen node from triggering unseal.

When a site cannot reach its parent, an administrator can manually unseal it via the local craft interface. The Control Tower, which has no parent, always requires external unseal after a restart.

For step-by-step unseal procedures, see How to unseal a site locally.

Tenant isolation​

Each tenant's data is encrypted with a unique key, separate from the sealkey. Tenant keys are unique per site, so a compromise of one tenant's key at one site does not expose data for any other tenant or for the same tenant at other sites. Data in transit between sites is additionally protected by a tenant-specific transfer key, which can be rotated immediately to isolate a compromised site.

Distribution​

Secrets flow in one direction only: from the Control Tower downward through the site tree. An edge site never propagates secrets upward. Distribution must be explicitly configured -- secrets are not shared to any site unless the operator says so, and only to sites that the tenant has been assigned to.

When a site comes online (or transitions from blocked to unblocked), it pulls its initial secret state from its parent. Subsequent changes are propagated as minimal encrypted diffs using the tenant's transfer key, keeping bandwidth low even at scale.

Vault: key/value store​

Vault is an encrypted key-value store. Multiple vaults can coexist within a tenant, each with its own distribution settings. Within a vault, each secret is a dictionary of key/value pairs treated as an atomic unit.

Vault supports two modes:

  • Standard vault: stores the current value only.
  • Versioned vault: keeps a configurable history of previous values, supports read-back of any prior version, and supports cooperative locking (a write is rejected unless the caller supplies the expected current version).

Secrets can be mounted into containers automatically at startup, either as files in a volume (keys become filenames, values become file content) or as individual environment variables. This lets applications consume secrets without any Strongbox API calls.

Crypto functions and transit keys​

Applications should not hold encryption keys. Instead, Strongbox exposes cryptographic operations over an authenticated API under the transit path. The key material never leaves Strongbox in plain text (unless explicitly exported), but the application can ask Strongbox to encrypt, decrypt, sign, or verify on its behalf.

Supported operations include:

  • Encryption and decryption (symmetric and asymmetric ciphers)
  • Signing and signature verification
  • HMAC calculation
  • Key derivation: derive many logical keys from one stored key using a caller-supplied context, without storing additional key material
  • Convergent encryption: the same plaintext always produces the same ciphertext, useful for equality comparisons without decryption
  • Hash functions (SHA-2 and SHA-3 family)
  • BCrypt password hashing and verification
  • Data key generation: generate random key material wrapped under a transit key, for bulk encryption outside Strongbox
  • Key export and backup/restore: exportable keys can be checked out for bulk operations; backups can be restored under a different name

Key versioning and rotation​

Transit keys support multiple active versions. Each ciphertext carries the version used to encrypt it, making it straightforward to phase in a new key while old data is migrated. Minimum encryption and decryption versions are independently configurable, and keys can be trimmed once old versions are no longer needed. Auto-rotation can be configured to rotate keys on a schedule, with tunable activation and deprecation delays.

For configuration examples, see How to use transit keys.

Transformation​

Sometimes it is necessary to protect data while preserving its original format -- for example, encrypting a credit card number so that the result still looks like a credit card number to a downstream system that validates format but does not decrypt.

Strongbox provides two related capabilities:

  • Format-preserving encryption (FPE): encrypts data so the output has the same format as the input. The result can be decrypted to recover the original value.
  • Masking: replaces characters with a masking character (for example *). Masking is irreversible, making it suitable for logging and audit trails where the actual value must never appear.

Both modes are configured with a template that specifies the allowed alphabet and a pattern identifying which parts of the input are subject to transformation. A tweak value can be used as additional input to the encryption, making ciphertexts context-sensitive and harder to correlate.

For step-by-step setup, see How to use transformation.

SSH​

Strongbox can act as an SSH Certificate Authority, issuing both host certificates and client certificates.

Why certificates instead of public keys​

Traditional SSH access requires each user's public key to be installed on every host they need to reach, and removing access means removing the key from every host. Similarly, users must manually accept and store a host's public key the first time they connect, which creates a risk if the key is accepted without verification.

SSH certificates solve both problems: clients trust all hosts whose certificates are signed by the Strongbox CA, and hosts trust all users whose certificates are signed by the CA. Revoking access requires only that a user's certificate expires or a new certificate is issued with restricted principals -- no changes are needed on individual hosts.

Host certificates​

Host certificates identify the server to the connecting client. Clients configure their SSH client to trust certificates signed by the Strongbox CA, eliminating the need to manually verify and store individual host keys.

Client certificates​

Client certificates authenticate the user to the server. Certificates are issued with a limited validity period. When a user's access should be revoked, no new certificate is issued; the old one expires naturally. Hosts only need to trust the CA, not individual users.

One-time passwords​

As an alternative to certificates, Strongbox can issue SSH one-time passwords (OTPs). An OTP is bound to a specific user and IP address and can be verified exactly once, preventing replay attacks.

Roles​

SSH roles configure what kind of certificates and OTPs a given path can issue -- for example, restricting principals, validity periods, or source IP addresses.

For configuration and usage examples, see How to manage SSH certificates.

TLS/SSL​

Strongbox can act as a TLS Certificate Authority, either as a self-signed root CA or by importing an existing CA certificate. This is primarily intended for securing application-to-application communication using mutual TLS, but it also works for web traffic.

Strongbox supports issuing intermediate CA certificates, allowing organizations to build a distributed trust hierarchy.

Certificates are issued through roles that define what the CA is allowed to sign -- for example, which hostnames and domains are permitted, whether client or server certificates are allowed, and the maximum validity period. A certificate revocation list (CRL) is maintained and can be served on demand.

Certificates can be configured for automatic rotation before they expire, reducing operational overhead for long-running services.

For configuration examples, see How to set up an SSL CA.

ACME certificates​

Strongbox supports the ACME protocol, enabling applications to automatically request and renew TLS certificates using the same challenge-response mechanism used by Let's Encrypt. This is useful for edge workloads that need valid TLS certificates without manual provisioning.

For setup instructions, see How to use ACME certificates.

Kubernetes secrets​

Secrets stored in Strongbox can be injected directly into Kubernetes workloads running at edge sites. This allows Kubernetes-native applications to consume Avassa-managed secrets without requiring any code changes.

See the Kubernetes secrets provider tutorial for a complete walkthrough.