Skip to main content

Infrastructure networking

This chapter describes the site-to-site communication, and the host-to-host communication within a site, and how this communication is secured.

Initial edge site communication​

When a host is first started, it calls home to the Control Tower. If the Control Tower accepts the host, it generates a host-specific API certificate, a host-specific certificate for the site-local traffic, and a site-specific access token, and sends the host. These certificates and the access token are stored encrypted in strongbox on the host.

Once the hosts in a site have successfully completed the call-home procedure, they form a cluster by communicating using mutual TLS.

The API and site certificates are automatically rotated by strongbox at regular intervals.

When the site cluster has been formed, strongbox uses the site-specific access token to initiate its seal.

Initial site network

Host authentication during call-home​

By default, the Control Tower accepts a call-home if the host identifier is known and not already in use. For deployments that require stronger assurance of host identity at first contact, the Control Tower can be configured to require a challenge-response handshake based on an x509 certificate issued by the Control Tower's Strongbox CA. The host fetches a server-issued nonce, signs it with its private key, and submits the signature together with the certificate chain in the host-init request. The Control Tower verifies the certificate chain against the relevant CRLs and verifies the signature.

This provides cryptographic proof of host identity without requiring mutual TLS at the transport layer, so it works behind load balancers (such as AWS) that cannot enforce per-CA client-certificate restrictions. The credential is only needed for the initial call-home; once it succeeds, the host uses the certificates issued by the Control Tower for all subsequent communication.

Flow 1 — Standard call-home​

What this gives you:

  • Trust direction: the host verifies the Control Tower using the system's CA certificates. . The Control Tower trusts the host because an operator pre-registered the host-id.
  • Private key never leaves the host: the Control Tower only sees the public key, exactly as with a CSR-based enrolment.
  • When to use it: simple deployments where pre-registering each host-id is acceptable.

Flow 2 — Call-home with x509 challenge​

What this gives you:

  • Mutual proof at first contact: before the host has any cluster identity, it has already proven it controls a key the Control Tower issued.
  • Revocable in real time: revoking the certificate (revoke-call-home-cert) takes effect on the next attempt — no need to roll out config to the host.
  • Bound to host-id (default policy): a leaked credential cannot onboard a different host. The site and system-ca policies relax this when a single credential should serve a group of hosts.
  • One-time use: after the first successful call-home, the host has full cluster credentials; the challenge credential can be deleted from the host.

See Challenge-Response Call-Home Authentication for configuration and credential management.

Site to site communication​

Once the site is up and running, there are two site-to-site communication paths. Both are initiated from the edge sites.

The first is volga infrastructure, which uses QUIC with mutual authentication if UDP port 443 is reachable, otherwise falls back to mutual TLS on TCP port 443.

The second is communication with the container image registry, over the standard protocol defined by the Open Container Initiative Distribution Specification. It uses authenticated TLS with the site-specific access token.

Site to site network

Site to parent site communication using a proxy​

Sometimes some or all hosts within an edge site are connected to a restricted network and cannot directly communicate with the parent site. In this case it is possible to configure proxies which handles all communication upwards to the parent for other hosts. A proxy is an Edge Enforcer that has connectivity to both the restricted network and the Control Tower, and it may proxy parent site connections for both hosts within the same edge site and hosts in other edge sites.

The kind of communication is the same as described in Initial edge site communication and Site to site communication. The call-home, strongbox init and volga connections are end-to-end encrypted, so that the proxy host is not aware of the contents of the communication. The image download connection is currently end-to-end encrypted too, but will be replaced by a caching proxy in the future.

Site to site proxy network

The parent proxy is configured as described in the Control Tower proxy.

Site communication​

The hosts within a site form a cluster and use mutual TLS for the site's internal traffic.

Cluster network

Application traffic​

The application networks run VXLAN over wireguard. There is one key pair per host. The private keys are stored encrypted by strongbox, and are never written in clear text in the filesystem.

In the picture below, application App-1 runs on host A and C, and application App-2 on all hosts.

Application network