Skip to main content

Infrastructure networking

This chapter describes the site-to-site communication, and the host-to-host communication within a site, and how this communication is secured.

Initial edge site communication​

When a host is first started, it calls home to the Control Tower. If the Control Tower accepts the host, it generates a host-specific API certificate, a host-specific certificate for the site-local traffic, and a site-specific access token, and sends the host. These certificates and the access token are stored encrypted in strongbox on the host.

Once the hosts in a site have successfully completed the call-home procedure, they form a cluster by communicating using mutual TLS.

The API and site certificates are automatically rotated by strongbox at regular intervals.

When the site cluster has been formed, strongbox uses the site-specific access token to initiate its seal.

Initial site network

Site to site communication​

Once the site is up and running, there are two site-to-site communication paths. Both are initiated from the edge sites.

The first is volga infrastructure, which uses QUIC with mutual authentication if UDP port 443 is reachable, otherwise falls back to mutual TLS on TCP port 443.

The second is communication with the container image registry, over the standard protocol defined by the Open Container Initiative Distribution Specification. It uses authenticated TLS with the site-specific access token.

Site to site network

Site to parent site communication using a proxy​

Sometimes some or all hosts within an edge site are connected to a restricted network and cannot directly communicate with the parent site. In this case it is possible to configure proxies which handles all communication upwards to the parent for other hosts. A proxy is an Edge Enforcer that has connectivity to both the restricted network and the Control Tower, and it may proxy parent site connections for both hosts within the same edge site and hosts in other edge sites.

The kind of communication is the same as described in Initial edge site communication and Site to site communication. The call-home, strongbox init and volga connections are end-to-end encrypted, so that the proxy host is not aware of the contents of the communication. The image download connection is currently end-to-end encrypted too, but will be replaced by a caching proxy in the future.

Site to site proxy network

The parent proxy is configured as described in the Control Tower proxy.

Site communication​

The hosts within a site form a cluster and use mutual TLS for the site's internal traffic.

Cluster network

Application traffic​

The application networks run VXLAN over wireguard. There is one key pair per host. The private keys are stored encrypted by strongbox, and are never written in clear text in the filesystem.

In the picture below, application App-1 runs on host A and C, and application App-2 on all hosts.

Application network