Sites

A site is a group of one or more hosts that act together as a cluster. Sites can be as small as a single host or span many hosts. The main property of a site is that the Control Tower treats it as a deployment target for applications.

Hosts

Each host in a site runs an Edge Enforcer. Hosts can be physical or virtual machines and may reside in the same physical location or be geographically separate.

note

Stable IP connectivity between hosts within a single site is an important requirement for the site to be able operate efficiently. The reason for this is that hosts rely on a consensus algorithm for scheduling and other coordination, but also a stable network is required for tasks such as image replication and application networking within the site.

Hosts can be added to or removed from a site. For information about replacing or removing hosts, see Site maintenance.

Each host has a unique host identifier (for example, the serial number of the device). A site and the host identifiers of all its hosts must be configured in the Control Tower.

Controllers

Depending on how many hosts a site has, 1, 3, or 5 hosts will be designated as the controllers. By default, the Avassa system designates controllers automatically, but explicit configuration is also possible.

The controller nodes run the Raft algorithm. All shared state (for example, all configuration) is stored on the controller nodes.

A non-controller node does not store any state, and its sole purpose is to run applications. Note that controller nodes also run applications.

Explicit configuration of controllers

When the hosts on a site are configured, it is possible to explicitly configure a host as a controller. If no hosts are configured to be controllers, the system designates controllers automatically. It is also possible to configure a host to not be a controller. In this case the system will not pick that host when it designates controllers.

Application deployment and scheduling

The Control Tower deploys applications to sites and is not concerned with which individual hosts the service instances end up being scheduled to. Each site has a local scheduler that distributes service instances across the available hosts.

Within a site, hosts replicate application images, secrets, and configuration so that every host is ready to run any application assigned to the site. Note that internal application state such as the data in an application's volumes is not replicated.

If a worker host or a minority of controller hosts fail, the site continues to operate. When applicable, service instances that were running on the failed host are restarted on the remaining hosts, using the already-replicated images and configuration.

Autonomous operation

Each site has an autonomous local control plane. Even when connectivity to the Control Tower is interrupted, the site continues to run its applications. The local scheduler, Strongbox secrets manager, and local message delivery on Volga all operate independently of the Control Tower. A site disconnected from the Control Tower can be managed locally as described in the Managing disconnected sites how-to guide.

Call-home

When a host starts up for the very first time, it calls home to its configured parent site and provides its unique host identifier. The parent site either accepts the host and informs it of its role, or rejects it. A host is rejected if its host identifier is unknown or already in use.

After a successful call-home, the host creates a certificate used for internal site communication.

Initial site call-home

When a site is configured, the system creates a wrapped site bundle for the site. This bundle contains an access token and a key used for secure communication with the Control Tower. For security reasons, the site bundle can only be unwrapped once.

When the first controller hosts in a site call home, they form a local cluster, unwrap the site bundle, and create the Strongbox seal. See The Seal for details.

Sites and multi-tenancy

While it is possible to use the Avassa system with only a single tenant, there is built-in support for multi-tenancy. There are two kinds of tenants defined:

• Site provider, owns the edge infrastructure and manages the sites and hosts
• Application owner, develops and deploys applications to sites

note

The concept of multi-tenancy in the Avassa system is covered in detail in the Multi-tenancy and users tutorial.

In this hierarchy, the site provider tenant is the one who owns the sites and manages system configuration. The site provider also decides which sites the application owner tenants may deploy their applications to and determines which resources (how much CPU, memory, which devices or GPUs) the application owner tenants have access to on each site.

The site provider gives access to a certain site by assigning this site to an application owner:

supctl create tenants acme assigned-sites <<EOF
name: stockholm-sergel
EOF

Each site may be assigned to one or more application owners. Site providers may deploy applications to any site without an explicit assignment.

The site provider defines which resources the application owner should have access to by means of tenant-resource-profiles.