Applications and Deployments

The fundamental workload in the system is an application. An application consists of a set of containers that run together to provide some services. An application is deployed to sites as one versioned unit.

An application is declaratively described in an application specification, in YAML or JSON. The application specification has a version, and defines which containers should run, and how, within one site.

How an application is deployed is declaratively described in an application deployment specification, in YAML or JSON. The application deployment specification defines which sites the application should run on, and how the application and new versions of the application should be rolled out in the system.

The following example application deployment specification says that the application ticket-manager should be deployed to all sites that have a label system/type with the value edge. It also says that whenever a new version of the application specification is available, the system should do a rolling deployment to all edge sites, i.e., deploy to 2 sites at the time, and wait 15 minutes between each batch of sites:

name: ticket-deployment
application: ticket-manager
application-version: "*"
placement:
  match-site-labels: >
    system/type = edge
deploy-to-sites:
  sites-in-parallel: 2
  healthy-time: 15m

The following example application specification says that the ticket-manager application consists of one service with one container, which should run in one instance in each site to which it is deployed:

name: ticket-manager
version: "1.0"
services:
  - name: ticket-manager-service
    containers:
      - name: ticket-blipper
        image: "avassa-public/movie-theaters-demo/ticket-blipper:v1.0"
    mode: replicated
    replicas: 1

Application

An application defines a set of services, and each service has a set of containers. Within a site, each service in an application can run in one or more instances. A service instance is the smallest scheduled unit, which means that all containers in a service are always scheduled together.

All service instances in an application running in a site have their own isolated ip network (the application network), alternatively share an isolated ip network with selected other applications within the same tenant (shared application network).

An application specification has a version. The system keeps track of the last four versions of the application specifications.

Application deployment

An application deployment serves two purposes - define which sites an application should run in, and define how new versions (including the initial version) of the application should be deployed to these sites.

Site placement

The tenant that creates the application deployment in the system has access to a set of assigned sites. The application can be deployed to all of these sites, or to a subset of them. This is defined with a label match expression in the application deployment specification. Whenever an application deployment is updated, the label match expression is re-evaluated, and the deployment is adjusted to the set of sites that match expression. If a site's labels are modified, or a new site is assigned to the tenant, the application is not automatically redeployed. However, the application can be redeployed manually or programmatically at any time.

Deployment strategies

The system supports three deployment strategies,

1. All at once
2. Canary deployment
3. Rolling deployment

All at once means that the application is deployed to all selected sites simultaneously. This is the default strategy.

The canary deployment strategy means that a small configurable set of sites are selected as canary sites. The application is first deployed to these sites. If the deployment succeeds on these sites, the system waits for a healthy timer to expire, before continuing with the rest of the sites, either all at once, or rolling.

In a rolling deployment, the system selects a configurable number of sites at random, and the application is deployed to these sites. As in the canary deployment, the system waits for the healthy timer to expire before continuing with a new set of sites and so on.

If an error occurs during a canary deployment or a rolling deployment, the deployment is aborted.

Application versioning

The application deployment specification has a field application-version that controls which version of the application specification to deploy to the system.

If the application-version field has the special value "*", the deployment is used for any version of the application. This means that whenever the application specification is updated and its version is modified, the new version will be deployed according to the application deployment specification.

If the application-version field has some other value, the system will strive to deploy that specific version of the application. If that version doesn't exist, the deployment will wait for it to be configured in the system.

If an application is updated without changing its version, the changes are immediately propagated to all sites where it is deployed.

Deployment failures

If an application fails on some site during a canary or rolling deployment, the system will stop the deployment, i.e., it will not further deploy the application to more sites. This situation can be handled in two different ways; either rollback or deploy a new version of the application.

In order to rollback to the previous version, update the application-version field in application deployment specification with the value of the previous application version.

Since the system keeps track of four versions of the applications, so it is not necessary to update the application specification at this stage.

Note that if an application is deployed to a site, and the connection to that site either is down, or goes down before the site has reported the result of the deployment, this is not treated as an error. When the connection is restored, the deployment will continue.

Application upgrades

When a new version of an application is deployed to a site, the application specification contains instructions for how the new version is introduced in the site.

The system supports two application upgrade methods - stop-and-restart and per-service.

• The upgrade method stop-and-restart means that all service instances that are running the old version of the application are stopped, and when all these have stopped, the service instances are started using the new version of the application.

• The upgrade method per-service contains specific instructions for each service in the application. Each service can either do a rolling upgrade or be stopped and then restarted.

Application upgrade windows

The system supports per-site application upgrade windows, which are used to define windows of time when the applications can be upgraded by the system on the site.

The application upgrade windows can be defined both by the site provider and by application owners.

Application queue

When a new version of an application is deployed to a site, in the simplest case it is sent to the site, which picks it up and immediately upgrades the application. However, if there is an application upgrade window on the site the upgrade might have to wait. Also, a site might be offline for a longer period of time, which means that the new version cannot be sent to the site until the site comes online.

In order to handle these situations, the system implements an application queue. When a site receives a new version of an application, it is stored in this queue. When a site that has been offline comes online, there may be a number of application versions entering the queue at the same time.

When the queue is non-empty and the application upgrade window opens, the system checks the queue to see if there are application versions that can be skipped. For example, if the system is running version 1.2.0 and the queue contains patch versions 1.2.1, 1.2.2, 1.2.3, 1.2.4, it may be safe to skip 1.2.1 - 1.2.3 and just install 1.2.4. This upgrade behavior is specified per application in the application specification.