Skip to content

Commit Based Deployment Pipelines

Use cases

  1. Deploying infrastructure.
  2. Deploying Open edX instances.
  3. Performing mass upgrades.
  4. Archiving instance

Current state

Currently, we are generating our deployment pipeline with the generate-pipelines.py script. We are generating them once we add a commit to the default branch. Instance deployments need manual confirmation.

There is an existing format for auto-deployment commit messages.

[AutoDeploy] $INSTANCE_NAME:$DEPLOYMENT_REQUEST_ID - $MESSAGE

The value of $MESSAGE can be either of deploy new instance :rocket: - when creating a new instance or apply changes :tools: - updating an existing instance. $INSTANCE_NAME is self-explanatory and $DEPLOYMENT_REQUEST_ID is provided by the GitLab trigger client to track the job.

But regardless of the message, currently, generate-pipelines.py generates the same pipeline for every commit.

Since Grove manages multiple instances in a Kubernetes cluster and the infrastructure for the cluster, we want to have fine control over what deployment jobs are running and when. We don't want to run all jobs every time there is a new commit.

A pipeline generation based on commit messages would help up achieve that and open room for the optimized pipeline in different situations.

Proposed solution

We propose a new format for deployment commits:

[AutoDeploy][$TYPE] $REST

We will only perform deployment-related CI/CD jobs if the commit title has the [AutoDeploy] tag in it.

So for different situations, we will have the following commit messages:

Deploy infrastructure using Terraform

[AutoDeploy][Infrastructure] <any message here>

Create and deploy a new Open edX instance

[AutoDeploy][Create] <INSTANCE_NAME>|<DEPLOYMENT_ID>,<INSTANCE_NAME>|<DEPLOYMENT_ID>,...

Where <INSTANCE_NAME> is the instance name, <DEPLOYMENT_ID> is an optional client-provided deployment id. This will first deploy infrastructure using terraform and then deploy the instance. The pipeline will only contain jobs related to provided <INSTANCE_NAME> list. Other instances in the repository will be ignored from the pipeline.

Update an Open edX instance

[AutoDeploy][Update] <INSTANCE_NAME>|<DEPLOYMENT_ID>,<INSTANCE_NAME>|<DEPLOYMENT_ID>,...

Similar to [AutoDeploy][Create] except it will not run Terraform jobs.

Archive Open edX instances

[AutoDeploy][Archive] <INSTANCE_NAME>,<INSTANCE_NAME>,...

Archives Open edX instances. A list of comma-separated instance names can be supplied.

Update instances in batch

[AutoDeploy][Batch][Upgrade] <EDX_PLATFORM_REPOSITORY|EDX_PLATFORM_VERSION>,...

Builds images for all instances that match with provided <EDX_PLATFORM_REPOSITORY|EDX_PLATFORM_VERSION> in the cluster and redeploys them. These EDX_PLATFORM_REPOSITORY and EDX_PLATFORM_VERSION will be matched against the instance's config.yml value. It will first upgrade base images and then upgrade the instances. The pipeline generator should check grove.yml if BATCH_UPGRADE_DISABLED is set to True and ignore that instance in the pipeline.

Update instances in batch to particular commit hash

[AutoDeploy][Batch][UpgradeTo] <COMMIT_HASH>

Same as the previous one, except we can specify a commit hash. This is useful to roll back/upgrade instances to a particular commit hash.