Commit Based Deployment Pipelines¶
- Deploying infrastructure.
- Deploying Open edX instances.
- Performing mass upgrades.
- Archiving instance
Currently, we are generating our deployment pipeline with the generate-pipelines.py script. We are generating them once we add a commit to the default branch. Instance deployments need manual confirmation.
There is an existing format for auto-deployment commit messages.
[AutoDeploy] $INSTANCE_NAME:$DEPLOYMENT_REQUEST_ID - $MESSAGE
The value of
$MESSAGE can be either of
deploy new instance :rocket: - when creating a new instance or
apply changes :tools: - updating an existing instance.
$INSTANCE_NAME is self-explanatory and
$DEPLOYMENT_REQUEST_ID is provided by the GitLab trigger client to track the job.
But regardless of the message, currently, generate-pipelines.py generates the same pipeline for every commit.
Since Grove manages multiple instances in a Kubernetes cluster and the infrastructure for the cluster, we want to have fine control over what deployment jobs are running and when. We don't want to run all jobs every time there is a new commit.
A pipeline generation based on commit messages would help up achieve that and open room for the optimized pipeline in different situations.
We propose a new format for deployment commits:
We will only perform deployment-related CI/CD jobs if the commit title has the
[AutoDeploy] tag in it.
So for different situations, we will have the following commit messages:
Deploy infrastructure using Terraform¶
[AutoDeploy][Infrastructure] <any message here>
Create and deploy a new Open edX instance¶
<INSTANCE_NAME> is the instance name,
<DEPLOYMENT_ID> is an optional client-provided deployment id. This will first deploy infrastructure using terraform and then deploy the instance. The pipeline will only contain jobs related to provided
<INSTANCE_NAME> list. Other instances in the repository will be ignored from the pipeline.
Update an Open edX instance¶
[AutoDeploy][Create] except it will not run Terraform jobs.
Archive Open edX instances¶
Archives Open edX instances. A list of comma-separated instance names can be supplied.
Update instances in batch¶
Builds images for all instances that match with provided
<EDX_PLATFORM_REPOSITORY|EDX_PLATFORM_VERSION> in the cluster and redeploys them. These
EDX_PLATFORM_VERSION will be matched against the instance's
config.yml value. It will first upgrade base images and then upgrade the instances. The pipeline generator should check
BATCH_UPGRADE_DISABLED is set to
True and ignore that instance in the pipeline.
Update instances in batch to particular commit hash¶
Same as the previous one, except we can specify a commit hash. This is useful to roll back/upgrade instances to a particular commit hash.