Installing tutor plugins per instance¶
In this document we consider the effort and implications required for each Grove instance to have it's own set of tutor plugins.
At OpenCraft we expect to deploy multiple instances on a single cluster. These instances will have different
requirements for their tutor configuration. The plugin list could be different, while
it might also be needed to run different versions of
tutor (especially in the case of upgrading between releases).
eduNEXT created called Tutor Version Manager for managing
tutor and Python environments. It is well suited to the task.
Unfortunately, there are architectural decisions that preclude it's inclusion into Grove.
- It has to be installed globally. While unlikely, it's possible that different versions will be needed per instance, which is not possible.
- The app state is stored globally under a
.tvmdirectory. This means that deleting an instance will be more complicated as Grove will need to run the requisite
tvmcommands when deleting an instance. In the unlikely event that the
.tvmdirectory is corrupted, all instances are affected.
- It's an extra dependency and dependencies are bad (not really, but we try to limit them where possible).
Python3 has built in virtual environments via the venv module. By utilising this module to store virtual environments in each instance directory, it becomes possible to have completely separate Grove instances from a Python point of view.
Within an instance's directory, we'll create a
.venv directory which will house the virtual environment. By default, we
tutor, it's related plugins and the tutor-grove plugin as they
are required for building an instance.
Any command within Grove that calls
tutor will need to be changed to point to the instance specific directory. Since the
needs to be managed, it is required to create
grove-cli commands to manage it. The
.venv folder will not be committed to
Since each instance, will have it's own list of requirements, we envision that a
grove-requirements.txt will be created
within each instance directory as well. This file will be tracked in
Technical Details: How to implement this¶
Add Grove CLI commands¶
We will create commands to manage the virtual environment under the
./grove venv update [instance-name]=> Updates the virtual environment and installs the packages in
grove-requirements.txt. It the virtual environmenth doesn't exist, it is created.
./grove venv reset [instance-name]=> Delete and recreates the virtual environment. In cases where the virtual environment becomes invalid somehow (eg. the user installed incompatible packages), this command will recreate it.
./grove venv activate [instance-name]=> Starts a shell in
dockerwith the relevant environment activated.
tutor invocations to use the virtual environment¶
tutor is invoked in various places in Grove. We will need to replace them with the equivalent virtual environment command.
Please note that the above is not an exhaustive list. It will be up to the implementor to find all instances.
Update pipelines to build and use the virtual environment¶
tutor's running within a virtual environment, we will need to make sure this virtual environment
exists when running deployment pipelines. For the create, update and delete pipelines we will need to build the virtual environment.
Since multiple images can be built in a single deploy, creating the virtual environment at the time of building the image isn't efficient. For this reason, we suggest adding an extra pipeline step to build the virtual environment, which will be shared to any later pipelines.
tutor and it's plugins from Grove¶
At this point,
tutor can be remove from Grove as their is no longer a direct dependency. Remove all
requirements from the
tools-container itself and add them to the
defaults directory in grove-template.