Skip to content

minikube

Grove minikube support is under active development. The main use case for this provider is local development.

Installation

These steps imply that in the end you'll have a Kubernetes cluster running, along with five additional processes. To make them easier to manage, you might want to create a tmux session with five windows:

  • kubectl proxy
  • localstack
  • kubectl proxy tunnel
  • localstack tunnel
  • minikube tunnel

minikube

Due to this issue, we can't use Kubernetes 1.24 with minikube currently.

Install minikube, and run the following command to create a Kubernetes cluster:

minikube start --kubernetes-version=v1.23.8

Run the following to ensure that the cluster is up and running:

minikube kubectl -- get pods -n kube-system

kubectl proxy

Run minikube kubectl -- proxy to create a proxy to the Kubernetes API server.

We need this to simplify provider setup by bypassing security checks that would normally be needed if we use minikube ip to access the API server.

Run curl http://127.0.0.1:8001/version to verify that the proxy is working.

LocalStack

Grove needs an S3-compatible cloud storage for the Tutor environment.

minikube provider is configured to work with LocalStack, a drop-in replacement of many AWS services, including S3.

Install it and run (you can use pipx for the easy virtualenv management):

pipx install localstack
localstack start

Run the following to verify that LocalStack is working:

AWS_ACCESS_KEY_ID=mock_access_key AWS_SECRET_ACCESS_KEY=mock_secret_key aws --endpoint-url=http://127.0.0.1:4566 s3 ls

You may not have buckets yet, but the command shouldn't error out.

kubectl proxy tunnel

To access the Kubernetes API server from the GitLab workers, we need to create a tunnel to the kubectl proxy.

Install cloudflared, and run the following:

cloudflared tunnel --url http://127.0.0.1:8001 --http-host-header 127.0.0.1

Once all tunnel connections are registered, copy the tunnel URL (https://*.trycloudflare.com), and verify that the tunnel is working: curl <TUNNEL_HOST>/version. This is the URL you need to use to set up grove-template.

Note 1: You may encounter Unauthorized: Failed to get tunnel errors when creating a tunnel. Just wait for a few retries, and it'll succeed. Note 2: it's possible to use any tunneling solution that allows the host header overriding.

LocalStack tunnel

Similarly to the previous section, create a tunnel to LocalStack, to make GitLab worker able to save the Tutor environment in it:

cloudflared tunnel --url http://127.0.0.1:4566

Run the following to verify that the tunnel is working:

AWS_ACCESS_KEY_ID=mock_access_key AWS_SECRET_ACCESS_KEY=mock_secret_key aws --endpoint-url=<TUNNEL_URL> s3 ls

You may not have buckets yet, but the command shouldn't error out.

minikube tunnel

Run minikube tunnel to create a route to services deployed with type LoadBalancer. This is required by ingress-nginx.

You should see similar output when the ingress controller is installed:

Status:
        machine: minikube
        pid: 1303594
        route: 10.96.0.0/12 -> 192.168.49.2
        minikube: Running
        services: [ingress-nginx-controller]
    errors:
                minikube: no errors
                router: no errors
                loadbalancer emulator: no errors

LMS/CMS tunnel

Similar to the other sections, create a tunnel to LMS/CMS to make your LMS/CMS available on the wider internet.

cloudflared tunnel --url http://127.0.0.1:8002

Then set either LMS_HOST or CMS_HOST to the generated TUNNEL_HOST in your instance's config.yml. At this time, it's not possible to have subdomains on CloudFlare tunnels, so you can only use one at a time.

After you've deployed your LMS, forward your ingress to port 8002 with:

kubectl port-forward -nkube-system svc/ingress-nginx-controller 8002:80

And visit the TUNNEL_HOST in your browser to see your LMS/CMS.