Install the CSI driver

From nothing to a mounted volume: install the CSI driver against a running storage cluster, provision a PVC, mount it in a pod.

This is the Kubernetes half, and it assumes the storage cluster already exists. If it does not, start at Install the storage cluster — that page ends with the four values the Helm install below needs.

Prerequisites

Step 1 — Install the snapshot CRDs

Skip this if the cluster already has them.

sh
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.2.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.2.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.2.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml

If the cluster has no snapshot controller of its own, let the chart deploy one with --set externalSnapshotter.enabled=true below. A single snapshot controller serves every CSI driver in a cluster, so do not add a second.

Step 2 — Install the driver

Two things go into this command: the cluster connection from Install the storage cluster, and the StorageClass parameters Sizing the pool printed — the cache ratios, the ceilings and the QoS caps, worked out from the volumes the pool has to serve.

bbctl prints the command for a cluster it deployed, for example for the dev config:

sh
bbctl sc dev

The output looks like this:

sh
helm install mgx-csi-driver oci://docker.io/migrx/mgx-csi-driver \
  --namespace mgx-system --create-namespace \
  --version 0.1.0 \
  --set csiSecret.clusterConfig.nodes='{172.31.96.14:8082,172.31.96.15:8082,172.31.96.16:8082}' \
  --set csiSecret.clusterConfig.username=admin \
  --set csiSecret.clusterConfig.password=secret \
  --set storageclass.ratioCacheRCacheSize=0.1 \
  --set storageclass.ratioCacheRWCacheSize=0.03 \
  --set storageclass.maxCacheRCacheSize=476837 \
  --set storageclass.maxCacheRWCacheSize=143051 \
  --set storageclass.qosRMBPerSec=125 \
  --set storageclass.qosWMBPerSec=125 \
  --set storageclass.qosRWMBPerSec=125 \
  --set storageclass.qosRWIOPS=3000 \
  --set externalSnapshotter.enabled=true

nodes is every management API endpoint of the storage cluster as <ip>:8082 — list all of them, the driver fails over between them — and the credentials are its admin user.

The StorageClass parameters from the calculator

Every value in the calculator's second block, and the Helm flag it goes in as. The parameter names are what a hand-written StorageClass uses instead — see StorageClass parameters.

Sizing outputHelm valueWhat it is
ratio_cache_r_cache_sizestorageclass.ratioCacheRCacheSizeRead cache as a ratio of volume size. This is the volume's whole footprint on the cache disks: it serves reads and writes alike.
ratio_cache_rw_cache_sizestorageclass.ratioCacheRWCacheSizeHow much of that footprint may be dirty and waiting to drain — 30% of the read ratio, so the larger share of the cache stays clean for reads.
max_cache_r_cache_sizestorageclass.maxCacheRCacheSizeRead-cache ceiling, MiB. The cache is clamp(ratio × volume size, floor, ceiling), so this is what a volume at the sized size actually gets.
max_cache_rw_cache_sizestorageclass.maxCacheRWCacheSizeWrite-cache ceiling, MiB — inside the read cache, not beside it.
qos_r_mbytes_per_secstorageclass.qosRMBPerSecRead bandwidth cap per volume, MB/s — the read figure the pool was sized for.
qos_w_mbytes_per_secstorageclass.qosWMBPerSecWrite bandwidth cap per volume, MB/s.
qos_rw_mbytes_per_secstorageclass.qosRWMBPerSecCombined read+write bandwidth cap per volume, MB/s. This is the peak either direction reaches, not their sum — the two caps above are not added up here.
qos_rw_ios_per_secstorageclass.qosRWIOPSCombined read+write IOPS cap per volume, the per-volume IOPS the pool was sized for. There is no per-direction IOPS cap.

These have to match the pool they point at. The ceilings are what the pool's per-disk r_cache_size_in_mib and rw_cache_size_in_mib budgets were sized from, so a class with larger ones places fewer volumes per node than the sizing assumed, and the scheduler starts rejecting volumes earlier than expected — see Caching and tiering.

Nothing here is required: leave a parameter out and the chart's default applies, and an unset QoS cap is omitted from the class entirely rather than sent as a zero. The rest of the chart's values are on the Helm values page.

Step 3 — Create a test pod with a volume

A PVC and a pod that mounts it, in one file — test-pod.yaml. Nothing about the volume is special at the pod level:

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: mgxcsi-sc
  resources:
    requests:
      storage: 1Ti
---
apiVersion: v1
kind: Pod
metadata:
  name: mgx-test
spec:
  containers:
    - name: app
      image: busybox
      command: ["sleep", "infinity"]
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: data
sh
kubectl apply -f test-pod.yaml
kubectl get pvc data -w                    # waits for Bound
kubectl wait --for=condition=Ready pod/mgx-test --timeout=5m

Next