Single pool mode

The smallest thing that serves a volume: one pool, no management plane. Good for a lab, a proof of concept, or a first volume before committing to a layout.

Two Terraform stacks, applied in order — network/ first, then pool/, which reads the network's outputs from its state file.

#Step 1 — Clone

git clone https://github.com/migrx-io/backedblock.git
cd backedblock/single-pool

#Step 2 — Size the pool

Volumes first, machines second. Sizing the pool turns the volumes you intend to serve into the two things this deployment asks for: the pool block for Step 4, and the StorageClass parameters for the driver install afterwards. Keep both to hand.

#Step 3 — Configure the storage network

Each stack is configured by editing the module block in its own main.tf — here, network/main.tf.

What to set in network/main.tf

Variable	What to set
`vpc_id`	The existing VPC to deploy into — normally the one your workloads already run in.
`azs`	The availability zones to build subnets in.
`mgmt_subnet_cidrs` · `storage_subnet_cidrs`	One free CIDR per AZ, in the same order as `azs`: the control-traffic subnet and the volume-traffic subnet.
`bastion.vpc_subnet`	An existing public subnet. It hosts the bastion and the NAT gateway.
`bastion.ami`	An Ubuntu image for the bastion, valid in your region.
`bastion.whitelist_ips`	Who may ssh to the bastion. It ships as `0.0.0.0/0` — narrow it to your own CIDR before applying.
`key_name` · `ssh_public_key_path`	The key pair Terraform registers and then uses to reach the nodes.

The bastion is required in this layout: nodes have no public IP, and SSH through the bastion is how Terraform provisions them.

#Step 4 — Configure the storage pool

In pool/main.tf, paste in what Sizing the pool produced — nodes_count, nodes_instance_type, nvme_node_disks_count, max_volumes_count, the two cache sizes and the ebs_volumes block. Beyond those, only the names need attention: pool_name, the two s3_bucket_names lists, az, and the nodes_ami for your region. Everything else ships with a working default.

Every variable in pool/main.tf

Variable	What to set
`cluster`	Leave it as `main`. It names the cluster the nodes join, and the CSI driver expects that name unless you change it there too.
`pool_name`	Name of the pool, and the prefix on its resources.
`description`	Free text, shown against the pool in the API and the CLI.
`labels`	Comma-separated `key=value` pairs on the pool, e.g. `env=dev`. They are what a scheduler config's `pool_selector` matches, so they only start to matter once there is more than one pool — see Volume scheduling.
`az`	The single AZ the pool sits in. It must be one of the network's `azs` — EBS cache volumes are AZ-bound, so a pool does not straddle zones.
`nodes_ami`	The published node image for your region — see Node AMIs.
`nodes_instance_type` · `nodes_count`	From Step 2. A node runs the NVMe-oF target, the cache and the S3 client, so the instance type sets how much volume traffic it can carry, and the architecture has to match the AMI — see Storage compute.
`ebs_volumes` · `nvme_node_disks_count`	From Step 2. The cache disks on each node: with `raid_level = 0` they are striped RAID0, and `nvme_node_disks_count` must equal the total `ebs_volumes` count — see Caching and tiering.
`r_cache_size_in_mib` · `rw_cache_size_in_mib`	From Step 2. Per-disk cache sizes; this is what gives a volume its latency — see Caching and tiering.
`max_volumes_count`	From Step 2. The volume cap per node, which the scheduler checks along with the cache budgets before placing a volume.
`s3_bucket_names` · `s3_backup_bucket_names`	Globally unique bucket names — block data, and snapshots.
`enable_metrics`	Runs Prometheus on the nodes, collecting from core and its plugins.
`enable_grafana`	Adds Grafana on the cluster's VIP node, reachable through the bastion. Needs `enable_metrics`.
`cross_peer_scrape`	Leave it `true` here. Every node scrapes each of its peers, so any node holds a full copy of the pool's metrics — the right setting for a pool with no management plane in front of it. A pool attached to one sets it `false` and lets mgmt federate instead.
`ssh_user` · `ssh_private_key_path`	The account and key Terraform uses on the nodes at provision time.

#Step 5 — Create the secret

cd pool
cp ../secrets.env.example secrets.env    # then fill in real values
cd ..

Every line is KEY=value and every one has to be filled — the placeholders are not defaults, and a node that reads an unset value will come up unusable.

What each secret is

Variable	What it is
`CASS_USER` · `CASS_PASSWD`	The Cassandra superuser the nodes create and then authenticate with. Internal to the cluster — nothing outside it uses these.
`MGX_GW_X_API_KEY`	The API key the management API accepts in the `X-API-KEY` header.
`MGX_X_API_KEY`	The same idea for a node's own API. It does not have to differ from the gateway key, which is why the example carries one placeholder for both.
`MGX_GW_ADMIN_PASSWD`	Password for the `admin` account on the API. This is the one you need again later — it is what the CSI driver authenticates with, and what the CLI logs in with.
`GRAFANA_USER` · `GRAFANA_PASSWD`	Grafana's admin login, used only when `enable_metrics` and `enable_grafana` are on.

#Step 6 — Apply

Network first — the pool stack reads its outputs from that state file:

(cd network && terraform init && terraform apply)
(cd pool    && terraform init && terraform apply)

Tear down in reverse: (cd pool && terraform destroy) then (cd network && terraform destroy).

#Step 7 — Connect to the cluster

Nodes have no public IP, so SSH goes through the bastion. Both addresses come from the stacks:

(cd network && terraform output bastion_public_ip)
(cd pool    && terraform output node_mgmt_private_ips)

mgx-cli is already on PATH on every node, and any node can drive the whole cluster:

ssh -J ubuntu@<bastion_public_ip> ubuntu@<node_mgmt_private_ip>
mgx-cli

text

mgx-core:127.0.0.1:nologin> login admin --cluster main --ns main
Password:
 logged..

The password is the MGX_GW_ADMIN_PASSWD from Step 5. Two commands say whether the deployment worked.

The main cluster with every node in it, and which one currently holds the VIP — the node running the cluster-wide work, and the one Grafana lands on in the next step:

text

mgx-core:127.0.0.1:main:main:admin> cluster list

The pool, whose status has to read READY. PENDING means it is still forming, so give it a moment and run it again:

text

mgx-core:127.0.0.1:main:main:admin> cache pool list

#Step 8 (Optional) — Grafana

Metrics are off unless the pool asked for them, so this step exists only if Step 4 set both:

hcl

enable_metrics = true
enable_grafana = true

Grafana then runs on the cluster's VIP node — the node cluster list marked in Step 7 — on Grafana's own port 3000, with the GRAFANA_USER and GRAFANA_PASSWD from Step 5. It has no public address either, so reach it by forwarding the port over the bastion:

ssh -N -J ubuntu@<bastion_public_ip> -L 3000:localhost:3000 ubuntu@<vip_node_ip>

and open http://localhost:3000. Two dashboards matter:

Dashboard	What it is for
Cache	The state of the cache nodes — one row per node, and on a healthy pool every one of them is green. This is the dashboard to open first: a node that is not green here is a node the scheduler will stop placing on.
Storage	The volumes themselves — what is provisioned, and each volume's metrics. This is where a volume's throughput, IOPS and cache behaviour show up once workloads are on it.

#Step 9 — Point the driver at the pool

The CSI driver points at the pool nodes themselves, since there is no management plane in this layout:

(cd pool && terraform output node_mgmt_private_ips)

Each of those on port 8082 is an API endpoint for csiSecret.clusterConfig.nodes — carry them into Install the K8s CSI driver, with admin and MGX_GW_ADMIN_PASSWD as the credentials.

Growing past one pool is a fresh deployment in the other layout rather than a migration — see Multi pool mode.