bbctl (Installer)
bbctl deploys and operates the storage cluster in your AWS account. It is a
single binary: it downloads a pinned Terraform and the
terraform-aws-mgx modules,
renders a stack for the network, each pool and the optional management plane
from one config file, and applies them in order.
Overview
| Command | What it does |
|---|---|
bbctl init | Download Terraform and the deployment modules. |
bbctl config create · list · delete | Create, list and delete storage configs. |
bbctl plan · bbctl apply | Show what apply would change; create or update the deployment. |
bbctl status | Each stack's outputs and its nodes' EC2, SSM and provisioning state. --wait blocks until provisioning finishes. |
bbctl pool add · list · remove | Add, list or remove the storage pools in a config. |
bbctl secrets generate · set · get | Manage the node secret in SSM Parameter Store. |
bbctl sc | Print the helm install for the CSI driver against this storage. |
bbctl cli · bbctl connect · bbctl exec | Open mgx-cli on a node, open a shell or forward a port, or run a non-interactive command, all over SSM. |
bbctl ui | Open a dashboard of the storage's pools in the browser. |
bbctl destroy | Destroy the deployment: mgmt, pools, then network. --purge also deletes the node secret and the state. |
bbctl version | Print bbctl and the pinned tool versions. |
Every command that uses a config takes its name first, e.g. bbctl apply dev.
Storage configuration
<name>.bbctl.yaml describes one storage cluster: its network, its pools and,
optionally, a management plane in front of them. bbctl config create writes
it (see Install the storage cluster),
and every other command reads it. Edit it before the first bbctl apply, or
afterwards to change a deployment, then re-apply. Several configs can sit side
by side in one directory; each one's name prefixes its resources and its state,
so storages can share an account.
Example
A single pool of three nodes, as bbctl config create dev writes it:
name: dev
region: us-east-1
cluster: main
state:
bucket: dev-tfstate-123456789012-us-east-1
modules:
ref: main
nodes_ami: ami-0a5b4fc05dc8ee0d1
secrets_ssm_path: /mgx/dev/secrets
network:
vpc_id: vpc-0abc
azs: [us-east-1a, us-east-1b, us-east-1c]
mgmt_subnet_cidrs: [172.31.96.0/20, 172.31.112.0/20, 172.31.128.0/20]
storage_subnet_cidrs: [172.31.144.0/20, 172.31.160.0/20, 172.31.176.0/20]
nat_public_subnet_id: subnet-0def
pools:
- name: pool1
az: us-east-1a
description: Pool pool1
labels: name=pool1
nodes_count: 3
instance_type: c6gn.4xlarge
raid_level: 0
max_volumes_count: 10
r_cache_size_in_mib: 437737
rw_cache_size_in_mib: 131321
ebs_volumes:
- size: 460
type: gp3
iops: 3000
throughput: 125
count: 10
s3_bucket_names: [dev-123456789012-pool1-data]
s3_backup_bucket_names: [dev-123456789012-pool1-backup]
s3_force_destroy: true
enable_metrics: true
storage_class:
max_cache_r_cache_size: "476837"
max_cache_rw_cache_size: "143051"
ratio_cache_r_cache_size: "0.1"
ratio_cache_rw_cache_size: "0.03"
qos_r_mbytes_per_sec: "125"
qos_w_mbytes_per_sec: "125"
qos_rw_mbytes_per_sec: "125"
qos_rw_ios_per_sec: "3000"
Unknown fields are an error, so a typo fails the command instead of being ignored.
Top level
| Field | What it is |
|---|---|
name | The storage name from config create. It prefixes every resource and names the state, so two deployments can share an account. |
region · cluster | Where it runs, and the cluster the nodes join. Leave cluster as main: the CSI driver expects that name unless you change it there too. |
state.bucket | The S3 bucket for every stack's Terraform state. bbctl creates it on the first apply, after you approve. |
modules.ref | The git ref of terraform-aws-mgx to deploy. modules.path uses a local checkout instead. |
terraform.version · terraform.path | Optional. A Terraform version other than the pinned one, or an existing binary to use instead of downloading one. |
nodes_ami | The published node image for the region, shared by every node. Its architecture has to match every pool's instance_type. |
secrets_ssm_path | The SecureString the nodes read their secrets from. See Node secret. |
network
| Field | What it is |
|---|---|
vpc_id | The existing VPC to deploy into. |
azs | The AZs the network builds subnets in: one mgmt and one storage subnet each. |
mgmt_subnet_cidrs · storage_subnet_cidrs | One free CIDR per AZ, in the same order as azs: the control-traffic subnet and the volume-traffic subnet. |
nat_public_subnet_id | An existing public subnet. It hosts the NAT gateway, which gives the nodes their route to SSM and S3. |
ssh_public_key_path | Optional. A public key to register as the nodes' EC2 key pair. Empty means bbctl generates one under .bbctl/<name>/. |
bastion | Optional. enable: true, an Ubuntu ami for your region, an instance_type and whitelist_ips restricted to your own CIDR. |
Nodes have no public IP, and none is needed: they are provisioned through SSM
Run Command, and you reach them through Session Manager. Add a bastion only if
you explicitly want SSH access.
azs is a network-level list. Each pool pins itself to one of them with its
own az, because EBS cache volumes are AZ-bound. Pools can be spread across
zones, but a pool does not straddle them.
pools
config create sizes each pool with the calculator. Bucket
names are already globally unique: they include the storage name and the
account ID.
To size the pools, bbctl sends the workload (volume count, size, throughput,
IOPS) to https://backedblock.io/api/estimate. We keep a record of the
workloads priced, without your IP address or anything else that identifies
you or your account.
| Field | What to set |
|---|---|
name | Name of the pool, and the prefix on its resources. |
description | Free text, shown against the pool in the API and the CLI. |
labels | Comma-separated key=value pairs on the pool, e.g. env=dev. A scheduler config's pool_selector matches them, so they're how volumes get steered to one pool rather than another. See Volume scheduling. |
az | The single AZ the pool sits in, one of the network's azs. |
instance_type · nodes_count | From the calculator. A node runs the NVMe-oF target, the cache and the S3 client, so the instance type sets how much volume traffic it can carry. Its architecture has to match nodes_ami. See Storage compute. |
ebs_volumes · raid_level | From the calculator. The cache disks on each node. With raid_level: 0 they are striped RAID0, and bbctl derives the disk count from this list. See Caching and tiering. |
r_cache_size_in_mib · rw_cache_size_in_mib | From the calculator. Per-disk cache sizes; they're what give a volume its latency. See Caching and tiering. |
max_volumes_count | From the calculator. The volume cap per node, which the scheduler checks along with the cache budgets before placing a volume. |
s3_bucket_names · s3_backup_bucket_names | Globally unique bucket names: one for block data, one for snapshots. |
s3_bucket_access_names | Other pools' buckets this pool may read, e.g. to restore their snapshots. See Volumes. |
s3_force_destroy | Lets bbctl destroy delete the buckets with the data still in them. |
enable_metrics | Runs Prometheus on the nodes, collecting from core and its plugins. |
storage_class | From the calculator. The pool's StorageClass parameters: cache sizes and QoS caps. bbctl sc passes them to the driver's chart; the modules don't use them. |
extra | Any other variable of the pool module, passed through as-is: cache_filesystem: xfs, scheduler_node_down_wait: 120, the qos_* and snapshot_* settings. |
mgmt
The management plane. Leave it out for independent pools; config create --mgmt
or bbctl pool add --with-mgmt adds it.
| Field | What to set |
|---|---|
nodes_count · instance_type | Three t4g.xlarge by default. The management plane carries no volume traffic. |
enable_metrics | On: every pool's metrics are federated here. |
extra | Any other variable of the mgmt module, passed through as-is. |
Node secret
bbctl apply creates it on the first run, once you approve it: a random value
for every key, in the SSM SecureString at secrets_ssm_path. To create it
beforehand, run bbctl secrets generate dev. The nodes fetch it themselves
during provisioning, so it never enters Terraform state or a file on disk. An
existing secret is left alone. Set a single key with
bbctl secrets set dev KEY=value; read one back with bbctl secrets get dev KEY.
| Variable | What it is |
|---|---|
CASS_USER · CASS_PASSWD | The Cassandra superuser the nodes create and then authenticate with. Internal to the cluster: nothing outside it uses these. |
MGX_GW_X_API_KEY | The API key the management API accepts in the X-API-KEY header. |
MGX_X_API_KEY | The same idea for a node's own API. |
MGX_GW_ADMIN_PASSWD | Password for the admin account on the API. You'll need this one again: the CSI driver authenticates with it, and so does the CLI. bbctl secrets get dev MGX_GW_ADMIN_PASSWD prints it. |