Volume scheduling

A volume is not created on a node you pick. The scheduler picks one: given the size, cache and QoS the claim asks for, it finds a pool, a node in that pool and a disk on that node with room for it, and places the volume there. Everything below is what "with room for it" means, and how to steer the choice.

With a single pool there is not much to steer — this page starts to matter when there is more than one, or when nodes are not interchangeable.

#How a volume is placed

Candidates are the pools joined to their READY nodes. A node is out of the running if it is draining, and a pool is out if the selectors do not match it. Of what is left, a candidate must still have room:

Guard	What it checks
Volume count	The volumes already on the node, plus this one, stay under `max_volumes_count` — a per-node cap.
Write cache	The write cache already committed, plus this volume's, stays inside `rw_cache_size_in_mib` across the node's disks.
Read cache	The same for `r_cache_size_in_mib`.

The cache budgets are why a pool stops accepting volumes long before its buckets are full: capacity is S3's, but the cache is finite and every volume reserves a slice of it. See Caching and tiering for how those per-volume slices are sized.

#Spread or pack

load_strategy on the scheduler config decides which surviving candidate wins:

LeastAllocated (default) — the node holding the fewest volumes. Spreads load, which is what you want when volumes are alike.
MostAllocated — the node holding the most. Packs volumes together and leaves whole nodes empty, which is what you want before scaling a pool down.

#Selectors

Three optional selectors narrow the candidate set before the strategy is applied. Each is matched against the labels or buckets already on the object:

Selector	Matches
`pool_selector`	The pool's `labels` — the `labels` you set in Terraform, e.g. `env=prod`.
`node_selector`	A node's labels.
`bucket_selector`	The pool's S3 data bucket. With one bucket per pool this is a second way of naming the pool, and it is also what sets the bucket the volume's blocks are written to.

Pool labels are the usual lever: label pools by environment, hardware or tenant in the pool's Terraform, then write a scheduler config that selects them. They are set per pool as a comma-separated list — labels = "name=pool-1,env=dev" — and a pool with no labels matches only a config with no pool_selector.

Volume labels are the other half of it. The labels parameter on a StorageClass tags the volume, and the volume's labels choose which config is applied: the scheduler picks the enabled config that shares a key=value with them, falling back to the default one. So a class reaches a pool through a config — see Binding a class to a pool.

#Draining a node

Draining takes a node out of the candidate set and empties it: its volumes are rescheduled onto the survivors and its cache array is handed to a peer. It is the way you free a node before replacing or resizing it:

text

scheduler node drain <node>
scheduler node uncordon <node>

Drain marks the node system_drain=yes and the scheduler skips it from then on; uncordon removes the mark and the node reclaims its array. Both are ordinary CLI operations, and both are idempotent. What it costs and how to sequence it is in Pool node management; the same machinery runs unattended when a node fails, which is Failover.

#The config itself

One scheduler config is the default, and that is the one volumes are placed by unless their labels match another. It is created and inspected with the CLI:

text

scheduler config list
scheduler config show --name <name>
scheduler config add -h

Field
`enabled`	Whether the config is in use.
`default`	Mark it the one to schedule with.
`load_strategy`	`LeastAllocated` (default) or `MostAllocated`.
`pool_selector` · `node_selector` · `bucket_selector`	The filters above.
`labels`	Labels on the config itself.