Planning a run

Planning chooses a legal decomposition before the scheduler chooses a visit order. It records phase boundaries, block grids, types and source footprints. These choices determine which computations may be combined and which outputs must be stored; they cannot be revised silently halfway through a run.

Explicit assembly and automatic selection

PlanBuilder serves a caller who knows the intended pipeline. Its PlanBuilder::pixels appends one pixel phase containing the supplied Chain. The builder derives that chain’s required reach, folds its output type and records source-image dependencies. The operation requirements determine the halo; the caller does not need to restate a symmetric margin beside every filter constructor.

PlanBuilder::fragments and PlanBuilder::iterate represent different work shapes. They cannot be reduced to an arbitrary list of pixel kernels without losing fragment coverage or iteration boundaries. PlanBuilder::regrid changes the grid for subsequent phases, which is useful when a broad filter and a later pointwise stage have different useful block shapes.

For a caller who knows a chain but wants its materialisation points priced, PlanBuilder::partition delegates to a Strategy with explicit Constraints. That is a search over legal partitions of a recorded chain, not a general compiler that discovers arbitrary algebraic identities or rewrites every image DAG. Source leaves continue to refer to the original plan’s image identities.

flowchart TB
  A["Operations and source declarations"] --> B["Candidate partitions and block grids"]
  H["Host, budget and cost model"] --> B
  B --> C{"Geometry and types valid?"}
  C -- no --> X["Reject candidate"]
  C -- yes --> D{"One active block fits?"}
  D -- no --> X
  D -- yes --> E["Price useful concurrency, work and traffic"]
  E --> F["Choose and record decomposition"]
Figure 1: A candidate must satisfy contracts before its estimated completion time is useful.

What a candidate has to prove

The resulting Decomposition preserves operation order and declares image types. Its checks ensure that valid output regions tile the desired volume, source references name available images, and operation-specific block constraints hold. BlockConstraint can express restrictions that a halo alone cannot: an algorithm may need a complete axis or a minimum useful block extent. A shape-changing operation needs mapped geometry, not a copied same-grid halo.

An insufficient halo must invalidate the candidate rather than produce a faster answer at internal seams. A larger-than-required halo can remain correct while wasting reads and arithmetic. This asymmetry is why required reach is derived from the algorithm independently of the amount granted by a candidate.

Assembly packages the workflow, decomposition and phase work together. Its handles remove manual image-number bookkeeping, while the checks remain necessary: a plan can also arrive from another strategy or through a remote execution path.

The block-size tradeoff in numbers

Suppose an interior output block is a cube with edge 64 and a chain needs eight voxels on either side. Its input is an 80-cube: approximately 1.95 input voxels per output voxel. A 128-cube needs a 144-cube, approximately 1.42 per output. The larger block repeats less halo work, but each active task holds larger input, output and temporary buffers.

For an illustrative per-task charge of 300 MiB and a usable active-work budget of 1 GiB, at most three such tasks fit, even on an eight-worker host. A finer candidate charged at 120 MiB may run eight tasks at once but repeat more halo work. The decision is therefore predicted completion time under the feasible active set, rather than smallest traffic or highest worker count in isolation. These numbers illustrate arithmetic; they are not measured Blockflow timings.

strategy::choose_grid builds candidate grids and prices complete assemblies against Host and a CostModel. price_under_budget rejects a candidate when one block’s expected active charge exceeds the budget. It otherwise reduces the number of in-flight blocks to what can fit. That plan-selection rule differs from the executor’s progress-oriented fallback for a manually supplied oversized block, described under memory.

What is priced and what remains uncertain

Useful terms include fetched voxel counts, output traffic, arithmetic work, chain intermediate lifetimes, additional source buffers and declared fragment scratch. BlockLadder controls candidate granularity in partitioning. Tile alignment can change a candidate’s effective reach, so a block edge divisible by a kernel’s fixed tile size can be cheaper than a nearby unaligned edge.

The model also considers the cost of halo rereads when the active set leaves little room for cache retention. This is a model of likely reuse, not evidence that a particular storage backend will cache all those reads. Compression, filesystem cache state, kernel implementation and current machine load can change the ranking.

Fragment cardinality and convergence depth are especially difficult before reading data. A component-equivalence reduction can grow with the segmentation; an iterative image phase is priced per substage without knowing the final count. Treat estimates as candidate-selection evidence and verify representative runs with progress events and release-build measurements.

The scheduler can adjust concurrency and ready-task order while honoring the recorded decomposition. It cannot repair a wrong required reach or invent a new phase partition. This is the boundary between planning and execution policy.

For a decision-by-decision account of explicit assembly, opt-in searches and runtime defaults, see what Blockflow chooses automatically. The execution story follows an explicit grid through source chunk reads and output writes.