Design overview

Blockflow turns an image algorithm into work that can run over blocks without making internal block edges part of the scientific result. Understanding that translation requires three separate questions: what an operation reads, how the plan partitions that computation, and how the executor schedules the resulting work.

flowchart TB
  A["Algorithm, types and required source regions"] --> B["Plan: phases, grids and valid outputs"]
  B --> C["Task graph and barrier gates"]
  C --> D["Dynamic scheduling and resource admission"]
  D --> E["Committed image cores and diagnostics"]
Figure 1: Algorithm requirements become binding geometry; execution policy chooses among legal schedules.

Three layers of responsibility

An operation states a dependency contract. A local filter expresses it through BlockOp; a computation producing rows or partial measurements uses FragmentOp; a repeated state update may use IterativeOp. These interfaces share execution infrastructure, but their different outputs and synchronization requirements remain explicit.

A PlanBuilder records a composition and yields an Assembly. Its Decomposition records phase geometry, image types, source references and the partition into tasks. The geometry is binding: an executor cannot silently change a halo, reorder operations or skip a required source to make a run fit.

The executor receives performance preferences through Hints. It can choose ready-task order and concurrency within the recorded dependencies. A poor choice can increase I/O or idle workers; it must not change the owned result. This separation is the central reason scheduling experiments can share one correctness model.

Read the design by question

Question Explanation
What do images, phases and tasks represent? Data model
How do channels, time, series and pyramid levels fit together? Multidimensional acquisitions
What happens from a stored chunk to a finished output? One complete execution story
Which decisions are automatic and which must I configure? Automatic choices
Which neighboring data must a block receive? Dependencies and halos
Who chooses phase boundaries and block grids? Planning
When does composing operations save storage? Fusion
When can a task start, and why might it wait? Scheduler
Which allocations count toward the working set? Memory and storage
How can a global result be computed without a global image read? Global dependencies
What happens between repeated local updates? Iteration
How do CPU, GPU and multiple machines differ? Parallel execution
Which properties must an operation author prove? Correctness contracts

Scaling is a property of the algorithm and its route

An algorithm implemented in a block-processing library is not automatically bounded in memory. A whole-axis solve may be bounded by a line, a reduction may retain one entry per object, and the general iterative image route currently holds two whole-volume state buffers. The separate finite-PSF deconvolution solver uses region reads and disk-backed state instead.

The design pages distinguish these cases. Use the operation catalog to choose an actual execution route, and the example walkthroughs to see storage and metadata handling around it. Detailed historical investigations remain in the design records; these pages explain the current mechanisms rather than reproducing the order in which they were built.