Skip to main content

tile_concurrency

Function tile_concurrency 

Source
pub fn tile_concurrency(cfg: &TileConfig, cost: &TileCost) -> usize
Expand description

How many tiles tiled_scatter keeps in flight at once.

Natively this is just the rayon pool: memory is cheap and tile-level parallelism scales far better than tract’s intra-op threading. On WASM the limit is memory, not cores. Every concurrent tile holds its own activations in the single shared heap, so how much of the 4 GB ceiling a tiled run occupies is set by the pool size, the patch size and the network together — and by nothing else, the volume included. Tiling a bigger volume adds tiles, not concurrent ones.

Measured in Chromium (cross-origin-isolated) before this cap existed, the wasm heap high-water tracked the per-patch arithmetic: xQSM at 64³ patches, 1.1–1.4 GB with a 4-thread pool and 2.6–3.4 GB with 14. Raising the core one step is what makes it dangerous: core: 96 (104³ patches) on a 4-thread pool peaked at 4.24 GB, 98.7% of the ceiling. It did not abort there. It finished one inference and then stopped making progress, never completing a second within 30 minutes — a stall with no error, which is harder to diagnose than a crash. Nothing in the library noticed (astewartau/QSM.rs#89).

So on WASM the count is wasm_tiles_in_flight: whatever fits WASM_TILE_ACTIVATION_BUDGET at the network’s own cost per tile, and never more than the pool. The cost differs a lot between networks — a 64³ tile of BFRnet holds 3.4 times what one of iQSM does, and 1.8 times xQSM’s — so one network’s figure cannot stand in for the others.