Module synthseg 

SynthSeg whole-brain segmentation (onnx feature).

SynthSeg is a 3D U-Net trained purely on synthetic images sampled from anatomical label maps with fully randomised contrast and resolution. That domain randomisation is what makes it contrast-agnostic: one set of weights segments any MR contrast without retraining, which is why it is the whole-brain parcellation that can be run directly on a GRE magnitude image rather than on a separately acquired T1w scan.

This module is a port of the non-robust path of SynthSeg/predict_synthseg.py around the exported network (synthseg.onnx, see crate::models):

1. resample to 1 mm isotropic (anti-aliased, linear) if the input is not already there;
2. permute and flip the axes into RAS orientation using the NIfTI affine;
3. optionally centre-crop, then rescale the 0.5–99.5 percentile range to [0, 1];
4. pad to a multiple of 32 (at least 128) and run the network, giving one posterior map per label;
5. blur the posteriors, and average them with a left-right flipped second pass;
6. keep the largest connected component of the foreground and of each topological class, renormalise, take the argmax, and undo the crop and the orientation change.

Feed it magnitude, not susceptibility. On a χ map the white/grey boundary is faint and the brainstem is badly under-segmented (Dice ≈0.70 against the same subject’s magnitude segmentation); on the echo-combined GRE magnitude the structure volumes come out in the normal physiological range. For multi-echo data the root-sum-of-squares over echoes works.

The default whole-volume run reproduces FreeSurfer’s mri_synthseg (SynthSeg 2.0, non-robust): on three 1 mm scans (a 184×256×128 GRE magnitude, a 180×241×131 GRE magnitude and a 176×240×256 MP2RAGE), every label’s Dice against it is at least 0.9999 and the soft volumes agree to 0.03 % on average.

Memory is the practical constraint: the network takes about 0.95 KB per voxel of its input at once (2.2 GB at 128³, 6.0 GB on a 184×256×128 scan padded to 192×256×128, measured natively), more than a 32-bit WASM heap (4 GB) holds. The whole volume needed 6–12 GB on those scans, with or without the head crop and half-precision posteriors, because the network’s activations dominate; no whole-volume setting fits the browser. Three settings bound it:

• SynthSegInference::SlidingWindow runs the network on overlapping cubic windows whose posteriors are blended with a Gaussian weight, as nnU-Net does, so the network’s memory follows the window instead of the volume. This is what makes the browser possible, at an accuracy cost set by the window’s size (see SynthSegInference::LOW_MEMORY).
• SynthSegParams::head_crop runs everything (network, posteriors, post-processing) on the head’s bounding box plus a margin instead of the padded field of view.
• PosteriorPrecision::F16 keeps the posteriors (n_labels values per voxel, the one whole-volume buffer the windows cannot bound) at half precision.

SynthSegParams::low_memory combines them for the browser.

Reference pipeline

From the SynthSeg repository (BBillot/SynthSeg 2a2aa3b):

• Contrast and resolution. Any contrast, “any resolution up to 10mm slice spacing” (README.md:5-7); inputs are resampled to 1 mm (step 1). The reference returns its segmentation at 1 mm isotropic (README.md:105-106,122); this port maps it back onto the input grid (nearest neighbour), and on 1 mm scans matches mri_synthseg as above.
• Orientation. Needs the image’s affine to reorient to RAS (step 2); synthseg takes it as an argument (the NIfTI sform/qform, as carried by ScanMetadata::affine). It does not use the B0 direction.
• Robust model not ported. --robust (SynthSeg-robust) is the authors’ variant for scans with low SNR or tissue contrast, or clinical data with large slice spacing (README.md:67-70,96-97; scripts/commands/SynthSeg_predict.py:38). Only the non-robust network is ported, so thick-slice or low-SNR GRE magnitudes get the model the authors suggest replacing for them.
• Defaults. Non-robust, flip averaging and topology cleanup on (--fast turns both off), posterior smoothing 0.5, no crop: SynthSegParams::default matches.
• WASM (4 GiB heap). No whole-volume setting fits (see above); SynthSegParams::low_memory (96³ windows) does. 128³ windows fit only network grids up to about 192×256×128 (SynthSegInference::LOW_MEMORY); synthseg_network_grid gives the grid for a scan.

Reference: Billot, B., Greve, D.N., Puonti, O., et al. (2023). “SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining.” Medical Image Analysis, 86:102789. https://doi.org/10.1016/j.media.2023.102789

Reference implementation: https://github.com/BBillot/SynthSeg (Apache-2.0).

Pipeline rules

The facts above that a configuration can depart from are published as rules in crate::pipeline::rules::RULES (each with its strength, basis, reason and source): synthseg.wasm-windows, synthseg.wasm-128-window.

Structs

Preprocessed 🔒

The network input plus everything needed to map predictions back onto the caller’s grid.

SynthSegParams

SynthSeg inference parameters.

SynthSegResult

The outcome of a SynthSeg run.

Enums

PosteriorPrecision

Storage for the posteriors, n_labels values per voxel of the network’s grid.

SynthSegInference

How the network is applied to the preprocessed (1 mm, RAS, padded) volume.

Constants

DIVISOR 🔒

Network down-sampling factor: the padded input must be a multiple of this on every axis.

HEAD_THRESHOLD 🔒

Normalised intensity (the 0.5–99.5 percentile range mapped to [0, 1]) above which a voxel counts as head when SynthSegParams::head_crop looks for the head’s bounding box.

LATTICE 🔒

The U-Net’s max-pooling lattice: four 2× poolings, so a window must start on a multiple of 16 voxels from the whole-volume input’s origin to pool the same voxels together, and any input to the network must be a multiple of 16 on every axis.

MIN_PAD 🔒

Minimum padded size per axis, as in the SynthSeg CLI when --crop is not given.

TARGET_RES 🔒

Target resolution (mm).

Traits

Posterior 🔒

A stored posterior value: f32, or half::f16 for PosteriorPrecision::F16. Arithmetic is in f32 either way.

Functions

align_to_ras 🔒

Gather a column-major (nx, ny, nz) volume into a C-order RAS-oriented f32 buffer.

blur_along 🔒

Convolve data, viewed as [outer, n, inner], along the middle axis with zero padding.

blur_posteriors 🔒

Per-channel separable Gaussian blur of a [d0, d1, d2, n_ch] posterior map.

crop_c_order 🔒

Crop a C-order [d0, d1, d2] volume of n_ch-element voxels to bounds.

flip_axis0 🔒

Reverse axis 0 of a C-order [d0, d1, d2] volume of n_ch-element voxels.

head_box 🔒

The head’s bounding box in the normalised, padded C-order volume data: the largest face-connected region above HEAD_THRESHOLD, grown by margin voxels on every side and rounded out to multiples of LATTICE (dims are multiples of it already). None when nothing is above the threshold or the box is the whole volume.

infer 🔒

Network, flip averaging and post-processing, with the posteriors stored as T.

lattice_windows 🔒

Window origins along an axis of size voxels for windows of p (both multiples of LATTICE): the first at 0, the last at size − p, and the fewest windows whose steps — each a multiple of the lattice, as even as the lattice allows — leave at least overlap voxels shared between neighbours.

merge_flipped 🔒

Average the posteriors of the left-right flipped pass flipped (still mirrored along axis 0) into post, mapping each channel to its left-right partner table[c]. In place: the mirrored copy is never materialised.

pad_c_order 🔒

Zero-pad a C-order [d0, d1, d2] volume of n_ch-element voxels so the data lands at at.

percentile 🔒

np.percentile(sorted, q) with linear interpolation between order statistics.

postprocess 🔒

preprocess 🔒

ras_alignment 🔒

The axis permutation and per-axis flips that take a volume into RAS orientation.

resample_labels_nearest 🔒

Nearest-neighbour resample of a column-major label volume back onto the caller’s grid.

resample_to_iso 🔒

Resample a column-major volume to 1 mm isotropic, matching lab2im’s resample_volume: an anti-aliasing Gaussian when down-sampling, then linear interpolation on a half-pixel grid.

round_up 🔒

run_pass 🔒

run_windows 🔒

Sliding-window posteriors (channels-last, [d0, d1, d2, n_labels]) of the network input data (dims multiples of LATTICE): the network on each patch-cube window (clamped to the volume on an axis it does not exceed), origins on the pooling lattice (lattice_windows), blended with nnU-Net’s Gaussian importance. With flip (the label table’s left-right partners), each window is also run mirrored along axis 0 and its posteriors, mirrored back with each channel mapped to its partner, averaged in, as the whole-volume flip averaging does. The blur is the caller’s, once, on the blended posteriors.

synthseg

Whole-brain segmentation via SynthSeg.

synthseg_network_grid

The padded 1 mm grid SynthSeg’s network runs on for a scan on grid, in the scan’s voxel axis order: resampled to 1 mm (unless within 0.05 mm already), centre-cropped by SynthSegParams::crop, and padded to a multiple of 32 and at least 128 per axis, as preprocess does. SynthSegParams::head_crop can only shrink it, so with a head crop this is an upper bound. This is the grid the memory figures in SynthSegInference::LOW_MEMORY refer to (e.g. 192×256×128 for a 184×256×128 1 mm scan).

unalign_from_ras 🔒

Scatter a C-order RAS-oriented volume back to column-major (nx, ny, nz).