mixle.reason.fusion module¶
Differentiable product-of-experts fusion for structured multimodal evidence aggregation.
Dense cross-attention fuses N tokens in O(N^2). When the tokens are (conditionally) independent evidence about
a shared latent – the common case for aggregating many partial observations (image patches, sensors, views) –
precision-weighted product-of-experts fuses them in O(N) with few parameters:
the fusion inductive bias is built in, not learned. Each expert is a diagonal
Gaussian N(mu_i, diag(1/prec_i)) over the latent, and
the posterior is their normalized product:
prec_fused = sum_i prec_i + prec_prior # precisions add
mu_fused = (sum_i prec_i * mu_i) / prec_fused # precision-weighted mean
The reference benchmark in examples/structured_fusion_vlm.py reports that
PoE fusion matches a cross-attention block’s accuracy with fewer parameters and
faster training on exchangeable-evidence tasks.
Boundary condition: PoE fusion is permutation-invariant and assumes conditional independence, so it cannot model token order or pairwise interactions. On a task that depends on a specific pair or position, attention reaches ~0.96 while PoE sits at chance. Use structured fusion where evidence is exchangeable, and attention where relational interactions are part of the signal.
mixle.reason’s exact core (GaussianBelief) does this fusion in closed form for inference; this is the
torch, end-to-end-trainable version – the encoders that emit the experts are learned, the fusion stays exact.
Torch is imported lazily.