mixle.reason.model module

A trainable cross-modal reasoning model: learn a shared latent from many modalities, unsupervised.

This is the piece that makes the reasoning stack trained rather than assembled. The amortized encoder (mixle.reason.encoder) needs (x, z) pairs – you must already know the latent. Here the latent is never observed: the model learns per-modality encoders and decoders jointly, through a shared latent, from unlabeled multimodal records, by maximizing a Product-of-Experts variational lower bound (a multimodal VAE / MVAE).

• Each modality m has an encoder q_m(z | x_m) = N(mu_m, diag(sig_m^2)) – a Gaussian expert.

• The belief given any subset of modalities is the product of experts with the prior: precisions add, so more modalities => a sharper belief. This is exactly mixle.inference.belief.GaussianBelief.fuse(), now learned rather than hand-specified.

• Each modality has a decoder p(x_m | z); training reconstructs every modality from the fused latent (with modality-subset subsampling, so inference works from any subset – one modality, all, or anything between).

After training: belief(obs) returns q(z | available modalities) as a GaussianBelief (so it flows into mixle.reason.reason(), mixle.inference.decompose_uncertainty(), conformal calibration, …); predict(obs, target) generates a missing modality from the ones you have. Uncertainty is native: the belief is a distribution, sharpened by each modality in proportion to how much that modality’s encoder trusts its own reading.

Torch is imported lazily; mixle.reason exposes this via a deferred attribute.

class CrossModalModel(latent_dim, *, seed=0)

Bases: object

A multimodal Product-of-Experts VAE: one shared latent learned from many modalities, unsupervised.

Parameters:

• latent_dim (int) – dimension of the shared latent z.

• seed (int) – torch RNG seed.

add_modality(name, in_dim, *, hidden=(64,))

Register a modality with its encoder q(z|x) and decoder p(x|z) (both learned).

Parameters:

• name (str)

• in_dim (int)

• hidden (tuple[int, ...])

Return type:

CrossModalModel

fit(data, *, epochs=600, lr=3e-3, beta=0.5, subsample=True)

Train encoders + decoders jointly on unlabeled multimodal data by the PoE-ELBO.

data maps each registered modality name to an (N, in_dim) array (all modalities share the same N rows – row i is one record’s several views). beta weights the KL rate; with subsample=True the ELBO is also evaluated on each single-modality subset so the model can infer z from any one modality alone (the MVAE training trick).

Parameters:

• data (dict[str, Any])

• epochs (int)

• lr (float)

• beta (float)

• subsample (bool)

Return type:

CrossModalModel

belief(obs)

The belief q(z | available modalities) as a GaussianBelief (product of experts).

Parameters:

obs (dict[str, Any])

Return type:

GaussianBelief

encode(obs)

The posterior-mean latent code for obs (a shared-space embedding usable as store keys).

Parameters:

obs (dict[str, Any])

Return type:

ndarray

predict(obs, target)

Generate the target modality from the modalities in obs (cross-modal generation).

Parameters:

• obs (dict[str, Any])

• target (str)

Return type:

ndarray

calibrate(cal_data, target, *, alpha=0.1)

Calibrate cross-modal prediction of target for finite-sample coverage (split conformal).

On a held-out calibration set, predict target from the other modalities, normalize the per-dimension residuals, and take the ceil((n+1)(1-alpha))-th largest max-normalized residual as the conformal radius. Using the max over dimensions makes the guarantee simultaneous: predict_interval() returns a box whose joint coverage over the whole target vector is >= 1 - alpha – distribution-free, regardless of model specification (unlike the Gaussian posterior interval).

Parameters:

• cal_data (dict[str, Any])

• target (str)

• alpha (float)

Return type:

CrossModalModel

predict_interval(obs, target)

A conformally-calibrated prediction box (lower, upper) for target given obs.

Requires a prior calibrate() call for target. Coverage is distribution-free and simultaneous: P(y in box) >= 1 - alpha jointly over the whole target vector.

Parameters:

• obs (dict[str, Any])

• target (str)

Return type:

tuple[ndarray, ndarray]

property modalities: Sequence[str]