mixle.reason.model module

Trainable cross-modal reasoning model with a shared latent.

This module learns per-modality encoders and decoders jointly from unlabeled multimodal records. The latent is not observed; training maximizes a Product-of-Experts variational lower bound for a multimodal VAE-style model.

• Each modality m has an encoder q_m(z | x_m) = N(mu_m, diag(sig_m^2)).

• The belief given any subset of modalities is the product of experts with the prior: precisions add, so more modalities produce a sharper belief. This matches mixle.inference.belief.GaussianBelief.fuse() with learned experts.

• Each modality has a decoder p(x_m | z); training reconstructs every modality from the fused latent. Modality-subset subsampling lets inference work from one modality, all modalities, or any subset between them.

After training: belief(obs) returns q(z | available modalities) as a GaussianBelief, and predict(obs, target) generates a missing modality from the available ones. Uncertainty remains part of the object: the returned belief is a distribution, sharpened by each modality in proportion to its learned precision.

Torch is imported lazily; mixle.reason exposes this via a deferred attribute.

class CrossModalModel(latent_dim, *, seed=0)

Bases: object

A multimodal Product-of-Experts VAE: one shared latent learned from many modalities, unsupervised.

Parameters:

• latent_dim (int) – dimension of the shared latent z.

• seed (int) – torch RNG seed.

add_modality(name, in_dim, *, hidden=(64,))

Register a modality with learned encoder q(z|x) and decoder p(x|z).

Parameters:

• name (str)

• in_dim (int)

• hidden (tuple[int, ...])

Return type:

CrossModalModel

fit(data, *, epochs=600, lr=3e-3, beta=0.5, subsample=True)

Train encoders and decoders jointly on unlabeled multimodal data.

data maps each registered modality name to an (N, in_dim) array (all modalities share the same N rows – row i is one record’s several views). beta weights the KL rate; with subsample=True the ELBO is also evaluated on each single-modality subset so the model can infer z from any one modality alone (the MVAE training trick).

Parameters:

• data (dict[str, Any])

• epochs (int)

• lr (float)

• beta (float)

• subsample (bool)

Return type:

CrossModalModel

belief(obs)

The belief q(z | available modalities) as a GaussianBelief (product of experts).

Parameters:

obs (dict[str, Any])

Return type:

GaussianBelief

encode(obs)

The posterior-mean latent code for obs (a shared-space embedding usable as store keys).

Parameters:

obs (dict[str, Any])

Return type:

ndarray

predict(obs, target)

Generate the target modality from the modalities in obs (cross-modal generation).

Parameters:

• obs (dict[str, Any])

• target (str)

Return type:

ndarray

calibrate(cal_data, target, *, alpha=0.1)

Calibrate cross-modal prediction of target for finite-sample coverage (split conformal).

On a held-out calibration set, predict target from the other modalities, normalize the per-dimension residuals, and take the ceil((n+1)(1-alpha))-th largest max-normalized residual as the conformal radius. Using the max over dimensions makes the guarantee simultaneous: predict_interval() returns a box whose joint coverage over the whole target vector is >= 1 - alpha – distribution-free, regardless of model specification (unlike the Gaussian posterior interval).

Parameters:

• cal_data (dict[str, Any])

• target (str)

• alpha (float)

Return type:

CrossModalModel

predict_interval(obs, target)

A conformally-calibrated prediction box (lower, upper) for target given obs.

Requires a prior calibrate() call for target. Coverage is distribution-free and simultaneous: P(y in box) >= 1 - alpha jointly over the whole target vector.

Parameters:

• obs (dict[str, Any])

• target (str)

Return type:

tuple[ndarray, ndarray]

property modalities: Sequence[str]

Return modality names known to the cross-modal model.