mixle.inference.structure module

Automatic dependency-structure learning for heterogeneous records – the tagline, taken literally.

CompositeDistribution models a record’s fields as independent (Naive-Bayes under a mixture). But real heterogeneous data has cross-field dependence – a category shifts a real’s mean, a count’s rate tracks another field – and modeling it is worth a great deal of likelihood (a blatant category->Gaussian link is ~1000 nats on 600 rows). No mainstream tool discovers that structure across arbitrary families: Stan/PyMC make you write it, sklearn/pomegranate mixtures assume independence, bnlearn/pgmpy are discrete-or-Gaussian only.

This module closes the gap. Dependence is detected by modeling it: fit P(child) vs P(child | parent) and compare description length (dependency_gain()). The winning edges are assembled into a DependencyTreeDistribution – a directed forest over the record where each field is either a marginal or a per-parent-value conditional (a real ConditionalDistribution edge) – and learn_structure() picks the forest and fits it automatically. The result scores, samples, and composes like any mixle distribution, but models the dependence a composite drops.

dependency_gain(parent, child, child_estimator, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of modeling child conditioned on a discrete parent vs. independently.

Fits the marginal P(child) and the conditional P(child | parent) (a child model per parent value) on the same data and returns LL_cond - LL_marginal minus a complexity penalty for the extra parameters (BIC: 0.5 * (levels - 1) * k * ln n). Positive means the dependence is worth modeling. This is a model-based dependency test – it works across any pair of families, unlike a same-type MI estimate. rng seeds the fits’ EM initializations (None = a fixed seed: deterministic by default; matters when the child family needs a randomized init, e.g. a mixture).

Parameters:
Return type:

float

class LinearGaussianEdge(a, b, sigma2)[source]

Bases: object

A regression edge: P(child | parent) = Normal(a + b*parent, sigma2).

Where the current per-parent-bin conditional needs bins * k parameters (and coarse bins) to model a smooth continuous dependence, this captures it with a single slope b — far more statistically efficient and exact for a linear relationship. Slots into DependencyTreeDistribution as a factor with an identity binner (the raw parent value drives the conditional).

Parameters:
log_density(x)[source]

Evaluate log p(child | parent) for one parent-child pair.

Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]

Return the encoder for parent-child pairs.

Return type:

Any

seq_log_density(encoded)[source]

Evaluate log densities for encoded parent-child pairs.

Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the linear Gaussian edge.

Parameters:

seed (int | None)

Return type:

Any

fit_linear_gaussian_edge(pairs)[source]

OLS fit of a linear-Gaussian conditional child ~ a + b*parent (closed form).

Parameters:

pairs (Sequence[tuple])

Return type:

LinearGaussianEdge

regression_gain(parent, child, child_estimator, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of a linear-Gaussian regression edge child ~ a + b*parent over the child marginal. One extra parameter (the slope) vs. the bins * k a binned conditional spends — so for a real linear dependence this beats binning decisively. Returns -inf when a regression is undefined. rng seeds the marginal fit’s EM initialization (None = a fixed seed: deterministic by default).

Parameters:
Return type:

float

class GLMEdge(family, beta, link, phi=1.0)[source]

Bases: object

A generalized-linear regression edge: a count child’s rate = exp(a + b*parent) (Poisson log-link), a binary child’s probability = logit^-1(a + b*parent) (logistic) — the heterogeneous generalization of LinearGaussianEdge (McCullagh & Nelder 1989). One slope parameter, fit by IRLS via mixle.inference.glm.glm(). Models a count/binary child driven by a continuous parent far better than a coarse per-bin conditional.

Parameters:
log_density(x)[source]

Evaluate log p(child | parent) for one GLM edge pair.

Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]

Return the encoder for parent-child pairs.

Return type:

Any

seq_log_density(encoded)[source]

Evaluate log densities for encoded GLM edge pairs.

Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the GLM edge.

Parameters:

seed (int | None)

Return type:

Any

fit_glm_edge(pairs, family)[source]

Fit a GLM conditional child ~ g^-1(a + b*parent) (family’s canonical link) by IRLS.

Parameters:
Return type:

GLMEdge

glm_gain(parent, child, child_estimator, family, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of a GLM family regression edge over the child marginal (one extra slope parameter). Returns -inf when the fit is undefined or non-finite. rng seeds the marginal fit’s EM initialization (None = a fixed seed: deterministic by default).

Parameters:
Return type:

float

class DependencyTreeDistribution(parents, factors, binners=None)[source]

Bases: object

A directed-forest joint over a heterogeneous record: each field is a marginal or a conditional on its parent.

log_density(record) = sum_root log P(f_root) + sum_child log P(f_child | f_parent). The dependence a CompositeDistribution assumes away is modeled here as per-parent-value conditionals – while it still scores, samples, and composes like any mixle distribution.

Parameters:

  • parents (Sequence[int | None])

  • factors (Sequence[Any])

  • binners (Sequence[Any] | None)

log_density(x)[source]

Evaluate the dependency-tree joint log density for one record.

Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]

Evaluate dependency-tree joint log density for encoded records.

Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]

Return the encoder for dependency-tree record batches.

Return type:

Any

sampler(seed=None)[source]

Return a sampler for the dependency tree.

Parameters:

seed (int | None)

Return type:

Any

edges()[source]

The learned dependency edges (parent_field, child_field).

Return type:

list[tuple[int, int]]

class MixtureOfDependencyTrees(components, weights)[source]

Bases: object

A latent mixture whose components each carry their own discovered dependency structure.

log p(x) = logsumexp_k ( log w_k + log p_k(x) ) where each p_k is a DependencyTreeDistribution. This is the deep form of the tagline: it discovers both the clustering and the within-cluster cross-field dependence – so the same category can map to different reals in different clusters, which neither a single dependency tree (one relationship) nor a mixture of independent composites (no within-cluster dependence) can represent. Fit by learn_mixture_structure().

Parameters:

  • components (Sequence[DependencyTreeDistribution])

  • weights (Sequence[float])

property w: ndarray

Mixture-convention alias for weights – lets mixture-generic tooling (e.g. mixle.utils.hvis.model_fit_health / hvis_map) accept this model unchanged.

property log_w: ndarray

Mixture-convention alias for log_weights (see w).

log_density(x)[source]

Evaluate mixture log density for one dependency-tree record.

Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]

Evaluate mixture log density for encoded dependency-tree records.

Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]

Return the record encoder shared by all tree components.

Return type:

Any

responsibilities(data)[source]

Posterior p(component | record) for each record – the E-step and a soft cluster assignment.

Parameters:

data (Sequence[tuple])

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the mixture of dependency trees.

Parameters:

seed (int | None)

Return type:

Any

property n_components: int

Return the number of mixture components.

learn_mixture_structure(data, n_components, *, restarts=3, max_iter=15, seed=0, min_gain=0.0, n_bins=4, max_its=30, field_estimators=None)[source]

Fit a MixtureOfDependencyTrees by hard EM: discover clusters and each cluster’s dependency graph.

Each iteration re-learns a dependency forest per cluster on its currently-assigned points (M-step), then reassigns every record to its most-probable cluster (E-step), until assignments stabilize. Runs restarts random initializations and returns the highest-likelihood fit. Empty/tiny clusters are re-seeded so a component never collapses. Deterministic given seed: the one RandomState drives the k-means/random initializations AND every per-cluster fit’s EM init (via learn_structure()’s rng).

field_estimators pins each field’s family (forwarded to learn_structure()) instead of re-running the automatic detector per cluster per iteration. Beyond the speedup, pinning matters for identifiability: the detector models a multimodal column with a Gaussian MIXTURE, which lets ONE cluster absorb what the caller intended as two – with per-field families pinned to unimodal models, regimes that differ in level must separate into different components to score well. random initializations and returns the highest-likelihood fit. Empty or very small clusters are re-seeded so a component never collapses.

Parameters:
Return type:

MixtureOfDependencyTrees

mixture_structure_health(mot, data, *, merged_sep_threshold=None)[source]

The identifiability receipt for a fitted MixtureOfDependencyTrees.

The trap it names: a flexible per-field family (a Gaussian-mixture conditional, a heavy-tailed catch-all marginal) lets ONE component absorb what the caller intended as SEVERAL regimes – and because the absorbed fit scores well, likelihood-level receipts look healthy. Measured concretely: on two planted regimes, a one-component fit matches the two-component fit’s likelihood, so nothing downstream of the density can see the difference.

The check that can: STRUCTURE-CONDITIONAL multimodality. For every continuous field of every component, group that component’s dominated points by the field’s own learned conditioning (parent level for a binned conditional, regression residuals for a linear edge, the whole fiber for a root marginal) – after the tree has explained what it can, each group must be unimodal. A group that still splits (deterministic 2-means separation over the same 2.65 + 6/sqrt(n) finite-sample threshold the hvis merged-regime detector is calibrated to, minority share >= 20%) is a regime split the component is hiding, whichever family absorbed it.

Returns {"components": [...], "diagnosis": [str, ...]} – empty diagnosis means no component hides multimodal structure. Per component, multimodal_fields lists (field, where, separation) and mixture_factors lists factors the detector fitted as mixture families (supporting detail: where the absorbed structure went). The fix when it fires is more components or pinned unimodal field_estimators (see learn_mixture_structure()). Complementary to mixle.utils.hvis.model_fit_health, which audits density calibration and accepts this model directly.

Parameters:

  • mot (MixtureOfDependencyTrees)

  • data (Sequence[tuple])

  • merged_sep_threshold (float | None)

Return type:

dict

learn_structure(data, *, field_estimators=None, min_gain=0.0, max_levels=64, n_bins=4, max_its=30, rng=None)[source]

Discover the dependency forest for heterogeneous data and return the fitted joint model.

Any field can be a parent: a discrete one conditions directly, a continuous one is quantile-binned into n_bins conditioning levels (so a real can drive a count, a category, or another real). Scores every (parent -> child) pair by dependency_gain(), greedily builds a maximum-gain acyclic forest (each field at most one parent), and fits each factor. Falls back to independent marginals where no dependence clears min_gain – never worse than a composite, much better when structure exists. This is “automatic inference for composable models of heterogeneous data” made real.

rng seeds every internal fit’s EM initialization; None resolves to a FIXED seed, so two calls on the same data return the same model. (Before this knob, fits whose detected family needs a randomized init – a Gaussian-mixture conditional, say – drew fresh OS entropy per call, so the learned model itself was nondeterministic.)

Parameters:
Return type:

DependencyTreeDistribution