mixle.inference.structure module

Automatic dependency-structure learning for heterogeneous records – the tagline, taken literally.

CompositeDistribution models a record’s fields as independent (Naive-Bayes under a mixture). But real heterogeneous data has cross-field dependence – a category shifts a real’s mean, a count’s rate tracks another field – and modeling it is worth a great deal of likelihood (a blatant category->Gaussian link is ~1000 nats on 600 rows). No mainstream tool discovers that structure across arbitrary families: Stan/PyMC make you write it, sklearn/pomegranate mixtures assume independence, bnlearn/pgmpy are discrete-or-Gaussian only.

This module closes the gap. Dependence is detected the mixle way – by modeling it: fit P(child) vs P(child | parent) and compare description length (dependency_gain()). The winning edges are assembled into a DependencyTreeDistribution – a directed forest over the record where each field is either a marginal or a per-parent-value conditional (a real ConditionalDistribution edge) – and learn_structure() picks the forest and fits it automatically. The result scores, samples, and composes like any mixle distribution, but models the dependence a composite drops.

dependency_gain(parent, child, child_estimator, *, max_its=30, penalty='bic')[source]

Description-length gain (nats) of modeling child conditioned on a discrete parent vs. independently.

Fits the marginal P(child) and the conditional P(child | parent) (a child model per parent value) on the same data and returns LL_cond - LL_marginal minus a complexity penalty for the extra parameters (BIC: 0.5 * (levels - 1) * k * ln n). Positive means the dependence is worth modeling. This is a model-based dependency test – it works across any pair of families, unlike a same-type MI estimate.

Parameters:
Return type:

float

class LinearGaussianEdge(a, b, sigma2)[source]

Bases: object

A regression edge: P(child | parent) = Normal(a + b*parent, sigma2).

Where the current per-parent-bin conditional needs bins * k parameters (and coarse bins) to model a smooth continuous dependence, this captures it with a single slope b — far more statistically efficient and exact for a linear relationship. Slots into DependencyTreeDistribution as a factor with an identity binner (the raw parent value drives the conditional).

Parameters:
log_density(x)[source]
Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]
Return type:

Any

seq_log_density(encoded)[source]
Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]
Parameters:

seed (int | None)

Return type:

Any

fit_linear_gaussian_edge(pairs)[source]

OLS fit of a linear-Gaussian conditional child ~ a + b*parent (closed form).

Parameters:

pairs (Sequence[tuple])

Return type:

LinearGaussianEdge

regression_gain(parent, child, child_estimator, *, max_its=30, penalty='bic')[source]

Description-length gain (nats) of a linear-Gaussian regression edge child ~ a + b*parent over the child marginal. One extra parameter (the slope) vs. the bins * k a binned conditional spends — so for a real linear dependence this beats binning decisively. Returns -inf when a regression is undefined.

Parameters:
Return type:

float

class GLMEdge(family, beta, link, phi=1.0)[source]

Bases: object

A generalized-linear regression edge: a count child’s rate = exp(a + b*parent) (Poisson log-link), a binary child’s probability = logit^-1(a + b*parent) (logistic) — the heterogeneous generalization of LinearGaussianEdge (McCullagh & Nelder 1989). One slope parameter, fit by IRLS via mixle.inference.glm.glm(). Models a count/binary child driven by a continuous parent far better than a coarse per-bin conditional.

Parameters:
log_density(x)[source]
Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]
Return type:

Any

seq_log_density(encoded)[source]
Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]
Parameters:

seed (int | None)

Return type:

Any

fit_glm_edge(pairs, family)[source]

Fit a GLM conditional child ~ g^-1(a + b*parent) (family’s canonical link) by IRLS.

Parameters:
Return type:

GLMEdge

glm_gain(parent, child, child_estimator, family, *, max_its=30, penalty='bic')[source]

Description-length gain (nats) of a GLM family regression edge over the child marginal (one extra slope parameter). Returns -inf when the fit is undefined or non-finite.

Parameters:
Return type:

float

class DependencyTreeDistribution(parents, factors, binners=None)[source]

Bases: object

A directed-forest joint over a heterogeneous record: each field is a marginal or a conditional on its parent.

log_density(record) = sum_root log P(f_root) + sum_child log P(f_child | f_parent). The dependence a CompositeDistribution assumes away is modeled here as per-parent-value conditionals – while it still scores, samples, and composes like any mixle distribution.

Parameters:

  • parents (Sequence[int | None])

  • factors (Sequence[Any])

  • binners (Sequence[Any] | None)

log_density(x)[source]
Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]
Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]
Return type:

Any

sampler(seed=None)[source]
Parameters:

seed (int | None)

Return type:

Any

edges()[source]

The learned dependency edges (parent_field, child_field).

Return type:

list[tuple[int, int]]

class MixtureOfDependencyTrees(components, weights)[source]

Bases: object

A latent mixture whose components each carry their own discovered dependency structure.

log p(x) = logsumexp_k ( log w_k + log p_k(x) ) where each p_k is a DependencyTreeDistribution. This is the deep form of the tagline: it discovers both the clustering and the within-cluster cross-field dependence – so the same category can map to different reals in different clusters, which neither a single dependency tree (one relationship) nor a mixture of independent composites (no within-cluster dependence) can represent. Fit by learn_mixture_structure().

Parameters:

  • components (Sequence[DependencyTreeDistribution])

  • weights (Sequence[float])

log_density(x)[source]
Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]
Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]
Return type:

Any

responsibilities(data)[source]

Posterior p(component | record) for each record – the E-step and a soft cluster assignment.

Parameters:

data (Sequence[tuple])

Return type:

ndarray

sampler(seed=None)[source]
Parameters:

seed (int | None)

Return type:

Any

property n_components: int
learn_mixture_structure(data, n_components, *, restarts=3, max_iter=15, seed=0, min_gain=0.0, n_bins=4, max_its=30)[source]

Fit a MixtureOfDependencyTrees by hard EM – discover clusters AND each cluster’s dependency graph.

Each iteration re-learns a dependency forest per cluster on its currently-assigned points (M-step), then reassigns every record to its most-probable cluster (E-step), until assignments stabilize. Runs restarts random initializations and returns the highest-likelihood fit. Empty/tiny clusters are re-seeded so a component never collapses.

Parameters:
Return type:

MixtureOfDependencyTrees

learn_structure(data, *, field_estimators=None, min_gain=0.0, max_levels=64, n_bins=4, max_its=30)[source]

Discover the dependency forest for heterogeneous data and return the fitted joint model.

Any field can be a parent: a discrete one conditions directly, a continuous one is quantile-binned into n_bins conditioning levels (so a real can drive a count, a category, or another real). Scores every (parent -> child) pair by dependency_gain(), greedily builds a maximum-gain acyclic forest (each field at most one parent), and fits each factor. Falls back to independent marginals where no dependence clears min_gain – never worse than a composite, much better when structure exists. This is “automatic inference for composable models of heterogeneous data” made real.

Parameters:
Return type:

DependencyTreeDistribution