mixle.inference.structure module

Automatic dependency-structure learning for heterogeneous records – the tagline, taken literally.

CompositeDistribution models a record’s fields as independent (Naive-Bayes under a mixture). But real heterogeneous data has cross-field dependence – a category shifts a real’s mean, a count’s rate tracks another field – and modeling it is worth a great deal of likelihood (a blatant category->Gaussian link is ~1000 nats on 600 rows). No mainstream tool discovers that structure across arbitrary families: Stan/PyMC make you write it, sklearn/pomegranate mixtures assume independence, bnlearn/pgmpy are discrete-or-Gaussian only.

This module closes the gap. Dependence is detected by modeling it: fit P(child) vs P(child | parent) and compare description length (dependency_gain()). The winning edges are assembled into a DependencyTreeDistribution – a directed forest over the record where each field is either a marginal or a per-parent-value conditional (a real ConditionalDistribution edge) – and learn_structure() picks the forest and fits it automatically. The result scores, samples, and composes like any mixle distribution, but models the dependence a composite drops.

dependency_gain(parent, child, child_estimator, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of modeling child conditioned on a discrete parent vs. independently.

Fits the marginal P(child) and the conditional P(child | parent) (a child model per parent value) on the same data and returns LL_cond - LL_marginal minus a complexity penalty for the extra parameters (BIC: 0.5 * (levels - 1) * k * ln n). Positive means the dependence is worth modeling. This is a model-based dependency test – it works across any pair of families, unlike a same-type MI estimate. rng seeds the fits’ EM initializations (None = a fixed seed: deterministic by default; matters when the child family needs a randomized init, e.g. a mixture).

Parameters:
Return type:

float

class LinearGaussianEdge(a, b, sigma2)[source]

Bases: object

A regression edge: P(child | parent) = Normal(a + b*parent, sigma2).

Where the current per-parent-bin conditional needs bins * k parameters (and coarse bins) to model a smooth continuous dependence, this captures it with a single slope b — far more statistically efficient and exact for a linear relationship. Slots into DependencyTreeDistribution as a factor with an identity binner (the raw parent value drives the conditional).

Parameters:
log_density(x)[source]

Evaluate log p(child | parent) for one parent-child pair.

Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]

Return the encoder for parent-child pairs.

Return type:

Any

seq_log_density(encoded)[source]

Evaluate log densities for encoded parent-child pairs.

Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the linear Gaussian edge.

Parameters:

seed (int | None)

Return type:

Any

fit_linear_gaussian_edge(pairs)[source]

OLS fit of a linear-Gaussian conditional child ~ a + b*parent (closed form).

Parameters:

pairs (Sequence[tuple])

Return type:

LinearGaussianEdge

regression_gain(parent, child, child_estimator, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of a linear-Gaussian regression edge child ~ a + b*parent over the child marginal. One extra parameter (the slope) vs. the bins * k a binned conditional spends — so for a real linear dependence this beats binning decisively. Returns -inf when a regression is undefined. rng seeds the marginal fit’s EM initialization (None = a fixed seed: deterministic by default).

Parameters:
Return type:

float

class GLMEdge(family, beta, link, phi=1.0)[source]

Bases: object

A generalized-linear regression edge: a count child’s rate = exp(a + b*parent) (Poisson log-link), a binary child’s probability = logit^-1(a + b*parent) (logistic) — the heterogeneous generalization of LinearGaussianEdge (McCullagh & Nelder 1989). One slope parameter, fit by IRLS via mixle.inference.glm.glm(). Models a count/binary child driven by a continuous parent far better than a coarse per-bin conditional.

Parameters:
log_density(x)[source]

Evaluate log p(child | parent) for one GLM edge pair.

Parameters:

x (tuple)

Return type:

float

dist_to_encoder()[source]

Return the encoder for parent-child pairs.

Return type:

Any

seq_log_density(encoded)[source]

Evaluate log densities for encoded GLM edge pairs.

Parameters:

encoded (Any)

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the GLM edge.

Parameters:

seed (int | None)

Return type:

Any

fit_glm_edge(pairs, family)[source]

Fit a GLM conditional child ~ g^-1(a + b*parent) (family’s canonical link) by IRLS.

Parameters:
Return type:

GLMEdge

glm_gain(parent, child, child_estimator, family, *, max_its=30, penalty='bic', rng=None)[source]

Description-length gain (nats) of a GLM family regression edge over the child marginal (one extra slope parameter). Returns -inf when the fit is undefined or non-finite. rng seeds the marginal fit’s EM initialization (None = a fixed seed: deterministic by default).

Parameters:
Return type:

float

class DependencyTreeDistribution(parents, factors, binners=None)[source]

Bases: FitProvenanceCarrier

A directed-forest joint over a heterogeneous record: each field is a marginal or a conditional on its parent.

log_density(record) = sum_root log P(f_root) + sum_child log P(f_child | f_parent). The dependence a CompositeDistribution assumes away is modeled here as per-parent-value conditionals – while it still scores, samples, and composes like any mixle distribution.

Parameters:

  • parents (Sequence[int | None])

  • factors (Sequence[Any])

  • binners (Sequence[Any] | None)

log_density(x)[source]

Evaluate the dependency-tree joint log density for one record.

Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]

Evaluate dependency-tree joint log density for encoded records.

Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]

Return the encoder for dependency-tree record batches.

Return type:

Any

sampler(seed=None)[source]

Return a sampler for the dependency tree.

Parameters:

seed (int | None)

Return type:

Any

edges()[source]

The learned dependency edges (parent_field, child_field).

Return type:

list[tuple[int, int]]

class MixtureOfDependencyTrees(components, weights)[source]

Bases: object

A latent mixture whose components each carry their own discovered dependency structure.

log p(x) = logsumexp_k ( log w_k + log p_k(x) ) where each p_k is a DependencyTreeDistribution. This is the deep form of the tagline: it discovers both the clustering and the within-cluster cross-field dependence – so the same category can map to different reals in different clusters, which neither a single dependency tree (one relationship) nor a mixture of independent composites (no within-cluster dependence) can represent. Fit by learn_mixture_structure().

Parameters:

  • components (Sequence[DependencyTreeDistribution])

  • weights (Sequence[float])

property w: ndarray

Mixture-convention alias for weights – lets mixture-generic tooling (e.g. mixle.utils.hvis.model_fit_health / hvis_map) accept this model unchanged.

property log_w: ndarray

Mixture-convention alias for log_weights (see w).

log_density(x)[source]

Evaluate mixture log density for one dependency-tree record.

Parameters:

x (tuple)

Return type:

float

seq_log_density(encoded)[source]

Evaluate mixture log density for encoded dependency-tree records.

Parameters:

encoded (Any)

Return type:

ndarray

dist_to_encoder()[source]

Return the record encoder shared by all tree components.

Return type:

Any

responsibilities(data)[source]

Posterior p(component | record) for each record – the E-step and a soft cluster assignment.

Parameters:

data (Sequence[tuple])

Return type:

ndarray

sampler(seed=None)[source]

Return a sampler for the mixture of dependency trees.

Parameters:

seed (int | None)

Return type:

Any

property n_components: int

Return the number of mixture components.

learn_mixture_structure(data, n_components, *, restarts=3, max_iter=15, seed=0, min_gain=0.0, n_bins=4, max_its=30, field_estimators=None, tie_tolerance=1.0)[source]

Fit a MixtureOfDependencyTrees by hard EM: discover clusters and each cluster’s dependency graph.

Each iteration re-learns a dependency forest per cluster on its currently-assigned points (M-step), then reassigns every record to its most-probable cluster (E-step), until assignments stabilize. Runs restarts random initializations and returns the highest-likelihood fit. Empty/tiny clusters receive a rescue fit and temporary proposal mass, but the returned weights count only actual hard memberships and may therefore contain an honest zero. Deterministic given seed: the one RandomState drives initialization, rescue, and every per-cluster fit.

field_estimators pins each field’s family (forwarded to learn_structure()) instead of re-running the automatic detector per cluster per iteration. Beyond the speedup, pinning matters for identifiability: the detector models a multimodal column with a Gaussian MIXTURE, which lets ONE cluster absorb what the caller intended as two – with per-field families pinned to unimodal models, regimes that differ in level must separate into different components to score well.

tie_tolerance (nats) is how close two restarts’ log-likelihoods have to be before mixture_structure_health() breaks the tie. On a two-regime corpus a restart search finds two optima – the regime split, whose components pass the receipt, and a category split whose components each absorb both regimes – and which one it returned depended on the seed (A-03). Inside the tolerance the healthy candidate wins; outside it likelihood decides, because the receipt is a diagnosis and must not overrule a materially better fit. Set it to 0.0 for pure likelihood selection.

The default of one nat is deliberately narrow, and on the two-regime corpus that motivated the rule it does not reach: measured over 48 (corpus, seed) pairs, the likelihood winner is the healthy candidate outright in most runs, and in the three where it is not, the healthy candidate gives up 6.0, 6.4 and 84.8 nats – so the tie-break correctly declines and the receipt reports the difference, exactly as it did before the rule existed. A caller who wants health to arbitrate a gap that wide has to say so with a larger tie_tolerance; the default will not do it silently, because at six nats the two candidates are no longer tied on anything.

Parameters:
Return type:

MixtureOfDependencyTrees

mixture_structure_health(mot, data, *, merged_sep_threshold=None)[source]

The identifiability receipt for a fitted MixtureOfDependencyTrees.

The trap it names: a flexible per-field family (a Gaussian-mixture conditional, a heavy-tailed catch-all marginal) lets ONE component absorb what the caller intended as SEVERAL regimes – and because the absorbed fit scores well, likelihood-level receipts look healthy. Measured concretely: on two planted regimes, a one-component fit matches the two-component fit’s likelihood, so nothing downstream of the density can see the difference.

The check that can: STRUCTURE-CONDITIONAL multimodality. For every continuous field of every component, group that component’s dominated points by the field’s own learned conditioning (parent level for a binned conditional, regression residuals for a linear edge, the whole fiber for a root marginal) – after the tree has explained what it can, each group must be unimodal. A group that still splits (deterministic 2-means separation over the same 2.65 + 6/sqrt(n) finite-sample threshold the hvis merged-regime detector is calibrated to, minority share >= 20%) is a regime split the component is hiding, whichever family absorbed it.

Returns {"components": [...], "diagnosis": [str, ...]} – empty diagnosis means no component hides multimodal structure. Per component, multimodal_fields lists (field, where, separation) and mixture_factors lists factors the detector fitted as mixture families (supporting detail: where the absorbed structure went). The fix when it fires is more components or pinned unimodal field_estimators (see learn_mixture_structure()). Complementary to mixle.utils.hvis.model_fit_health, which audits density calibration and accepts this model directly.

Parameters:

  • mot (MixtureOfDependencyTrees)

  • data (Sequence[tuple])

  • merged_sep_threshold (float | None)

Return type:

dict

learn_structure(data, *, field_estimators=None, min_gain=0.0, max_levels=64, n_bins=4, max_its=30, rng=None)[source]

Discover the dependency forest for heterogeneous data and return the fitted joint model.

Any field can be a parent: a discrete one conditions directly, a continuous one is quantile-binned into n_bins conditioning levels (so a real can drive a count, a category, or another real). Scores every (parent -> child) pair by dependency_gain(), greedily builds a maximum-gain acyclic forest (each field at most one parent), and fits each factor. Falls back to independent marginals where no dependence clears min_gain – never worse than a composite, much better when structure exists. This is “automatic inference for composable models of heterogeneous data” made real.

rng seeds every internal fit’s EM initialization; None resolves to a FIXED seed, so two calls on the same data return the same model. (Before this knob, fits whose detected family needs a randomized init – a Gaussian-mixture conditional, say – drew fresh OS entropy per call, so the learned model itself was nondeterministic.)

Parameters:
Return type:

DependencyTreeDistribution