mixle.stats.compute.exp_family module

Uniform exponential-family canonical map for mixle distributions.

Every exponential-family (and conditional-exponential-family) model is expressed in the canonical form

p(x) = h(x) * exp( <eta, T(x)> - A(eta) ) # unconditional p(y|x) = h(y) * exp( <eta(x), T(y)> - A(eta(x)) ) # conditional

This module surfaces that form as a first-class object. The per-family math already lives in each distribution’s ExponentialFamilySpec (sufficient_statistics T, natural_parameters eta, log_partition A, base_measure h); to_exponential_family() reads that declaration and wraps it, so adding a family is a matter of providing its spec – there is no type switch here.

The container threads a compute engine (numpy by default) so the map works under numpy and torch and stays autograd-friendly for ExponentialFamilyForm.mean_parameters().

class ExponentialFamilyForm(distribution, spec, engine=<mixle.engines.numpy_engine.NumpyEngine object>)[source]

Bases: object

Canonical exponential-family view of a single distribution.

Holds the source distribution and its ExponentialFamilySpec, and exposes the canonical pieces (eta, T(x), A(eta), log h(x)) plus derived quantities. All array methods are vectorized over a leading sample axis and routed through engine (numpy by default).

Parameters:

  • distribution (ProbabilityDistribution)

  • spec (ExponentialFamilySpec)

  • engine (Any)

distribution: ProbabilityDistribution
spec: ExponentialFamilySpec
engine: Any = <mixle.engines.numpy_engine.NumpyEngine object>
property dim: int

Length of the natural-parameter / sufficient-statistic vector.

natural_parameters()[source]

Return the natural parameters eta(theta) for the current parameters.

Return type:

ndarray

sufficient_statistics(x)[source]

Return the sufficient statistics T(x) with shape (n, dim).

Parameters:

x (Any)

Return type:

Any

log_partition(eta=None)[source]

Return the log-partition A.

With eta=None (default) this is A(eta(theta)) for the current parameters. Supplying eta requires the dual map theta(eta); it is evaluated by reconstructing a distribution via from_natural() and is only available where from_natural has a closed form.

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)[source]

Return log h(x) (in log-space to avoid e.g. 1/x! overflow).

Parameters:

x (Any)

Return type:

Any

log_density(x)[source]

Return <eta, T(x)> - A(eta) + log h(x) – the reconstructed log-density.

Parameters:

x (Any)

Return type:

Any

mean_parameters(eps=1.0e-6, n_samples=200000, seed=0)[source]

Return the mean (expectation) parameters grad A(eta) = E[T(x)].

When the family exposes a closed-form dual map (exp_family_from_natural) this is the exact gradient of A by central finite differences in natural coordinates. Otherwise it falls back to a Monte-Carlo estimate of E[T(x)] over n_samples draws – approximate, but universal for any samplable family.

Parameters:
Return type:

ndarray

fisher_information(n_samples=200000, seed=0)[source]

Return the Fisher information in natural coordinates, I(eta) = Cov[T(x)] = grad^2 A(eta).

For an exponential family the Fisher information with respect to the natural parameters is exactly the covariance of the sufficient statistic (equivalently the Hessian of the log-partition). This is the second-order companion to mean_parameters() (grad A = E[T]); it is estimated by the sample covariance of T(x) over n_samples draws – approximate, but universal for any samplable family – and returned as a (dim, dim) symmetric positive-semidefinite matrix.

Parameters:

  • n_samples (int)

  • seed (int | None)

Return type:

ndarray

from_natural(eta)[source]

Return theta(eta) as a reconstructed distribution, or None.

The default has no generic inverse link; families with a closed form attach exp_family_from_natural(eta) -> ProbabilityDistribution to their class.

Parameters:

eta (Any)

Return type:

ProbabilityDistribution | None

class ProductExponentialFamilyForm(distribution, components, engine=<mixle.engines.numpy_engine.NumpyEngine object>, extract=None)[source]

Bases: object

Canonical exponential-family view of an independent product of distributions.

A product of exponential families is itself an exponential family with eta = concat(eta_i), T(x) = concat(T_i(x_i)), A = sum A_i, and log h(x) = sum log h_i(x_i) (the closure rule for an independent product). Used for CompositeDistribution.

Parameters:

  • distribution (ProbabilityDistribution)

  • components (tuple[ExponentialFamilyForm, ...])

  • engine (Any)

  • extract (Any)

distribution: ProbabilityDistribution
components: tuple[ExponentialFamilyForm, ...]
engine: Any = <mixle.engines.numpy_engine.NumpyEngine object>
extract: Any = None
property dim: int

Total natural-parameter dimension (sum over components).

natural_parameters()[source]

Return the concatenated natural parameters of all components.

Return type:

ndarray

sufficient_statistics(x)[source]

Return concatenated per-component sufficient statistics (n, dim).

Parameters:

x (Any)

Return type:

ndarray

log_partition(eta=None)[source]

Return A = sum_i A_i (current parameters only; eta override unsupported).

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)[source]

Return log h(x) = sum_i log h_i(x_i) row-wise.

Parameters:

x (Any)

Return type:

ndarray

log_density(x)[source]

Return the reconstructed log-density sum_i log p_i(x_i) row-wise.

Parameters:

x (Any)

Return type:

ndarray

mean_parameters(**kwargs)[source]

Return the concatenated mean parameters of all components.

Parameters:

kwargs (Any)

Return type:

ndarray

class IIDExponentialFamilyForm(distribution, element, engine=<mixle.engines.numpy_engine.NumpyEngine object>)[source]

Bases: object

Canonical exponential-family view of an iid sequence of a fixed leaf family.

For a fixed-length-agnostic iid sequence the joint sufficient statistic is the sum of the per-element statistics, T(x) = sum_t T_0(x_t), the natural parameters are shared (eta = eta_0), and A / log h scale with the element count. Used for SequenceDistribution when the length is not separately modeled.

Parameters:

  • distribution (ProbabilityDistribution)

  • element (ExponentialFamilyForm)

  • engine (Any)

distribution: ProbabilityDistribution
element: ExponentialFamilyForm
engine: Any = <mixle.engines.numpy_engine.NumpyEngine object>
property dim: int

Natural-parameter dimension (same as the element family).

natural_parameters()[source]

Return the shared element natural parameters.

Return type:

ndarray

sufficient_statistics(x)[source]

Return per-sequence summed element statistics (n, dim).

Parameters:

x (Any)

Return type:

ndarray

log_partition(eta=None)[source]

Return the per-element A (the joint scales by the element count).

Parameters:

eta (Any)

Return type:

Any

log_density(x)[source]

Return the reconstructed iid log-density sum_t log p_0(x_t) per sequence.

Parameters:

x (Any)

Return type:

ndarray

class MultinomialExponentialFamilyForm(distribution, element, engine=<mixle.engines.numpy_engine.NumpyEngine object>)[source]

Bases: object

Canonical exponential-family view of a multinomial over an exp-family element.

A multinomial observation is a bag {(v_j, c_j)} of values with counts, and the (non length-normalized, no separate trial distribution) log-density is the count-weighted sum of the element log-densities, sum_j c_j log p_0(v_j). So the natural parameters are the element’s (eta = eta_0), the sufficient statistic is the count-weighted sum T(x) = sum_j c_j T_0(v_j), log h(x) = sum_j c_j log h_0(v_j), and A is the element’s per-trial partition (the joint scales by the total count n = sum_j c_j). Built by to_exponential_family().

Parameters:

  • distribution (ProbabilityDistribution)

  • element (ExponentialFamilyForm)

  • engine (Any)

distribution: ProbabilityDistribution
element: ExponentialFamilyForm
engine: Any = <mixle.engines.numpy_engine.NumpyEngine object>
property dim: int

Natural-parameter dimension (same as the element family).

natural_parameters()[source]

Return the shared element natural parameters.

Return type:

ndarray

sufficient_statistics(x)[source]

Return the per-observation count-weighted element statistics (n, dim).

Parameters:

x (Any)

Return type:

ndarray

log_partition(eta=None)[source]

Return the per-trial A (the joint scales by the total count).

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)[source]

Return log h(x) = sum_j c_j log h_0(v_j) per observation.

Parameters:

x (Any)

Return type:

ndarray

log_density(x)[source]

Return the reconstructed log-density sum_j c_j log p_0(v_j) per observation.

Parameters:

x (Any)

Return type:

ndarray

mean_parameters(**kwargs)[source]

Return the element mean parameters (the per-trial expectation of T).

Parameters:

kwargs (Any)

Return type:

ndarray

class ConditionalExponentialFamilyForm(response_family, natural_fn, log_partition_fn=None, mean_fn=None, dispersion=None, engine=<mixle.engines.numpy_engine.NumpyEngine object>)[source]

Bases: object

Canonical exponential-family view of a conditional model p(y | x).

The response family fixes T, A, and h; natural_parameters(x) supplies the per-row natural parameters eta(x) (the linear predictor for a canonical link). dispersion carries a nuisance/dispersion parameter when the response family has one (e.g. a Gaussian variance).

Parameters:

  • response_family (ProbabilityDistribution)

  • natural_fn (Any)

  • log_partition_fn (Any)

  • mean_fn (Any)

  • dispersion (Any)

  • engine (Any)

response_family: ProbabilityDistribution
natural_fn: Any
log_partition_fn: Any = None
mean_fn: Any = None
dispersion: Any = None
engine: Any = <mixle.engines.numpy_engine.NumpyEngine object>
natural_parameters(x)[source]

Return the per-row natural parameters eta(x).

Parameters:

x (Any)

Return type:

ndarray

sufficient_statistics(y)[source]

Return the response sufficient statistics T(y).

Parameters:

y (Any)

Return type:

Any

log_base_measure(y)[source]

Return log h(y) for the response family.

Parameters:

y (Any)

Return type:

Any

log_partition(eta)[source]

Return A(eta) row-wise for a stack of natural parameters.

Uses the family-specific log_partition_fn (supplied by the GLM wiring, which knows the response family) when present; otherwise falls back to the response family’s dual map (only available where from_natural is closed-form).

Parameters:

eta (Any)

Return type:

ndarray

mean(x)[source]

Return the conditional mean E[y|x] = link_inv(eta(x)) (the inverse link).

Parameters:

x (Any)

Return type:

ndarray

log_density(y, x)[source]

Return log h(y) + <eta(x), T(y)> - A(eta(x)) row-wise.

Parameters:
Return type:

ndarray

to_exponential_family(dist, engine=NUMPY_ENGINE)[source]

Return the canonical exponential-family view of dist or None.

Mirrors mixle.inference.fisher.to_fisher(): a thin top-level helper that defers to ProbabilityDistribution.to_exponential_family(). Returns None when dist is not a (single) exponential family.

Parameters:

  • dist (ProbabilityDistribution)

  • engine (Any)

Return type:

ExponentialFamilyForm | None

is_exponential_family(dist)[source]

Return whether dist exposes a (single) exponential-family canonical form.

Parameters:

dist (ProbabilityDistribution)

Return type:

bool