mixle.stats.compute.exp_family module

Uniform exponential-family canonical map for mixle distributions.

Every exponential-family (and conditional-exponential-family) model is expressed in the canonical form

p(x) = h(x) * exp( <eta, T(x)> - A(eta) ) # unconditional p(y|x) = h(y) * exp( <eta(x), T(y)> - A(eta(x)) ) # conditional

This module surfaces that form as a first-class object. The per-family math already lives in each distribution’s ExponentialFamilySpec (sufficient_statistics T, natural_parameters eta, log_partition A, base_measure h); to_exponential_family() reads that declaration and wraps it, so adding a family is a matter of providing its spec – there is no type switch here.

The container threads a compute engine (numpy by default) so sufficient-statistic maps preserve the selected backend, including its device and gradient lineage.

class NumericalEstimate(value, error_estimate, method, evaluations, n_samples=None)

Bases: object

A numerical value together with its method and elementwise error estimate.

Parameters:

• value (ndarray)

• error_estimate (ndarray | None)

• method (str)

• evaluations (int)

• n_samples (int | None)

class ExponentialFamilyForm(distribution, spec, engine=<mixle.engines.numpy_engine.NumpyEngine object>)

Bases: object

Canonical exponential-family view of a single distribution.

Holds the source distribution and its ExponentialFamilySpec, and exposes the canonical pieces (eta, T(x), A(eta), log h(x)) plus derived quantities. All array methods are vectorized over a leading sample axis and routed through engine (numpy by default).

Parameters:

• distribution (ProbabilityDistribution)

• spec (ExponentialFamilySpec)

• engine (Any)

property dim: int

Length of the natural-parameter / sufficient-statistic vector.

natural_parameters()

Return the natural parameters eta(theta) for the current parameters.

Return type:

ndarray

sufficient_statistics(x)

Return the sufficient statistics T(x) with shape (n, dim).

Parameters:

x (Any)

Return type:

Any

log_partition(eta=None)

Return the log-partition A.

With eta=None (default) this is A(eta(theta)) for the current parameters. Supplying eta requires the dual map theta(eta); it is evaluated by reconstructing a distribution via from_natural() and is only available where from_natural has a closed form.

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)

Return log h(x) (in log-space to avoid e.g. 1/x! overflow).

Parameters:

x (Any)

Return type:

Any

log_density(x)

Return <eta, T(x)> - A(eta) + log h(x) – the reconstructed log-density.

Parameters:

x (Any)

Return type:

Any

analytic_mean_parameters()

Return an explicitly declared analytic grad A implementation.

Return type:

ndarray

autodiff_mean_parameters()

Return grad A through an explicitly declared natural-space autodiff map.

Return type:

ndarray

finite_difference_mean_parameters(eps=1.0e-6)

Estimate grad A by guarded central differences with a truncation-error estimate.

Parameters:

eps (float)

Return type:

NumericalEstimate

monte_carlo_mean_parameters(n_samples=200000, seed=0)

Estimate E[T] by sampling and report its elementwise standard error.

Parameters:

• n_samples (int)

• seed (int | None)

Return type:

NumericalEstimate

estimate_mean_parameters(*, method='auto', eps=1.0e-6, n_samples=200000, seed=0, error_tolerance=None)

Return mean parameters with explicit method and numerical-error provenance.

Parameters:

• method (str)

• eps (float)

• n_samples (int)

• seed (int | None)

• error_tolerance (float | None)

Return type:

NumericalEstimate

mean_parameters(eps=1.0e-6, n_samples=200000, seed=0, *, method='auto', error_tolerance=None)

Return grad A = E[T] using an explicit or automatically selected method.

Use estimate_mean_parameters() when method/error provenance is needed. auto prefers declared analytic and autodiff maps, then guarded finite differences, and finally Monte Carlo.

Parameters:

• eps (float)

• n_samples (int)

• seed (int | None)

• method (str)

• error_tolerance (float | None)

Return type:

ndarray

estimate_fisher_information(n_samples=200000, seed=0, *, error_tolerance=None)

Return the Fisher information in natural coordinates, I(eta) = Cov[T(x)] = grad^2 A(eta).

For an exponential family the Fisher information with respect to the natural parameters is exactly the covariance of the sufficient statistic (equivalently the Hessian of the log-partition). This is the second-order companion to mean_parameters() (grad A = E[T]); it is estimated by the sample covariance of T(x) over n_samples draws – approximate, but universal for any samplable family – and returned as a (dim, dim) symmetric positive-semidefinite matrix.

Parameters:

• n_samples (int)

• seed (int | None)

• error_tolerance (float | None)

Return type:

NumericalEstimate

fisher_information(n_samples=200000, seed=0, *, error_tolerance=None)

Return a Monte-Carlo Fisher estimate; use estimate_fisher_information for its error.

Parameters:

• n_samples (int)

• seed (int | None)

• error_tolerance (float | None)

Return type:

ndarray

from_natural(eta)

Return theta(eta) as a reconstructed distribution, or None.

The default has no generic inverse link; families with a closed form attach exp_family_from_natural(eta) -> ProbabilityDistribution to their class.

Parameters:

eta (Any)

Return type:

ProbabilityDistribution | None

class ProductExponentialFamilyForm(distribution, components, engine=<mixle.engines.numpy_engine.NumpyEngine object>, extract=None)

Bases: object

Canonical exponential-family view of an independent product of distributions.

A product of exponential families is itself an exponential family with eta = concat(eta_i), T(x) = concat(T_i(x_i)), A = sum A_i, and log h(x) = sum log h_i(x_i) (the closure rule for an independent product). Used for CompositeDistribution.

Parameters:

• distribution (ProbabilityDistribution)

• components (tuple[ExponentialFamilyForm, ...])

• engine (Any)

• extract (Any)

property dim: int

Total natural-parameter dimension (sum over components).

natural_parameters()

Return the concatenated natural parameters of all components.

Return type:

ndarray

sufficient_statistics(x)

Return concatenated per-component sufficient statistics (n, dim).

Parameters:

x (Any)

Return type:

Any

log_partition(eta=None)

Return A = sum_i A_i (current parameters only; eta override unsupported).

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)

Return log h(x) = sum_i log h_i(x_i) row-wise.

Parameters:

x (Any)

Return type:

ndarray

log_density(x)

Return the reconstructed log-density sum_i log p_i(x_i) row-wise.

Parameters:

x (Any)

Return type:

ndarray

mean_parameters(**kwargs)

Return the concatenated mean parameters of all components.

Parameters:

kwargs (Any)

Return type:

ndarray

class IIDExponentialFamilyForm(distribution, element, engine=<mixle.engines.numpy_engine.NumpyEngine object>)

Bases: object

Canonical exponential-family view of an iid sequence of a fixed leaf family.

For a fixed-length-agnostic iid sequence the joint sufficient statistic is the sum of the per-element statistics, T(x) = sum_t T_0(x_t), the natural parameters are shared (eta = eta_0), and A / log h scale with the element count. Used for SequenceDistribution when the length is not separately modeled.

Parameters:

• distribution (ProbabilityDistribution)

• element (ExponentialFamilyForm)

• engine (Any)

property dim: int

Natural-parameter dimension (same as the element family).

natural_parameters()

Return the shared element natural parameters.

Return type:

ndarray

sufficient_statistics(x)

Return per-sequence summed element statistics (n, dim).

Parameters:

x (Any)

Return type:

ndarray

log_partition(eta=None)

Return the per-element A (the joint scales by the element count).

Parameters:

eta (Any)

Return type:

Any

log_density(x)

Return the reconstructed iid log-density sum_t log p_0(x_t) per sequence.

Parameters:

x (Any)

Return type:

ndarray

class MultinomialExponentialFamilyForm(distribution, element, engine=<mixle.engines.numpy_engine.NumpyEngine object>)

Bases: object

Canonical exponential-family view of a multinomial over an exp-family element.

A multinomial observation is a bag {(v_j, c_j)} of values with counts, and the (non length-normalized, no separate trial distribution) log-density is the multinomial coefficient plus the count-weighted sum of the element log-densities. So the natural parameters are the element’s (eta = eta_0), the sufficient statistic is the count-weighted sum T(x) = sum_j c_j T_0(v_j), log h(x) = log(n! / prod_j c_j!) + sum_j c_j log h_0(v_j), and A is the element’s per-trial partition (the joint scales by the total count n = sum_j c_j). Built by to_exponential_family().

Parameters:

• distribution (ProbabilityDistribution)

• element (ExponentialFamilyForm)

• engine (Any)

property dim: int

Natural-parameter dimension (same as the element family).

natural_parameters()

Return the shared element natural parameters.

Return type:

ndarray

sufficient_statistics(x)

Return the per-observation count-weighted element statistics (n, dim).

Parameters:

x (Any)

Return type:

ndarray

log_partition(eta=None)

Return the per-trial A (the joint scales by the total count).

Parameters:

eta (Any)

Return type:

Any

log_base_measure(x)

Return the multinomial coefficient plus child base measures.

Parameters:

x (Any)

Return type:

ndarray

log_density(x)

Return the normalized multinomial count-vector log mass per observation.

Parameters:

x (Any)

Return type:

ndarray

mean_parameters(**kwargs)

Return the element mean parameters (the per-trial expectation of T).

Parameters:

kwargs (Any)

Return type:

ndarray

class ConditionalExponentialFamilyForm(response_family, natural_fn, log_partition_fn=None, mean_fn=None, dispersion=None, engine=<mixle.engines.numpy_engine.NumpyEngine object>)

Bases: object

Canonical exponential-family view of a conditional model p(y | x).

The response family fixes T, A, and h; natural_parameters(x) supplies the per-row natural parameters eta(x) (the linear predictor for a canonical link). dispersion carries a nuisance/dispersion parameter when the response family has one (e.g. a Gaussian variance).

Parameters:

• response_family (ProbabilityDistribution)

• natural_fn (Any)

• log_partition_fn (Any)

• mean_fn (Any)

• dispersion (Any)

• engine (Any)

natural_parameters(x)

Return the per-row natural parameters eta(x).

Parameters:

x (Any)

Return type:

ndarray

sufficient_statistics(y)

Return the response sufficient statistics T(y).

Parameters:

y (Any)

Return type:

Any

log_base_measure(y)

Return log h(y) for the response family.

Parameters:

y (Any)

Return type:

Any

log_partition(eta)

Return A(eta) row-wise for a stack of natural parameters.

Uses the family-specific log_partition_fn (supplied by the GLM wiring, which knows the response family) when present; otherwise falls back to the response family’s dual map (only available where from_natural is closed-form).

Parameters:

eta (Any)

Return type:

ndarray

mean(x)

Return the conditional mean E[y|x] = link_inv(eta(x)) (the inverse link).

Parameters:

x (Any)

Return type:

ndarray

log_density(y, x)

Return log h(y) + <eta(x), T(y)> - A(eta(x)) row-wise.

Parameters:

• y (Any)

• x (Any)

Return type:

ndarray

to_exponential_family(dist, engine=NUMPY_ENGINE)

Return the canonical exponential-family view of dist or None.

Mirrors mixle.inference.fisher.to_fisher(): a thin top-level helper that defers to ProbabilityDistribution.to_exponential_family(). Returns None when dist is not a (single) exponential family.

Parameters:

• dist (ProbabilityDistribution)

• engine (Any)

Return type:

ExponentialFamilyForm | None

is_exponential_family(dist)

Return whether dist exposes a (single) exponential-family canonical form.

Parameters:

dist (ProbabilityDistribution)

Return type:

bool