mixle.stats.multivariate.categorical_multinomial module

“Create, estimate, and sample from a MultinomialDistribution.

Defines the MultinomialDistribution, MultinomialSampler, MultinomialAccumulatorFactory, MultinomialAccumulator, MultinomialEstimator, and the MultinomialDataEncoder classes for use with mixle.

Let P_dist(V_k) be a distribution for a countable set of discrete observations of values V_k of type T. Denote

p_k = P_dist(V_k),

as the probability of success for value V_k. Then sum_{k=0}^{inf} p_k = 1. Let x = (x_0, x_1,….,x_{n-1}) be a multinomial observation for a ‘n’ trials, where each x_i = (V_j, n_j) for some value V_j in the observation space and n_j is the associated number of success for the value. (note: sum n_j = n). Then, denoting p_j = p_mat(V_j), we score the un-normalized log-density:

log(p_mat(x)) = sum_{j=0}^{n-1} n_j * log(p_j) + log(P_len(n)),

where P_len(n) is a distribution for the number of trials in the multinomial having support on the non-negative integers. The multinomial coefficient (log(n!) - sum_j log(n_j!)) is intentionally omitted, so this is a per-category scoring form rather than a normalized probability mass over count vectors.

The multinomial is assumed to have data type: Sequence[Tuple[T, float]], where T is the data type of the ‘categories’.

class MultinomialDistribution(dist, len_dist=NullDistribution(), len_normalized=False, name=None)

Bases: SequenceEncodableProbabilityDistribution

Multinomial distribution over count vectors.

Parameters:

• dist (SequenceEncodableProbabilityDistribution)

• len_dist (SequenceEncodableProbabilityDistribution | None)

• len_normalized (bool)

• name (str | None)

compute_capabilities()

compute_declaration()

to_exponential_family(engine=None)

Return the multinomial exponential-family view, or None.

A multinomial over an exponential-family element is itself an exponential family with the shared element eta and the count-weighted sufficient statistic T(x) = sum_j c_j T_0(v_j). This holds only when the trial count is not separately modeled (len_dist is Null) and the density is not length-normalized (those break the single-exp-family form); it also requires the value element to be an exponential family. Otherwise returns None.

Parameters:

engine (Any)

density(x)

Returns the density of multinomial evaluated at observation x.

See log_density() for details.

Parameters:

x (Sequence[Tuple[T, float]]) – Tuples of observed multinomial values and success s.t. success sum to number of trials.

Returns:

Density evaluated at x.

Return type:

float

log_density(x)

Returns the log-density of multinomial evaluated at observation x.

Let P_dist(V_k) be a distribution for a countable set of discrete observations of values V_k of type T. Denote

p_k = P_dist(V_k),

as the probability of success for value V_k. Then sum_{k=0}^{inf} p_k = 1. Let x = (x_0, x_1,….,x_{n-1}) be a multinomial observation for a ‘n’ trials, where each x_i = (V_j, n_j) for some value V_j in the observation space and n_j is the associated number of success for the value. (note: sum n_j = n). Then, denoting p_j = p_mat(V_j), we score the un-normalized log-density:

log(p_mat(x)) = sum_{j=0}^{n-1} n_j * log(p_j) + log(P_len(n)),

where P_len(n) is a distribution for the number of trials in the multinomial having support on the non-negative integers. The multinomial coefficient is intentionally omitted (see the module docstring), so this is a per-category scoring form, not a normalized mass over count vectors.

Parameters:

x (Sequence[Tuple[T, float]]) – Tuples of observed multinomial values and success s.t. success sum to number of trials.

Returns:

Log-density evaluated at x.

Return type:

float

seq_log_density(x)

Vectorized evaluated of log-density for an encoded sequence of iid multinomial observations.

See log_density() for details on the log-density function for MultinomialDistribution.

Arg ‘x’ is a tuple of size 7 containing:

x[0] (ndarray[int]): Observation index of sequence values. x[1] (ndarray[float]): Trial size for each observation. x[2] (ndarray[float]): Non-zero trial size indices. x[3] (T1): Sequence encoded flattened list of values from x. x[4] (Optional[T2]): Sequence encoded flatted list of trial sizes. x[5] (np.ndarray[float]): Flattened array of counts for values. x[6] (ndarray[float]): Flattened array of trial sizes.

Parameters:

x – See above for details.

Returns:

Numpy array of the log-density at each encoded observation of x.

Return type:

ndarray

backend_seq_log_density(x, engine)

Engine-neutral vectorized log-density for encoded count-vector observations.

Parameters:

engine (Any)

Return type:

Any

classmethod backend_stacked_params(dists, engine)

Return stacked child routes for homogeneous multinomial mixtures.

Parameters:

• dists (Sequence[MultinomialDistribution])

• engine (Any)

Return type:

dict[str, Any]

classmethod backend_stacked_log_density(x, params, engine)

Return an (n, k) matrix of multinomial log densities.

Parameters:

• x (tuple[Any, ...])

• params (dict[str, Any])

• engine (Any)

Return type:

Any

classmethod backend_stacked_sufficient_statistics_with_estimator(x, weights, params, engine, estimator)

Return per-component legacy multinomial sufficient statistics.

Parameters:

• x (tuple[Any, ...])

• weights (Any)

• params (dict[str, Any])

• engine (Any)

• estimator (Any)

Return type:

tuple[Any, …]

to_fisher(**kwargs)

Structural Fisher view for the multinomial bag.

sampler(seed=None)

Create a MultinomialSampler object from MultinomialDistribution object instance.

Parameters:

seed (Optional[int]) – Set the seed for sampling from MultinomialDistribution.

Returns:

MultinomialSampler object.

Return type:

MultinomialSampler

estimator(pseudo_count=None)

Create an MultinomialEstimator object from an MultinomialDistribution object instance.

Parameters:

pseudo_count (Optional[float]) – Re-weight member sufficient statistics when estimating from aggregated data.

Returns:

MultinomialEstimator object.

Return type:

MultinomialEstimator

dist_to_encoder()

Create a MultinomialDataEncoder object from object instance.

Return type:

MultinomialDataEncoder

enumerator()

Returns MultinomialEnumerator iterating (value, count)-pair lists in descending probability order.

Return type:

MultinomialEnumerator

class MultisetProductEnumerator(stream, n, combine=list, offset=0.0)

Bases: object

Best-first enumeration of the size-n multisets drawn from a sorted (value, log_prob) stream.

Yields (combine(pairs), log_prob) where pairs is a tuple of (value, count) entries with distinct values and counts summing to n, and log_prob = offset + the sum of the n chosen element log probs, in non-increasing order. Each multiset is represented exactly once as a non-decreasing tuple of ranks into the shared BufferedStream; successors increment a single rank while preserving sorted order (only the right-most rank of a run of equal ranks may move), which keeps every multiset reachable exactly once and, because the stream is sorted, makes successor scores monotone non-increasing.

Parameters:

• stream (BufferedStream)

• n (int)

• combine (Callable[[tuple[tuple[Any, int], ...]], Any])

• offset (float)

class MultinomialEnumerator(dist)

Bases: DistributionEnumerator

Enumerates multinomial observations (lists of (value, count) pairs) in descending probability order.

Parameters:

dist (MultinomialDistribution)

class MultinomialSampler(dist, seed=None)

Bases: DistributionSampler

Parameters:

• dist (MultinomialDistribution)

• seed (int | None)

sample(size=None)

Draw samples from multinomial distribution.

Note: If len_sampler can draw n=0, an empty list is returned for that sample.

Parameters:

size (Optional[int]) – Number of iid samples to draw from multinomial.

Returns:

Sequence of ‘size’ iid observations if size is not None, else a single multinomial sample.

Return type:

Sequence[Sequence[tuple[Any, float]]] | Sequence[tuple[Any, float]]

class MultinomialAccumulator(accumulator, len_normalized, len_accumulator=NullAccumulator(), keys=None)

Bases: SequenceEncodableStatisticAccumulator

Parameters:

• accumulator (SequenceEncodableStatisticAccumulator)

• len_normalized (bool)

• len_accumulator (SequenceEncodableStatisticAccumulator | None)

• keys (str | None)

update(x, weight, estimate)

Update the sufficient statistics of MultinomialAccumulator object instance with single obseration x.

Parameters:

• x (Sequence[Tuple[T, float]]) – A single observation of multinomial distribution.

• weight (float) – Observation weight.

• estimate (Optional[MultinomialDistribution]) – Optional previous estimate for multinomial distribution.

Returns:

None.

Return type:

None

initialize(x, weight, rng)

Parameters:

• x (Sequence[Tuple[T, float]]) – A single observation of multinomial distribution.

• weight (float) – Observation weight.

• rng (Optional[RandomState]) – RandomState object for initializing random number generator.

Return type:

None

Returns:

seq_update(x, weights, estimate)

Vectorized update of encoded sequence of iid observations from multinomial distribution.

Arg ‘x’ is a tuple of size 7 containing:

x[0] (ndarray[int]): Observation index of sequence values. x[1] (ndarray[float]): Trial size for each observation. x[2] (ndarray[float]): Non-zero trial size indices. x[3] (T1): Sequence encoded flattened list of values from x. x[4] (Optional[T2]): Sequence encoded flatted list of trial sizes. x[5] (np.ndarray[float]): Flattened array of counts for values. x[6] (ndarray[float]): Flattened array of trial sizes.

Parameters:

• x – See above for details.

• weights (np.ndarray) – Array of observation weights.

• estimate (Optional[MultinomialDistribution]) – Optional previous estimate for multinomial distribution.

Returns:

None.

Return type:

None

seq_update_engine(x, weights, estimate, engine)

Engine-resident E-step: the per-value and per-length weights are formed on the active engine and the value/length accumulators are routed through the engine. Matches seq_update.

Parameters:

• weights (Any)

• estimate (MultinomialDistribution | None)

• engine (Any)

Return type:

None

seq_initialize(x, weights, rng)

Vectorized initialization of of sufficient statistics for an encoded sequence of observations.

Arg ‘x’ is a tuple of size 7 containing:

x[0] (ndarray[int]): Observation index of sequence values. x[1] (ndarray[float]): Trial size for each observation. x[2] (ndarray[float]): Non-zero trial size indices. x[3] (T1): Sequence encoded flattened list of values from x. x[4] (Optional[T2]): Sequence encoded flatted list of trial sizes. x[5] (np.ndarray[float]): Flattened array of counts for values. x[6] (ndarray[float]): Flattened array of trial sizes.

Parameters:

• x – See above for details.

• weights (np.ndarray) – Numpy array of observation weights.

• rng (RandomState) – RandomState object for setting seed.

Returns:

None.

Return type:

None

combine(suff_stat)

Combine the sufficient statistics of object instance with aggregated sufficient statistics in ‘suff_stat’.

Parameters:

suff_stat (Tuple[SS1, Optional[SS2]]) – Contains sufficient statistics for value distribution (SS1) and sufficient statistic for length distribution (SS2).

Returns:

MultinomialAccumulator object.

Return type:

MultinomialAccumulator

value()

Return object instance sufficient statistics as Tuple[SS1, Optional[SS2]].

Return type:

tuple[Any, Any | None]

from_value(x)

Set object instance sufficient statistics to arg ‘x’.

Parameters:

x (Tuple[SS1, Optional[SS2]]) – Contains sufficient statistics for value distribution (SS1) and sufficient statistic for length distribution (SS2).

Returns:

MultinomialAccumulator object.

Return type:

MultinomialAccumulator

scale(c)

Scale value and length sufficient statistics through their accumulators.

Parameters:

c (float)

Return type:

MultinomialAccumulator

key_merge(stats_dict)

Merge the sufficient statistics of object instance with matching keys of stats_dict.

Parameters:

stats_dict (Dict[str, Any]) – Maps keys to sufficient statistics.

Returns:

None.

Return type:

None

key_replace(stats_dict)

Replace the sufficient statistics of object instance with matching keys in stats_dict.

Parameters:

stats_dict (Dict[str, Any]) – Maps keys to sufficient statistics.

Returns:

None.

Return type:

None

acc_to_encoder()

Create a MultinomialDataEncoder object from object instance.

Return type:

MultinomialDataEncoder

class MultinomialAccumulatorFactory(est_factory, len_normalized, len_factory=NullAccumulatorFactory(), keys=None)

Bases: StatisticAccumulatorFactory

Parameters:

• est_factory (StatisticAccumulatorFactory)

• len_normalized (bool)

• len_factory (StatisticAccumulatorFactory)

• keys (str | None)

make()

Returns MultinomialAccumulator object.

Return type:

MultinomialAccumulator

class MultinomialEstimator(estimator, len_estimator=NullEstimator(), len_dist=None, len_normalized=False, name=None, keys=None)

Bases: ParameterEstimator

Parameters:

• estimator (ParameterEstimator)

• len_estimator (ParameterEstimator | None)

• len_dist (SequenceEncodableProbabilityDistribution | None)

• len_normalized (bool | None)

• name (str | None)

• keys (str | None)

accumulator_factory()

Create MultinomialAccumulatorFactory object from MultinomialEstimator object instance.

Return type:

MultinomialAccumulatorFactory

estimate(nobs, suff_stat)

Estimate a MultinomialDistribution object from aggregated data contained in arg ‘suff_stat’.

Parameters:

• nobs (Optional[float]) – Number of observations used in aggregation of ‘suff_stat’.

• suff_stat (Tuple[SS1, Optional[SS2]]) – Tuple of sufficient statistics for distribution of values and trial distribution.

Returns:

MultinomialDistribution object.

Return type:

MultinomialDistribution

class MultinomialDataEncoder(encoder, len_encoder)

Bases: DataSequenceEncoder

Parameters:

• encoder (DataSequenceEncoder)

• len_encoder (DataSequenceEncoder)

seq_encode(x)

Encode a sequence of iid observations of multinomial distribution for use with vectorized functions.

Returns a tuple of size 7 containing:

rv1 (ndarray[int]): Observation index of sequence values. rv2 (ndarray[float]): Trial size for each observation. rv3 (ndarray[float]): Non-zero trial size indices. rv4 (T1): Sequence encoded flattened list of values from x. rv5 (Optional[T2]): Sequence encoded flatted list of trial sizes. rv6 (np.ndarray[float]): Flattened array of counts for values. rv7 (ndarray[float]): Flattened array of trial sizes.

Parameters:

x (Sequence[Sequence[Tuple[T, float]]]) – Sequence of iid observations of multinomial distributions.

Returns:

See above.

class MultinomialFisherView(dist)

Bases: SequenceFisherView

Fisher view for bag/count observations with a count-weighted child model.

The model Fisher uses the canonical multinomial/count sufficient-statistic moments that match estimation. The repo’s MultinomialDistribution log_density intentionally omits the multinomial coefficient in its enumerator score; that coefficient is a base-measure term, not an accumulator statistic.

Parameters:

dist (Any)