mixle.stats.sequences.sparse_markov_transform module¶

Sparse Markov hidden-association models over integer word-count bags.

Data type: Tuple[List[Tuple[int, float]], List[Tuple[int, float]]].

The SparseMarkovAssociation model is a generative model for two sets of words S_1 ={w_{1,1},…,w_{1,n}} and S_2 ={w_{2,1},…,w_{2,m}} over W possible words. The model assumes a hidden set of assignments A_2 = {a_{2,1},…,a_{2,m}} where a_{2,j} takes on values in {1,2,…,m}. The observed likelihood function is computed from P(S_1, S_2) = P(S_2 | S_1) P(S_1), where

log(P(S_2|S_1)) = sum_{i=1}^{m} log(P(w_{2,i}|w_{1,1},…,w_{1,n})
= sum_{i=1}^{m} log( (1/m)*sum_{j=1}^{n} (1-alpha)*P(w_{2,i} | w_{1,j}) + alpha/W).

log(P(S_1)) = sum_{j=1}^{n} log( (1-alpha)*P(w_{1,j} + alpha/W ).

This model is great for problems where one set is given like translations.

class SparseMarkovAssociationDistribution(init_prob_vec, cond_prob_mat, alpha=0.0, len_dist=NullDistribution(), low_memory=False)[source]

Bases: SequenceEncodableProbabilityDistribution

Distribution for a sparse count set S2 generated from a count set S1.

Parameters:

init_prob_vec (Sequence[float] | ndarray)
cond_prob_mat (csr_matrix)
alpha (float)
len_dist (SequenceEncodableProbabilityDistribution | None)
low_memory (bool)

density(x)[source]

Density of the sparse Markov association model at observation x.

See log_density() for details.

Parameters:: x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
Returns:: Density at observation x.
Return type:: float

log_density(x)[source]

Log-density of the sparse Markov association model at observation x.

Computes log(P(S2 | S1)) (see module docstring, eq. (1)) plus the log-density of the total counts [n1, n2] under len_dist.

Parameters:: x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
Returns:: Log-density at observation x.
Return type:: float

seq_log_density(x)[source]

Vectorized evaluation of log-density at sequence encoded input x.

Parameters:: x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
Returns:: Numpy array of log-densities, one per encoded observation.
Return type:: ndarray

compute_capabilities()[source]

Engine readiness for the dense scoring tail (numpy + torch).

The large word-by-word transition matrix is sliced/gathered host-side with SciPy sparse ops, but the per-pair smoothing, log, and segment reductions run on the active engine (see backend_seq_log_density), so the model composes on numpy and torch.

backend_seq_log_density(x, engine)[source]

Engine-routed sparse-association scoring.

The conditional probabilities for the observed word pairs are gathered host-side from the SciPy sparse matrix; the smoothing p*b + a, the logs, and the segment-sum reductions (initial-state and association terms) run on the active engine via index_add. Falls back to the engine-lifted NumPy path for the low-memory encoding that lacks the flat pair index.

Return type:: Any

sampler(seed=None)[source]

Create a sampler for this sparse Markov association distribution.

Parameters:: seed (Optional[int]) – Used to set seed in random sampler.
Returns:: Sampler bound to this distribution.
Return type:: SparseMarkovAssociationSampler

estimator(pseudo_count=None)[source]

Create an estimator initialized from this sparse Markov association distribution.

Parameters:: pseudo_count (Optional[float]) – Kept for protocol compatibility (unused).
Returns:: Estimator configured with matching size and sparsity settings.
Return type:: SparseMarkovAssociationEstimator

dist_to_encoder()[source]

Return a data encoder for sparse Markov association observations.

Return type:: SparseMarkovAssociationDataEncoder

class SparseMarkovAssociationSampler(dist, seed=None)[source]

Bases: DistributionSampler

Sampler for a sparse Markov-association distribution.

Parameters:

dist (SparseMarkovAssociationDistribution)
seed (int | None)

sample(size=None)[source]

Draw ‘size’ iid observations from the sparse Markov association model.

Each observation is a tuple (S1, S2) of lists of (value, count) pairs. If size is None a single observation is returned, else a list of ‘size’ observations is returned.

Parameters:: size (Optional[int]) – Number of observations to draw. Treated as a single draw if None.
Returns:: A single observation tuple, or a list of observation tuples when size is not None.
Return type:: tuple[list[tuple[int, float]], list[tuple[int, float]]] | Sequence[tuple[list[tuple[int, float]], list[tuple[int, float]]]]

class SparseMarkovAssociationAccumulator(num_vals, size_acc=NullAccumulator(), keys=(None, None), low_memory=True)[source]

Bases: InitTransKeyedAccumulator, SequenceEncodableStatisticAccumulator

Accumulator for sparse Markov-association sufficient statistics.

Parameters:

num_vals (int)
size_acc (SequenceEncodableStatisticAccumulator | None)
keys (tuple[str | None, str | None])
low_memory (bool)

update(x, weight, estimate)[source]

Update sufficient statistics with a single weighted observation.

Parameters:

x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
weight (float) – Weight of the observation.
estimate (SparseMarkovAssociationDistribution) – Previous estimate used to assign responsibility.

Returns:

None.

Return type:

None

initialize_rng(rng)[source]

Seed the internal RandomState for the size accumulator from rng (idempotent).

Parameters:: rng (RandomState) – Source of the seed.
Returns:: None.
Return type:: None

initialize(x, weight, rng)[source]

Initialize sufficient statistics with a single weighted observation (no previous estimate).

Parameters:

x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
weight (float) – Weight of the observation.
rng (RandomState) – Used to seed the size accumulator initialization.

Returns:

None.

Return type:

None

seq_initialize(x, weights, rng)[source]

Initialize sufficient statistics with a sequence of weighted encoded observations.

Parameters:

x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
weights (np.ndarray) – Weights, one per encoded observation.
rng (RandomState) – Used to seed the size accumulator initialization.

Returns:

None.

Return type:

None

seq_update(x, weights, estimate)[source]

Update sufficient statistics with a sequence of weighted encoded observations.

Parameters:

x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
weights (np.ndarray) – Weights, one per encoded observation.
estimate (SparseMarkovAssociationDistribution) – Previous estimate used to assign responsibility.

Returns:

None.

Return type:

None

combine(suff_stat)[source]

Merge the sufficient statistics of another accumulator into this one.

Parameters:: suff_stat (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – Tuple (init_count, trans_count, size_value) from another accumulator’s value().
Returns:: This SparseMarkovAssociationAccumulator object.
Return type:: SparseMarkovAssociationAccumulator

value()[source]

Returns the sufficient statistic tuple (init_count, trans_count, size_value).

Return type:: tuple[ndarray, lil_matrix | csr_matrix | None, Any]

from_value(x)[source]

Set the sufficient statistics from a value() tuple.

Parameters:: x (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – Tuple (init_count, trans_count, size_value).
Returns:: This SparseMarkovAssociationAccumulator object.
Return type:: SparseMarkovAssociationAccumulator

acc_to_encoder()[source]

Return a data encoder built from the size accumulator.

Return type:: SparseMarkovAssociationDataEncoder

class SparseMarkovAssociationAccumulatorFactory(num_vals, len_factory=NullAccumulatorFactory(), low_memory=True, keys=(None, None))[source]

Bases: StatisticAccumulatorFactory

Factory for sparse Markov association accumulators.

Parameters:

num_vals (int)
len_factory (StatisticAccumulatorFactory | None)
low_memory (bool)
keys (tuple[str | None, str | None])

make()[source]

Return a new sparse Markov association accumulator.

Return type:: SparseMarkovAssociationAccumulator

class SparseMarkovAssociationEstimator(num_vals=MISSING, alpha=0.0, len_estimator=NullEstimator(), suff_stat=None, pseudo_count=None, low_memory=True, keys=(None, None), num_values=MISSING)[source]

Bases: ParameterEstimator

Estimate sparse Markov association distributions from sufficient statistics.

Parameters:

num_vals (int)
alpha (float)
len_estimator (ParameterEstimator | None)
suff_stat (Any | None)
pseudo_count (float | None)
low_memory (bool)
keys (tuple[str | None, str | None])
num_values (int)

accumulator_factory()[source]

Return an accumulator factory configured from this estimator.

Return type:: SparseMarkovAssociationAccumulatorFactory

estimate(nobs, suff_stat)[source]

Estimate a sparse Markov association distribution from aggregated sufficient statistics.

Arg suff_stat is a Tuple of length 3 containing:: suff_stat[0] (np.ndarray): Weighted counts for the initial states P(S1). suff_stat[1] (Optional[Union[lil_matrix, csr_matrix]]): Counts for transitions used to estimate P(S2|S1). suff_stat[2] (SS1): Sufficient statistics from the accumulator of the size/len distribution.

Parameters:

nobs (Optional[float]) – Weighted number of observations.
suff_stat (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – See above for details.

Returns:

SparseMarkovAssociationDistribution.

Return type:

SparseMarkovAssociationDistribution

class SparseMarkovAssociationDataEncoder(len_encoder, low_memory)[source]

Bases: DataSequenceEncoder

Encode sparse Markov association observations for vectorized scoring.

Parameters:

len_encoder (DataSequenceEncoder)
low_memory (bool)

seq_encode(x)[source]

Encode a sequence of observations for vectorized calls.

Parameters:: x (Sequence[tuple[list[tuple[int, float]], list[tuple[int, float]]]]) – Sequence of observation tuples (S1, S2), each a list of (value, count) pairs.
Returns:: Tuple (rv, nn, vv, qq) where rv holds per-observation (values, counts) arrays, nn is the encoded length data, vv is the array of distinct (u, v) pairs, and qq holds flattened pair-index arrays for the vectorized path (None when low_memory is True).