mixle.stats.sequences.sparse_markov_transform module¶

Create, estimate, and sample from an integer sparse Markov hidden association model.

Defines the SparseMarkovAssociationDistribution, SparseMarkovAssociationSampler, SparseMarkovAssociationAccumulatorFactory, SparseMarkovAssociationAccumulator, SparseMarkovAssociationEstimator, and the SparseMarkovAssociationDataEncoder classes for use with mixle.

Data type: Tuple[List[Tuple[int, float]], List[Tuple[int, float]]].

The SparseMarkovAssociation model is a generative model for two sets of words S_1 ={w_{1,1},…,w_{1,n}} and S_2 ={w_{2,1},…,w_{2,m}} over W possible words. The model assumes a hidden set of assignments A_2 = {a_{2,1},…,a_{2,m}} where a_{2,j} takes on values in {1,2,…,m}. The observed likelihood function is computed from P(S_1, S_2) = P(S_2 | S_1) P(S_1), where

log(P(S_2|S_1)) = sum_{i=1}^{m} log(P(w_{2,i}|w_{1,1},…,w_{1,n})
= sum_{i=1}^{m} log( (1/m)*sum_{j=1}^{n} (1-alpha)*P(w_{2,i} | w_{1,j}) + alpha/W).

log(P(S_1)) = sum_{j=1}^{n} log( (1-alpha)*P(w_{1,j} + alpha/W ).

This model is great for problems where one set is given like translations.

class SparseMarkovAssociationDistribution(init_prob_vec, cond_prob_mat, alpha=0.0, len_dist=NullDistribution(), low_memory=False)[source]

Bases: SequenceEncodableProbabilityDistribution

SparseMarkovAssociationDistribution object modeling a count-set S2 generated from a count-set S1.

Parameters:

init_prob_vec (Sequence[float] | ndarray)
cond_prob_mat (csr_matrix)
alpha (float)
len_dist (SequenceEncodableProbabilityDistribution | None)
low_memory (bool)

density(x)[source]

Density of the sparse Markov association model at observation x.

See log_density() for details.

Parameters:: x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
Returns:: Density at observation x.
Return type:: float

log_density(x)[source]

Log-density of the sparse Markov association model at observation x.

Computes log(P(S2 | S1)) (see module docstring, eq. (1)) plus the log-density of the total counts [n1, n2] under len_dist.

Parameters:: x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
Returns:: Log-density at observation x.
Return type:: float

seq_log_density(x)[source]

Vectorized evaluation of log-density at sequence encoded input x.

Parameters:: x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
Returns:: Numpy array of log-densities, one per encoded observation.
Return type:: ndarray

compute_capabilities()[source]

Engine readiness for the dense scoring tail (numpy + torch).

The large word-by-word transition matrix is sliced/gathered host-side with SciPy sparse ops, but the per-pair smoothing, log, and segment reductions run on the active engine (see backend_seq_log_density), so the model composes on numpy and torch.

backend_seq_log_density(x, engine)[source]

Engine-routed sparse-association scoring.

The conditional probabilities for the observed word pairs are gathered host-side from the SciPy sparse matrix; the smoothing p*b + a, the logs, and the segment-sum reductions (initial-state and association terms) run on the active engine via index_add. Falls back to the engine-lifted NumPy path for the low-memory encoding that lacks the flat pair index.

Return type:: Any

sampler(seed=None)[source]

Create a SparseMarkovAssociationSampler object from this instance.

Parameters:: seed (Optional[int]) – Used to set seed in random sampler.
Returns:: SparseMarkovAssociationSampler object.
Return type:: SparseMarkovAssociationSampler

estimator(pseudo_count=None)[source]

Create a SparseMarkovAssociationEstimator object from this instance.

Parameters:: pseudo_count (Optional[float]) – Kept for protocol compatibility (unused).
Returns:: SparseMarkovAssociationEstimator object.
Return type:: SparseMarkovAssociationEstimator

dist_to_encoder()[source]

Returns a SparseMarkovAssociationDataEncoder object for encoding sequences of data.

Return type:: SparseMarkovAssociationDataEncoder

class SparseMarkovAssociationSampler(dist, seed=None)[source]

Bases: DistributionSampler

SparseMarkovAssociationSampler object for sampling from a SparseMarkovAssociationDistribution.

Parameters:

dist (SparseMarkovAssociationDistribution)
seed (int | None)

sample(size=None)[source]

Draw ‘size’ iid observations from the sparse Markov association model.

Each observation is a tuple (S1, S2) of lists of (value, count) pairs. If size is None a single observation is returned, else a list of ‘size’ observations is returned.

Parameters:: size (Optional[int]) – Number of observations to draw. Treated as a single draw if None.
Returns:: A single observation tuple, or a list of observation tuples when size is not None.
Return type:: tuple[list[tuple[int, float]], list[tuple[int, float]]] | Sequence[tuple[list[tuple[int, float]], list[tuple[int, float]]]]

class SparseMarkovAssociationAccumulator(num_vals, size_acc=NullAccumulator(), keys=(None, None), low_memory=True)[source]

Bases: InitTransKeyedAccumulator, SequenceEncodableStatisticAccumulator

SparseMarkovAssociationAccumulator object for accumulating sufficient statistics of the model.

Parameters:

num_vals (int)
size_acc (SequenceEncodableStatisticAccumulator | None)
keys (tuple[str | None, str | None])
low_memory (bool)

update(x, weight, estimate)[source]

Update sufficient statistics with a single weighted observation.

Parameters:

x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
weight (float) – Weight of the observation.
estimate (SparseMarkovAssociationDistribution) – Previous estimate used to assign responsibility.

Returns:

None.

Return type:

None

initialize_rng(rng)[source]

Seed the internal RandomState for the size accumulator from rng (idempotent).

Parameters:: rng (RandomState) – Source of the seed.
Returns:: None.
Return type:: None

initialize(x, weight, rng)[source]

Initialize sufficient statistics with a single weighted observation (no previous estimate).

Parameters:

x (tuple[list[tuple[int, float]], list[tuple[int, float]]]) – Observation tuple (S1, S2), each a list of (value, count) pairs.
weight (float) – Weight of the observation.
rng (RandomState) – Used to seed the size accumulator initialization.

Returns:

None.

Return type:

None

seq_initialize(x, weights, rng)[source]

Initialize sufficient statistics with a sequence of weighted encoded observations.

Parameters:

x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
weights (np.ndarray) – Weights, one per encoded observation.
rng (RandomState) – Used to seed the size accumulator initialization.

Returns:

None.

Return type:

None

seq_update(x, weights, estimate)[source]

Update sufficient statistics with a sequence of weighted encoded observations.

Parameters:

x – Encoded sequence (from SparseMarkovAssociationDataEncoder.seq_encode).
weights (np.ndarray) – Weights, one per encoded observation.
estimate (SparseMarkovAssociationDistribution) – Previous estimate used to assign responsibility.

Returns:

None.

Return type:

None

combine(suff_stat)[source]

Merge the sufficient statistics of another accumulator into this one.

Parameters:: suff_stat (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – Tuple (init_count, trans_count, size_value) from another accumulator’s value().
Returns:: This SparseMarkovAssociationAccumulator object.
Return type:: SparseMarkovAssociationAccumulator

value()[source]

Returns the sufficient statistic tuple (init_count, trans_count, size_value).

Return type:: tuple[ndarray, lil_matrix | csr_matrix | None, Any]

from_value(x)[source]

Set the sufficient statistics from a value() tuple.

Parameters:: x (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – Tuple (init_count, trans_count, size_value).
Returns:: This SparseMarkovAssociationAccumulator object.
Return type:: SparseMarkovAssociationAccumulator

acc_to_encoder()[source]

Returns a SparseMarkovAssociationDataEncoder object for encoding sequences of data.

Return type:: SparseMarkovAssociationDataEncoder

class SparseMarkovAssociationAccumulatorFactory(num_vals, len_factory=NullAccumulatorFactory(), low_memory=True, keys=(None, None))[source]

Bases: StatisticAccumulatorFactory

SparseMarkovAssociationAccumulatorFactory object for creating SparseMarkovAssociationAccumulator objects.

Parameters:

num_vals (int)
len_factory (StatisticAccumulatorFactory | None)
low_memory (bool)
keys (tuple[str | None, str | None])

make()[source]

Returns a new SparseMarkovAssociationAccumulator object.

Return type:: SparseMarkovAssociationAccumulator

class SparseMarkovAssociationEstimator(num_vals=MISSING, alpha=0.0, len_estimator=NullEstimator(), suff_stat=None, pseudo_count=None, low_memory=True, keys=(None, None), num_values=MISSING)[source]

Bases: ParameterEstimator

SparseMarkovAssociationEstimator object for estimating SparseMarkovAssociationDistribution objects.

Parameters:

num_vals (int)
alpha (float)
len_estimator (ParameterEstimator | None)
suff_stat (Any | None)
pseudo_count (float | None)
low_memory (bool)
keys (tuple[str | None, str | None])
num_values (int)

accumulator_factory()[source]

Returns a SparseMarkovAssociationAccumulatorFactory object for this estimator.

Return type:: SparseMarkovAssociationAccumulatorFactory

estimate(nobs, suff_stat)[source]

Estimate SparseMarkovAssociationDistribution objects from aggregated sufficient statistics.

Arg suff_stat is a Tuple of length 3 containing:: suff_stat[0] (np.ndarray): Weighted counts for the initial states P(S1). suff_stat[1] (Optional[Union[lil_matrix, csr_matrix]]): Counts for transitions used to estimate P(S2|S1). suff_stat[2] (SS1): Sufficient statistics from the accumulator of the size/len distribution.

Parameters:

nobs (Optional[float]) – Weighted number of observations.
suff_stat (tuple[ndarray, lil_matrix | csr_matrix | None, SS1]) – See above for details.

Returns:

SparseMarkovAssociationDistribution.

Return type:

SparseMarkovAssociationDistribution

class SparseMarkovAssociationDataEncoder(len_encoder, low_memory)[source]

Bases: DataSequenceEncoder

SparseMarkovAssociationDataEncoder object for encoding sequences of (S1, S2) count-set observations.

Parameters:

len_encoder (DataSequenceEncoder)
low_memory (bool)

seq_encode(x)[source]

Encode a sequence of observations for vectorized calls.

Parameters:: x (Sequence[tuple[list[tuple[int, float]], list[tuple[int, float]]]]) – Sequence of observation tuples (S1, S2), each a list of (value, count) pairs.
Returns:: Tuple (rv, nn, vv, qq) where rv holds per-observation (values, counts) arrays, nn is the encoded length data, vv is the array of distinct (u, v) pairs, and qq holds flattened pair-index arrays for the vectorized path (None when low_memory is True).