mixle.stats.latent.integer_probabilistic_latent_semantic_indexing module

Integer probabilistic latent semantic indexing models.

An observation is a document id paired with a sparse integer bag of word/value counts:

(doc_id, [(value_id, count), ...])

For S latent topics, V word values, and D document ids, the model uses:

• state_word_mat[v, s] = p(value=v | topic=s);

• doc_state_mat[d, s] = p(topic=s | document=d);

• doc_vec[d] = p(document=d); and

• an optional length model for total bag count.

The log-density combines the document prior, the optional length density, and the topic-marginalized word probabilities for each sparse count entry. Caller data should use stable integer ids for documents and word values.

class IntegerProbabilisticLatentSemanticIndexingDistribution(state_word_mat, doc_state_mat, doc_vec, len_dist=NullDistribution(), name=None)

Bases: SequenceEncodableProbabilityDistribution

Integer-valued probabilistic latent semantic indexing distribution.

Parameters:

• state_word_mat (list[list[float]] | ndarray)

• doc_state_mat (list[list[float]] | ndarray)

• doc_vec (list[float] | ndarray)

• len_dist (SequenceEncodableProbabilityDistribution | None)

• name (str | None)

compute_capabilities()

Return backend capability metadata for this concrete PLSI instance.

compute_declaration()

Return the symbolic distribution declaration for code generation and PPL introspection.

density(x)

Evaluate the density of PLSI model for an observation x.

See log_density() for details on the density evaluation.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – Single observation of integer PLSI.

Returns:

Density evaluated at observed value x.

Return type:

float

log_density(x)

Evaluate the log-density of PLSI model for an observation of x.

Consider an Integer PLSI model for a corpus of documents with S states, V word values, and D documents ids (authors).

Let x (Tuple[int, Sequence[Tuple[int, float]]]) be an observation from a PLSI model, consisting of x = (d, [(v_0, c_0), (v_1, c_1), …, (v_{k-1}, c_{k-1})]), where the ‘d’ is some document d_id in the corpus and each tuple (v_i, c_i) corresponds to a value-count couple in the corpus. The log-likelihood is given by

log(p_mat(x)) = log(p_mat(d)) + sum_{j=0}^{k-1} c_k*log( sum_{s=0}^{S-1} p_mat(d|s)p_mat(s|v_k) ) + log(P_len(nn)),

where P_len(nn) is the density of the length distribution for ‘nn’ representing the total number of words in the document.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – (doc_id, [(value_id, count_for_value)]). See above for details.

Returns:

Log-density evaluated at a single observation x.

Return type:

float

component_log_density(x)

Evaluate the log-density for each state in the PLSI.

Returns count*log(p_mat(W|S)) for each word-count pair in the document. Returned value is S by 1 where S is the number of components in the model.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – Single PLSI observation of form (doc_id, [(value_id, count_for_value)]).

Returns:

Numpy array of length S (num_states).

Return type:

ndarray

seq_log_density(x)

Vectorized evaluation of the log-density for an encoded sequence of iid observation from a PLSI model.

See log_density() function for details on the log-likelihood.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

Returns:

Numpy array of log-density evaluated at each observation in the encoded sequence.

Return type:

ndarray

backend_seq_log_density(x, engine)

Evaluate encoded PLSI log densities using a backend-neutral compute engine.

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]])

• engine (Any)

Return type:

Any

seq_component_log_density(x)

Vectorized evaluation of the component log-density for each observation in an encoded sequence of iid PLSI

observations.

See component_log_density() function for details on component log-likelihood evaluation.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

Returns:

2-d numpy array containing N rows of num_state sized arrays.

Return type:

ndarray

enumerator()

Enumerate PLSI observations (doc_id, bag) in descending probability order.

A PLSI observation factors as P(doc) * [prod_w q_d(w)^{c_w}] * P_len(n) where q_d is the per-document word distribution prob_mat @ state_mat[d] and n the total word count, so it is a document-labelled mixture of trial-count multinomials: for each document the bags enumerate by a multiset best-first search under a length frontier driven by len_dist (the real trial-count distribution), and the per-document streams are merged by descending score with the document log-probability as offset. Requires a modelled len_dist unless every per-document word distribution is sub-stochastic-free; an absent length distribution leaves the bag support infinite and is enumerated by the multinomial term alone.

Return type:

DistributionEnumerator

sampler(seed=None)

Return a sampler for iid integer PLSI observations.

Parameters:

seed (int | None)

Return type:

IntegerProbabilisticLatentSemanticIndexingSampler

estimator(pseudo_count=None)

Return an estimator initialized from this distribution’s dimensions.

Parameters:

pseudo_count (float | None) – Optional smoothing count for topic, word, and document counts.

Returns:

A configured integer PLSI estimator.

Return type:

IntegerProbabilisticLatentSemanticIndexingEstimator

dist_to_encoder()

Return an encoder for integer PLSI observations.

Return type:

IntegerProbabilisticLatentSemanticIndexingDataEncoder

multinomial_bag_stream(log_p_vec, min_val, len_dist, combine)

Enumerate integer count-vector bags in descending sum_w c_w*log p_w + log P_len(n) order.

Reuses the per-size multiset best-first search (MultisetProductEnumerator) under a length frontier driven by len_dist (the real trial-count distribution); combine maps the tuple of (value, count) pairs to the emitted bag. When len_dist is Null there is no length term and a synthetic n*log p_max frontier orders the (countably infinite) support by the multinomial term alone – matching IntegerMultinomialEnumerator. Shared by the coupled bag-of-counts models.

bag_stream(element_stream, len_dist, combine)

Enumerate bags (multisets) drawn from a sorted element stream, in descending bag-score order.

element_stream is a descending (value, log_prob) iterator over the element distribution (any enumerable element distribution – a fixed categorical, or e.g. a per-document/per-given mixture). A bag scores by the sum of its elements’ log-probs plus log P_len(n) from len_dist; bags enumerate by the per-size multiset best-first search (MultisetProductEnumerator) under a length frontier. When len_dist is Null there is no length term and a synthetic n*log p_max frontier orders the (countably infinite) support by the element term alone. combine maps the tuple of (value, count) pairs to the emitted bag.

class IntegerProbabilisticLatentSemanticIndexingEnumerator(dist)

Bases: DistributionEnumerator

Best-first enumerator for document-labelled integer PLSI bag observations.

Parameters:

dist (IntegerProbabilisticLatentSemanticIndexingDistribution)

class IntegerProbabilisticLatentSemanticIndexingSampler(dist, seed=None)

Bases: DistributionSampler

Sampler for integer PLSI document ids, word-count bags, and document lengths.

Parameters:

• dist (IntegerProbabilisticLatentSemanticIndexingDistribution)

• seed (int | None)

sample(size=None)

Generate iid samples from PLSI model.

Parameters:

size (Optional[int]) – Number of samples to generate. Defaults to 0 if size is None.

Returns:

Sequence of iid PLSI samples if size is not None, else a single sample from PLSI model.

Return type:

tuple[int, Sequence[tuple[int, float]]] | Sequence[tuple[int, Sequence[tuple[int, float]]]]

class IntegerProbabilisticLatentSemanticIndexingAccumulator(num_vals, num_states, num_docs, len_acc=NullAccumulator(), name=None, keys=(None, None, None))

Bases: SequenceEncodableStatisticAccumulator

EM sufficient-statistic accumulator for integer PLSI word, state, document, and length terms.

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_acc (SequenceEncodableStatisticAccumulator | None)

• name (str | None)

• keys (tuple[str | None, str | None, str | None] | None)

update(x, weight, estimate)

Update sufficient statistics from one weighted sparse-bag observation.

Parameters:

• x (tuple[int, Sequence[tuple[int, float]]]) – Integer PLSI observation as (doc_id, [(value_id, count), ...]).

• weight (float) – Observation weight.

• estimate (IntegerProbabilisticLatentSemanticIndexingDistribution) – Previous integer PLSI estimate.

Return type:

None

initialize(x, weight, rng)

Initialize sufficient statistics from one weighted sparse-bag observation.

Parameters:

• x (tuple[int, Sequence[tuple[int, float]]]) – Integer PLSI observation as (doc_id, [(value_id, count), ...]).

• weight (float) – Observation weight.

• rng (RandomState) – Random state used for seeded initialization.

Return type:

None

seq_initialize(x, weights, rng)

Vectorized initialization of sufficient statistics form an encoded sequence of observations in arg ‘x’.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI. See above for details.

• weights (ndarray) – Weights for observations in encoded sequence.

• rng (RandomState) – Used to initialize member RandomState variables.

Returns:

None.

Return type:

None

seq_update(x, weights, estimate)

Vectorized update of sufficient statistics for encoded sequence of iid observations in x.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

• weights (np.ndarray) – Numpy array of observation weights.

• estimate (IntegerProbabilisticLatentSemanticIndexingDistribution) – Prior estimate of IntegerProbabilisticLatentSemanticIndexingDistribution object.

Returns:

None.

Return type:

None

seq_update_engine(x, weights, estimate, engine)

Engine-resident E-step: the PLSI responsibility update (state-word x doc-state gather, per-pair normalization, and the word/doc segment sums) runs on the active engine, matching the host seq_update.

combine(suff_stat)

Merge aggregated integer PLSI sufficient statistics into this accumulator.

The tuple is interpreted as (word_count, comp_count, doc_count, length_stats).

Parameters:

suff_stat (tuple[ndarray, ndarray, ndarray, SS1 | None]) – Aggregated integer PLSI sufficient statistics.

Returns:

This accumulator.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

value()

Return sufficient statistics as (word_count, comp_count, doc_count, length_stats).

Return type:

tuple[ndarray, ndarray, ndarray, Any | None]

from_value(x)

Replace this accumulator’s sufficient statistics.

Parameters:

x (tuple[ndarray, ndarray, ndarray, SS1 | None]) – Aggregated sufficient statistics in value format.

Returns:

This accumulator.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

scale(c)

Scale linear latent counts and delegate document-length statistics.

Parameters:

c (float)

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

key_merge(stats_dict)

Merge this accumulator into stats_dict under configured keys.

If wc_key is set, merge the state/word count variable. If sc_key is set, merge the doc/state count variable. If dc_key is set, merge the author count variable.

The length accumulator receives the same merge request.

Parameters:

stats_dict (dict[str, Any]) – Mapping from merge keys to sufficient statistics.

Return type:

None

key_replace(stats_dict)

Replace sufficient statistics from matching keys in stats_dict.

If wc_key is set, set the state/word count variable to matching key in stats_dict. If sc_key is set, set the doc/state count variable to matching key in stats_dict. If dc_key is set, set the author count variable to matching key in stats_dict.

The length accumulator receives the same replace request.

Parameters:

stats_dict (dict[str, Any]) – Mapping from merge keys to sufficient statistics.

Return type:

None

acc_to_encoder()

Return an encoder compatible with integer PLSI observations.

Return type:

IntegerProbabilisticLatentSemanticIndexingDataEncoder

class IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory(num_vals, num_states, num_docs, len_factory=NullAccumulatorFactory(), keys=(None, None, None), name=None)

Bases: StatisticAccumulatorFactory

Factory for integer PLSI EM sufficient-statistic accumulators.

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_factory (StatisticAccumulatorFactory | None)

• keys (tuple[str | None, str | None, str | None] | None)

• name (str | None)

make()

Return a fresh integer PLSI accumulator.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

class IntegerProbabilisticLatentSemanticIndexingEstimator(num_vals, num_states, num_docs, len_estimator=NullEstimator(), pseudo_count=(None, None, None), suff_stat=(None, None, None), name=None, keys=(None, None, None))

Bases: ParameterEstimator

Estimator for integer PLSI word/state/document probabilities and the optional length model.

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_estimator (ParameterEstimator | None)

• pseudo_count (tuple[float | None, float | None, float | None] | None)

• suff_stat (tuple[ndarray | None, ndarray | None, ndarray | None] | None)

• name (str | None)

• keys (tuple[str | None, str | None, str | None] | None)

accumulator_factory()

Return an accumulator factory matching this estimator.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory

estimate(nobs, suff_stat)

Estimate an integer PLSI distribution from aggregated sufficient statistics.

Parameters:

• nobs (float | None) – Optional observation count, accepted for the estimator interface.

• suff_stat (tuple[ndarray, ndarray, ndarray, SS1 | None]) – Aggregated word, topic, document, and length statistics.

Returns:

A fitted integer PLSI distribution.

Return type:

IntegerProbabilisticLatentSemanticIndexingDistribution

class IntegerProbabilisticLatentSemanticIndexingDataEncoder(len_encoder=NullDataEncoder())

Bases: DataSequenceEncoder

Encode integer PLSI observations into flattened sparse bag arrays and length features.

Parameters:

len_encoder (DataSequenceEncoder | None)

seq_encode(x)

Encode iid PLSI observations for vectorized seq_* methods.

Input arg ‘x’ is a sequence of iid PLSI observations having form

x = [ (doc_id, [(value, count),…]),… ].

The return value has two entries. The first contains the optional length encoding. The second contains:

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document d_id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document d_id’s for the lengths above (len = len(x)).

Parameters:

x (Sequence[Tuple[int, Sequence[Tuple[int, float]]]]) – See above for details.

Returns:

See above for details.

Return type:

tuple[Any, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]

fast_seq_log_density(xv, xc, xd, xi, xm, wmat, smat, dvec, out)

Numba kernel for accumulating encoded integer PLSI log-density contributions.

fast_seq_component_log_density(xv, xc, xd, xi, xm, wmat, out)

Numba kernel for accumulating per-state component log-density contributions.

fast_seq_update(xv, xc, xd, xi, xm, weights, wmat, smat, wcnt, scnt, dcnt)

Numba kernel for the integer PLSI EM expected-count update.

index_dot(x, xi, y, yi, out)

Return row-wise dot products x[xi[i]] @ y[yi[i]] into out.

bincount(x, w, out)

Accumulate weighted one-dimensional group sums into out.

vec_bincount1(x, w, out)

Accumulate matrix-row weights into groups indexed by x.

vec_bincount2(x, w, y, out)

Accumulate rows w[y[i], :] into groups indexed by x.

vec_bincount3(x, w, out)

Numba bincount on the rows of matrix w for groups x.

Used to update comp counts for word/state probabilities.

N = len(x) S = number of states. U = unique values in x can take on (unique words in corpus).

Parameters:

• x (np.ndarray[np.float64]) – Group ids of columns of w.

• w (np.ndarray[np.float64]) – S by N numpy array with cols corresponding to x

• out (np.ndarray[np.float64]) – S by U matrix.

Returns:

Numpy 2-d array.

vec_bincount4(x, w, out)

Numba bincount on the rows of matrix w for groups x.

Used to initialize doc/state counts.

N = len(x) S = number of states. U = unique values in x can take on. (Unique number of authors).

Parameters:

• x (np.ndarray[np.float64]) – Group ids of columns of w.

• w (np.ndarray[np.float64]) – S by N numpy array with cols corresponding to x

• out (np.ndarray[np.float64]) – U by S matrix.

Returns:

Numpy 2-d array.