mixle.stats.latent.integer_probabilistic_latent_semantic_indexing module

Create, estimate, and sample from an integer PLSI model.

Defines the IntegerProbabilisticLatentSemanticIndexingDistribution, IntegerProbabilisticLatentSemanticIndexingSampler, IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory, IntegerProbabilisticLatentSemanticIndexingAccumulator, IntegerProbabilisticLatentSemanticIndexingEstimator, and the IntegerProbabilisticLatentSemanticIndexingDataEncoder classes for use with mixle.

Consider an Integer PLSI model for a corpus of documents with S states, V word values, and D authors (doc_ids).

Let x (Tuple[int, Sequence[Tuple[int, float]]]) be an observation from a PLSI model, consisting of

x = (d, [(v_0, c_0), (v_1, c_1), …, (v_{k-1}, c_{k-1})]),

where the ‘d’ is some author (doc_id) in the corpus and each tuple (v_i, c_i) corresponds to a value-count couple for some value ‘v_i’ in dictionary of words used in the corpus. Let w denote the distinct words {v_i} in the document represented by x. The density for the PLSI model is given by

p_mat(w, d) = P_len(nn)*p_mat(d) prod_{j=0}^{k-1} ( sum_{s=0}^{S-1} p_mat(v_j | s )p_mat(s | d) )^(c_j),

where P_len(nn) is the density of the length distribution for ‘nn’ representing the total number of words in the document (i.e. nn = sum_i c_i), p_mat(d) is the probability of observing a document from author ‘d’, p_mat(v_j|s) is the probability of observing word (integer-valued) given word-topic ‘s’, and p_mat(s|d) are the weights for the word-topic for author ‘d’.

Note: To use this distribution, convert your words and authors of the corpus to unique integer keys.

class IntegerProbabilisticLatentSemanticIndexingDistribution(state_word_mat, doc_state_mat, doc_vec, len_dist=NullDistribution(), name=None)

Bases: SequenceEncodableProbabilityDistribution

Integer-valued probabilistic latent semantic indexing distribution.

Parameters:

• state_word_mat (list[list[float]] | ndarray)

• doc_state_mat (list[list[float]] | ndarray)

• doc_vec (list[float] | ndarray)

• len_dist (SequenceEncodableProbabilityDistribution | None)

• name (str | None)

compute_capabilities()

Return backend capability metadata for this concrete PLSI instance.

compute_declaration()

density(x)

Evaluate the density of PLSI model for an observation x.

See log_density() for details on the density evaluation.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – Single observation of integer PLSI.

Returns:

Density evaluated at observed value x.

Return type:

float

log_density(x)

Evaluate the log-density of PLSI model for an observation of x.

Consider an Integer PLSI model for a corpus of documents with S states, V word values, and D documents ids (authors).

Let x (Tuple[int, Sequence[Tuple[int, float]]]) be an observation from a PLSI model, consisting of x = (d, [(v_0, c_0), (v_1, c_1), …, (v_{k-1}, c_{k-1})]), where the ‘d’ is some document d_id in the corpus and each tuple (v_i, c_i) corresponds to a value-count couple in the corpus. The log-likelihood is given by

log(p_mat(x)) = log(p_mat(d)) + sum_{j=0}^{k-1} c_k*log( sum_{s=0}^{S-1} p_mat(d|s)p_mat(s|v_k) ) + log(P_len(nn)),

where P_len(nn) is the density of the length distribution for ‘nn’ representing the total number of words in the document.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – (doc_id, [(value_id, count_for_value)]). See above for details.

Returns:

Log-density evaluated at a single observation x.

Return type:

float

component_log_density(x)

Evaluate the log-density for each state in the PLSI.

Returns count*log(p_mat(W|S)) for each word-count pair in the document. Returned value is S by 1 where S is the number of components in the model.

Parameters:

x (Tuple[int, Sequence[Tuple[int, float]]]) – Single PLSI observation of form (doc_id, [(value_id, count_for_value)]).

Returns:

Numpy array of length S (num_states).

Return type:

ndarray

seq_log_density(x)

Vectorized evaluation of the log-density for an encoded sequence of iid observation from a PLSI model.

See log_density() function for details on the log-likelihood.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

Returns:

Numpy array of log-density evaluated at each observation in the encoded sequence.

Return type:

ndarray

backend_seq_log_density(x, engine)

Evaluate encoded PLSI log densities using a backend-neutral compute engine.

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]])

• engine (Any)

Return type:

Any

seq_component_log_density(x)

Vectorized evaluation of the component log-density for each observation in an encoded sequence of iid PLSI

observations.

See component_log_density() function for details on component log-likelihood evaluation.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

Returns:

2-d numpy array containing N rows of num_state sized arrays.

Return type:

ndarray

enumerator()

Enumerate PLSI observations (doc_id, bag) in descending probability order.

A PLSI observation factors as P(doc) * [prod_w q_d(w)^{c_w}] * P_len(n) where q_d is the per-document word distribution prob_mat @ state_mat[d] and n the total word count, so it is a document-labelled mixture of trial-count multinomials: for each document the bags enumerate by a multiset best-first search under a length frontier driven by len_dist (the real trial-count distribution), and the per-document streams are merged by descending score with the document log-probability as offset. Requires a modelled len_dist unless every per-document word distribution is sub-stochastic-free; an absent length distribution leaves the bag support infinite and is enumerated by the multinomial term alone.

Return type:

DistributionEnumerator

sampler(seed=None)

Return an IntegerProbabilisticLatentSemanticIndexingSampler object from IntegerProbabilisticLatentSemanticIndexingDistribution instance.

Parameters:

seed (int | None)

Return type:

IntegerProbabilisticLatentSemanticIndexingSampler

estimator(pseudo_count=None)

Create an IntegerProbabilisticLatentSemanticIndexingEstimator object from IntegerProbabilisticLatentSemanticIndexingDistribution instance.

Parameters:

pseudo_count (Optional[float]) – Re-weight object instance sufficient statistics when passed to estimator.

Returns:

IntegerProbabilisticLatentSemanticIndexingEstimator object.

Return type:

IntegerProbabilisticLatentSemanticIndexingEstimator

dist_to_encoder()

Returns IntegerProbabilisticLatentSemanticIndexingDataEncoder object.

Return type:

IntegerProbabilisticLatentSemanticIndexingDataEncoder

multinomial_bag_stream(log_p_vec, min_val, len_dist, combine)

Enumerate integer count-vector bags in descending sum_w c_w*log p_w + log P_len(n) order.

Reuses the per-size multiset best-first search (MultisetProductEnumerator) under a length frontier driven by len_dist (the real trial-count distribution); combine maps the tuple of (value, count) pairs to the emitted bag. When len_dist is Null there is no length term and a synthetic n*log p_max frontier orders the (countably infinite) support by the multinomial term alone – matching IntegerMultinomialEnumerator. Shared by the coupled bag-of-counts models.

bag_stream(element_stream, len_dist, combine)

Enumerate bags (multisets) drawn from a sorted element stream, in descending bag-score order.

element_stream is a descending (value, log_prob) iterator over the element distribution (any enumerable element distribution – a fixed categorical, or e.g. a per-document/per-given mixture). A bag scores by the sum of its elements’ log-probs plus log P_len(n) from len_dist; bags enumerate by the per-size multiset best-first search (MultisetProductEnumerator) under a length frontier. When len_dist is Null there is no length term and a synthetic n*log p_max frontier orders the (countably infinite) support by the element term alone. combine maps the tuple of (value, count) pairs to the emitted bag.

class IntegerProbabilisticLatentSemanticIndexingEnumerator(dist)

Bases: DistributionEnumerator

Parameters:

dist (IntegerProbabilisticLatentSemanticIndexingDistribution)

class IntegerProbabilisticLatentSemanticIndexingSampler(dist, seed=None)

Bases: DistributionSampler

Parameters:

• dist (IntegerProbabilisticLatentSemanticIndexingDistribution)

• seed (int | None)

sample(size=None)

Generate iid samples from PLSI model.

Parameters:

size (Optional[int]) – Number of samples to generate. Defaults to 0 if size is None.

Returns:

Sequence of iid PLSI samples if size is not None, else a single sample from PLSI model.

Return type:

tuple[int, Sequence[tuple[int, float]]] | Sequence[tuple[int, Sequence[tuple[int, float]]]]

class IntegerProbabilisticLatentSemanticIndexingAccumulator(num_vals, num_states, num_docs, len_acc=NullAccumulator(), name=None, keys=(None, None, None))

Bases: SequenceEncodableStatisticAccumulator

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_acc (SequenceEncodableStatisticAccumulator | None)

• name (str | None)

• keys (tuple[str | None, str | None, str | None] | None)

update(x, weight, estimate)

Update the sufficient statistics of object instance for a single observation x.

Parameters:

• x (Tuple[int, Sequence[Tuple[int, float]]]) – An observation from integer PLSI model.

• weight (float) – Observation weight.

• estimate (IntegerProbabilisticLatentSemanticIndexingDistribution) – Prior estimate of IntegerProbabilisticLatentSemanticIndexingDistribution object.

Returns:

None.

Return type:

None

initialize(x, weight, rng)

Initialize sufficient statistics of object instance with observation x.

Parameters:

• x (Tuple[int, Sequence[Tuple[int, float]]]) – An observation from integer PLSI model.

• weight (float) – Observation weight.

• rng (RandomState) – RandomState object for seeded initialization.

Returns:

None.

Return type:

None

seq_initialize(x, weights, rng)

Vectorized initialization of sufficient statistics form an encoded sequence of observations in arg ‘x’.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI. See above for details.

• weights (ndarray) – Weights for observations in encoded sequence.

• rng (RandomState) – Used to initialize member RandomState variables.

Returns:

None.

Return type:

None

seq_update(x, weights, estimate)

Vectorized update of sufficient statistics for encoded sequence of iid observations in x.

The encoded sequence ‘x’ is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document id’s for the lengths above (len = len(x)).

Parameters:

• x (tuple[T1 | None, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]) – Encoded sequence of iid observations of PLSI model. See above for details.

• weights (np.ndarray) – Numpy array of observation weights.

• estimate (IntegerProbabilisticLatentSemanticIndexingDistribution) – Prior estimate of IntegerProbabilisticLatentSemanticIndexingDistribution object.

Returns:

None.

Return type:

None

seq_update_engine(x, weights, estimate, engine)

Engine-resident E-step: the PLSI responsibility update (state-word x doc-state gather, per-pair normalization, and the word/doc segment sums) runs on the active engine, matching the host seq_update.

combine(suff_stat)

Combine the sufficient statistics in arg ‘suff_stat’ with object instance.

Arg ‘suff_stat’ is Tuple[np.ndarray, np.ndarray, np.ndarray, Optional[SS1]] containing:

suff_stat[0] (np.ndarray): State/word counts with matching dimension of num_states by num_vals. suff_stat[1] (np.ndarray): Doc/state counts with matching dimension of num_docs by num_states. suff_stat[2] (np.ndarray): Author counts with length (num_docs). suff_stat[3] (Optional[SS1]): Sufficient statistics for the length of document distribution having type SS1.

Parameters:

suff_stat (tuple[ndarray, ndarray, ndarray, SS1 | None]) – See above for details.

Returns:

IntegerProbabilisticLatentSemanticIndexingAccumulator object.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

value()

Returns sufficient statistics of IntegerProbabilisticLatentSemanticIndexingAccumulator object instance.

Returned value ‘suff_stat’ is Tuple[np.ndarray, np.ndarray, np.ndarray, Optional[SS1]] containing:

suff_stat[0] (np.ndarray): State/word counts with matching dimension of num_states by num_vals. suff_stat[1] (np.ndarray): Doc/state counts with matching dimension of num_docs by num_states. suff_stat[2] (np.ndarray): Author counts with length (num_docs). suff_stat[3] (Optional[SS1]): Sufficient statistics for the length of document distribution having type SS1.

Return type:

tuple[ndarray, ndarray, ndarray, Any | None]

from_value(x)

Set the sufficient statistics of object instance to arg ‘x’ values.

Arg ‘x’ is Tuple[np.ndarray, np.ndarray, np.ndarray, Optional[SS1]] containing:

x[0] (np.ndarray): State/word counts with matching dimension of num_states by num_vals. x[1] (np.ndarray): Doc/state counts with matching dimension of num_docs by num_states. x[2] (np.ndarray): Author counts with length (num_docs). x[3] (Optional[SS1]): Sufficient statistics for the length of document distribution having type SS1.

Parameters:

x (tuple[ndarray, ndarray, ndarray, SS1 | None]) – Aggregated sufficient statistics. See above for details.

Returns:

IntegerProbabilisticLatentSemanticIndexingAccumulator object.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

scale(c)

Scale linear latent counts and delegate document-length statistics.

Parameters:

c (float)

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

key_merge(stats_dict)

Merge the sufficient statistics of object instance with matching keys.

If wc_key is set, merge the state/word count variable. If sc_key is set, merge the doc/state count variable. If dc_key is set, merge the author count variable.

Call key_merge() of accumulator for the length. Note nothing is done for this if default NullAccumulator is set.

Parameters:

stats_dict (Dict[str, Any]) – Maps keys to sufficient statistics.

Returns:

None.

Return type:

None

key_replace(stats_dict)

Set the sufficient statistics of object instance to matching key values in arg ‘stats_dict’.

If wc_key is set, set the state/word count variable to matching key in stats_dict. If sc_key is set, set the doc/state count variable to matching key in stats_dict. If dc_key is set, set the author count variable to matching key in stats_dict.

Call key_replace() of accumulator for the length. Note nothing is done for this if default NullAccumulator is set.

Parameters:

stats_dict (Dict[str, Any]) – Maps keys to sufficient statistics.

Returns:

None.

Return type:

None

acc_to_encoder()

Return an IntegerProbabilisticLatentSemanticIndexingDataEncoder object.

Return type:

IntegerProbabilisticLatentSemanticIndexingDataEncoder

class IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory(num_vals, num_states, num_docs, len_factory=NullAccumulatorFactory(), keys=(None, None, None), name=None)

Bases: StatisticAccumulatorFactory

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_factory (StatisticAccumulatorFactory | None)

• keys (tuple[str | None, str | None, str | None] | None)

• name (str | None)

make()

Returns IntegerProbabilisticLatentSemanticIndexingAccumulator object.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulator

class IntegerProbabilisticLatentSemanticIndexingEstimator(num_vals, num_states, num_docs, len_estimator=NullEstimator(), pseudo_count=(None, None, None), suff_stat=(None, None, None), name=None, keys=(None, None, None))

Bases: ParameterEstimator

Parameters:

• num_vals (int)

• num_states (int)

• num_docs (int)

• len_estimator (ParameterEstimator | None)

• pseudo_count (tuple[float | None, float | None, float | None] | None)

• suff_stat (tuple[ndarray | None, ndarray | None, ndarray | None] | None)

• name (str | None)

• keys (tuple[str | None, str | None, str | None] | None)

accumulator_factory()

Returns IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory object.

Return type:

IntegerProbabilisticLatentSemanticIndexingAccumulatorFactory

estimate(nobs, suff_stat)

Estimate IntegerProbabilisticLatentSemanticIndexingDistribution from aggregated sufficient statistics in arg ‘suff_stat’.

Parameters:

• nobs (Optional[float]) – Optional number of observations used to accumulate ‘suff_stat’.

• suff_stat (tuple[ndarray, ndarray, ndarray, SS1 | None]) – See above for details.

Returns:

IntegerProbabilisticLatentSemanticIndexingDistribution object.

Return type:

IntegerProbabilisticLatentSemanticIndexingDistribution

class IntegerProbabilisticLatentSemanticIndexingDataEncoder(len_encoder=NullDataEncoder())

Bases: DataSequenceEncoder

Parameters:

len_encoder (DataSequenceEncoder | None)

seq_encode(x)

Encode a sequence of iid PLSI observations for use with vectorized functions.

Input arg ‘x’ is a sequence of iid PLSI observations having form

x = [ (doc_id, [(value, count),…]),… ].

The return value is a Tuple length 2. The first component contains data type Optional[T1] corresponding to the sequence encoding of the lengths. The second component is a Tuple of length 6 containing

xv (ndarray[int]): Numpy array of flattened word values. xc (ndarray[float]): Numpy array of flattened counts for word values above. xd (ndarray[int]): Document d_id for each word-count pair in the arrays above. xi (ndarray[int]): Observed sequence index for each word-count pair in the arrays above. xn (ndarray[float]): Numpy array of the total number of words in each document. xm (ndarray[float]): Flattened array of document d_id’s for the lengths above (len = len(x)).

Parameters:

x (Sequence[Tuple[int, Sequence[Tuple[int, float]]]]) – See above for details.

Returns:

See above for details.

Return type:

tuple[Any, tuple[ndarray, ndarray, ndarray, ndarray, ndarray, ndarray]]

fast_seq_log_density(xv, xc, xd, xi, xm, wmat, smat, dvec, out)

fast_seq_component_log_density(xv, xc, xd, xi, xm, wmat, out)

fast_seq_update(xv, xc, xd, xi, xm, weights, wmat, smat, wcnt, scnt, dcnt)

index_dot(x, xi, y, yi, out)

bincount(x, w, out)

vec_bincount1(x, w, out)

vec_bincount2(x, w, y, out)

vec_bincount3(x, w, out)

Numba bincount on the rows of matrix w for groups x.

Used to update comp counts for word/state probabilities.

N = len(x) S = number of states. U = unique values in x can take on (unique words in corpus).

Parameters:

• x (np.ndarray[np.float64]) – Group ids of columns of w.

• w (np.ndarray[np.float64]) – S by N numpy array with cols corresponding to x

• out (np.ndarray[np.float64]) – S by U matrix.

Returns:

Numpy 2-d array.

vec_bincount4(x, w, out)

Numba bincount on the rows of matrix w for groups x.

Used to initialize doc/state counts.

N = len(x) S = number of states. U = unique values in x can take on. (Unique number of authors).

Parameters:

• x (np.ndarray[np.float64]) – Group ids of columns of w.

• w (np.ndarray[np.float64]) – S by N numpy array with cols corresponding to x

• out (np.ndarray[np.float64]) – U by S matrix.

Returns:

Numpy 2-d array.