mixle.stats.latent.labeled_lda module

Create, estimate, and sample from a labeled latent Dirichlet allocation (LabeledLDA) model.

Defines the LabeledLDADistribution, LabeledLDASampler, LabeledLDAEstimatorAccumulator, LabeledLDAEstimatorAccumulatorFactory, LabeledLDAEstimator, and the LabeledLDADataEncoder classes for use with mixle.

Data type: Tuple[Sequence[Tuple[T, float]], Sequence[int]]. Each observation is a document given as a bag of (value, count) pairs together with a list of label (neighborhood) indices selecting rows of the ‘alphas’ matrix.

LabeledLDA extends latent Dirichlet allocation (see mixle.stats.latent.lda) by attaching a set of labels to each document. The model keeps one Dirichlet parameter row alpha_a per label a (the ‘alphas’ matrix is num_alpha by nTopics). A document with labels {a_1,…,a_m} draws its topic weights from a Dirichlet whose parameter is formed from the alpha rows of its labels. Generation of a document of length N with L topics proceeds as:

1. Draw theta ~ Dirichlet(alpha_bar), where alpha_bar combines the alpha rows of the document labels.

2. Draw topic-counts z_1,…,z_L ~ Multinomial(N, theta).

3. For each topic l = 1,2,…,L draw z_l values from the topic distribution P_l() (data type T).

If included, ‘len_dist’ models the number of values N in a document, and ‘set_dist’ models the label sets (both are used for sampling).

Estimation uses a mean-field variational EM (per-document gamma updates). The expected log topic weights are aggregated per distinct label set, and the alpha rows are updated jointly by maximizing the coupled objective in which each document’s Dirichlet parameter is the average of its label rows (see ‘update_alpha_coupled()’). When every document carries exactly one label the objective decouples and the classic per-row fixed-point update (‘update_alpha()’) is used.

class LabeledLDADistribution(topics, alphas, set_dist=None, len_dist=None, gamma_threshold=1.0e-8, max_gamma_iter=100)

Bases: SequenceEncodableProbabilityDistribution

LabeledLDADistribution object defining a labeled LDA model for documents with label sets.

Compatible with data type Tuple[Sequence[Tuple[T, float]], Sequence[int]], where T is the data type of the topic distributions.

density(x)

Returns the density (exp of the variational lower bound) for a labeled document x.

Parameters:

x (Tuple[Sequence[Tuple[T, float]], Sequence[int]]) – Document as (value, count) pairs and a list of label indices.

Returns:

Density value for the document x.

density_semantics()

What log_density returns relative to the true log-density (default: exact).

Override to declare that this distribution’s log_density is a variational lower bound (ELBO), an upper bound, or an approximation rather than the exact log p(x). This is surfaced as the ExactDensity capability and noted in mixle.describe(), so code that needs an exact likelihood can require(x, ExactDensity) instead of silently trusting a bound.

log_density(x)

Returns the variational lower bound (ELBO) on the log-density for a labeled document x.

Parameters:

x (Tuple[Sequence[Tuple[T, float]], Sequence[int]]) – Document as (value, count) pairs and a list of label indices.

Returns:

Lower bound on the log-density of the document x.

seq_log_density(x)

Vectorized evaluation of the variational lower bound (ELBO) for encoded documents.

Arg ‘x’ is the output of ‘LabeledLDADataEncoder.seq_encode()’.

Parameters:

x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

Returns:

Numpy array with one lower-bound value per encoded document.

seq_encode(x)

Deprecated: encode a sequence of iid LabeledLDA observations for vectorized ‘seq_’ calls.

Use ‘dist_to_encoder()’ and ‘LabeledLDADataEncoder.seq_encode()’ instead.

Parameters:

x (Sequence[Tuple[Sequence[Tuple[T, float]], Sequence[int]]]) – Sequence of labeled documents.

Returns:

Encoded sequence (see LabeledLDADataEncoder.seq_encode()).

seq_component_log_density(x)

Vectorized per-topic log-density evaluation for encoded documents.

Parameters:

x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

Returns:

2-d numpy array (num_documents by nTopics) of per-topic document log-densities.

seq_posterior(x)

Vectorized posterior topic weights for encoded documents.

Parameters:

x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

Returns:

2-d numpy array (num_documents by nTopics) of normalized posterior topic weights.

compute_capabilities()

Return backend capability metadata for this concrete LabeledLDA instance.

backend_seq_log_density(x, engine)

Backend-neutral LabeledLDA variational lower-bound (ELBO) scoring.

enumerator()

Not supported: LabeledLDA’s log_density is a variational lower bound, not the true marginal.

Each document’s score is an ELBO obtained by running per-document variational inference (seq_posterior), not the exact marginal probability, so enumerating “in descending probability order” is ill-defined – the ELBO ranking need not match the true-probability ranking, and the per-document optimum isn’t a closed-form density over a countable support. Use sampler() and the (approximate) log_density directly.

Return type:

DistributionEnumerator

sampler(seed=None)

Create an LabeledLDASampler object with seed passed.

Note: Requires ‘set_dist’ and ‘len_dist’ to be set.

Parameters:

seed (Optional[int]) – Set seed for random sampling.

Returns:

LabeledLDASampler object.

estimator(pseudo_count=None)

Create an LabeledLDAEstimator for estimating models like this object instance.

Parameters:

pseudo_count (Optional[float]) – Used to re-weight sufficient statistics in estimation.

Returns:

LabeledLDAEstimator object.

dist_to_encoder()

Returns LabeledLDADataEncoder object for encoding sequences of iid LabeledLDA observations.

class LabeledLDASampler(dist, seed=None)

Bases: DistributionSampler

LabeledLDASampler object for sampling labeled documents from an LabeledLDADistribution.

Requires ‘dist.set_dist’ (label sets) and ‘dist.len_dist’ (document lengths) to be set.

sample(size=None)

Draw iid labeled documents from the LabeledLDA model.

If size is None, a single Tuple[List[T], List[int]] is returned containing the sampled document values and its labels. If size > 0, a list of ‘size’ such Tuples is returned.

Parameters:

size (Optional[int]) – Number of iid labeled documents to sample.

Returns:

Tuple[List[T], List[int]] or a List of such Tuples depending on arg size.

class LabeledLDALabelSetStats(stats=None)

Bases: object

Sufficient statistics for the coupled alpha update, grouped by distinct document label set.

Maps each distinct label set S (a sorted tuple of label indices, duplicates preserved) to a pair [n_S, m_S], where n_S is the total weight of the documents carrying label set S and m_S is the weighted sum of the per-document expected log topic weights E[log theta] (a vector with one entry per topic) over those documents.

add(label_set, weight, sum_log_p)

Accumulate document weight and summed expected log topic weights for one label set.

Parameters:

• label_set (Tuple[int, ...]) – Sorted tuple of label indices.

• weight (float) – Total document weight to add for the label set.

• sum_log_p (np.ndarray) – Weighted sum of per-document E[log theta] vectors to add.

Returns:

None.

combine(other)

Merge the statistics of another LabeledLDALabelSetStats instance into this instance.

Parameters:

other (LabeledLDALabelSetStats) – Statistics to merge in (left unmodified).

Returns:

LabeledLDALabelSetStats object (self).

copy()

Returns a deep copy of the LabeledLDALabelSetStats instance.

arrays()

Returns the statistics as parallel arrays in sorted label-set order.

Returns:

Tuple of the sorted label-set tuples (List[Tuple[int, …]]), the per-set weights n_S (1-d numpy array), and the per-set summed expected log topic weights m_S (2-d numpy array with one row per label set).

doc_label_sets(nbx, nbcnt)

Returns the sorted label-set tuple of each encoded document.

Parameters:

• nbx (np.ndarray) – Flattened array of label indices over all documents (document-contiguous).

• nbcnt (np.ndarray) – Number of labels for each document.

Returns:

List of sorted label-index tuples, one per document.

class LabeledLDAEstimatorAccumulator(accumulators, num_alphas, keys=(None, None), prev_alpha=None)

Bases: SequenceEncodableStatisticAccumulator

LabeledLDAEstimatorAccumulator object for aggregating sufficient statistics from labeled documents.

Tracks per-label-set expected log topic weights and document counts (‘set_stats’), per-label weighted document counts (‘doc_counts’), label-allocated topic counts (‘topic_counts’), and the topic distribution accumulators.

update(x, weight, estimate)

Update sufficient statistics of the accumulator with one labeled document.

Note: Not efficient. Encodes a singleton batch and delegates to ‘seq_update()’.

Parameters:

• x (Tuple[Sequence[Tuple[T, float]], Sequence[int]]) – Document as (value, count) pairs and a list of label indices.

• weight (float) – Weight for observation.

• estimate (LabeledLDADistribution) – Previous estimate of the LabeledLDA model.

Returns:

None.

initialize(x, weight, rng)

Initialize the accumulator with a single labeled document.

Draws document topic weights from a Dirichlet formed from the label rows of ‘prev_alpha’, randomly assigns each document value to a topic, and initializes the topic accumulators accordingly.

Parameters:

• x (Tuple[Sequence[Tuple[T, float]], Sequence[int]]) – Document as (value, count) pairs and a list of label indices.

• weight (float) – Weight for observation.

• rng (RandomState) – RandomState for random topic assignments.

Returns:

None.

seq_initialize(x, weights, rng)

Vectorized initialization of the accumulator from an encoded sequence of labeled documents.

Mirrors ‘initialize()’: per-document topic weights are drawn from a Dirichlet formed from the label rows of ‘prev_alpha’, each document value is randomly assigned to a topic, and the topic accumulators are initialized with smoothed per-value weights.

Parameters:

• x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

• weights (np.ndarray) – Numpy array of weights for the documents.

• rng (RandomState) – Used to seed member RandomState objects on first call.

Returns:

None.

seq_update(x, weights, estimate)

Vectorized update of the accumulator from an encoded sequence of labeled documents.

Computes the variational posterior for each document under ‘estimate’ and aggregates per-label-set expected log topic weights, per-label document counts, label-allocated topic counts, and the topic accumulator statistics.

Parameters:

• x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

• weights (np.ndarray) – Numpy array of weights for the documents.

• estimate (LabeledLDADistribution) – Previous EM estimate of the LabeledLDA model.

Returns:

None.

seq_update_engine(x, weights, estimate, engine)

Engine-resident LabeledLDA E-step (numpy or torch).

Runs the variational posterior and the per-label-set / topic-count aggregations on the active engine, feeding engine-computed responsibilities to the topic accumulators. Mirrors seq_update.

combine(suff_stat)

Combine the sufficient statistics of the accumulator with the suff_stat arg.

Sufficient statistics in suff_stat are a Tuple containing:

suff_stat[0] (Optional[np.ndarray]): Previous alphas matrix. suff_stat[1] (LabeledLDALabelSetStats): Per-label-set expected log topic weights and counts. suff_stat[2] (Union[float, np.ndarray]): Per-label weighted document counts. suff_stat[3] (np.ndarray): Label-allocated weighted topic counts. suff_stat[4] (Sequence): Topic distribution accumulator values.

Parameters:

suff_stat – See above for details.

Returns:

LabeledLDAEstimatorAccumulator object.

value()

Returns sufficient statistics of the accumulator instance.

Returns:

Tuple of previous alphas matrix, per-label-set statistics (LabeledLDALabelSetStats), per-label document counts, label-allocated topic counts, and the topic accumulator values.

from_value(x)

Set the sufficient statistics of the accumulator instance to the value x.

Parameters:

x – Tuple of sufficient statistics (see ‘value()’ for details).

Returns:

LabeledLDAEstimatorAccumulator object.

key_merge(stats_dict)

Merge the sufficient statistics of the accumulator instance into stats_dict for matching keys.

Parameters:

stats_dict (Dict[str, Any]) – Dictionary mapping keys to sufficient statistics.

Returns:

None.

key_replace(stats_dict)

Replace the sufficient statistics of the accumulator instance with values in stats_dict for matching keys.

Parameters:

stats_dict (Dict[str, Any]) – Dictionary mapping keys to sufficient statistics.

Returns:

None.

acc_to_encoder()

Returns LabeledLDADataEncoder object for encoding sequences of iid LabeledLDA observations.

class LabeledLDAEstimatorAccumulatorFactory(factories, dim, num_alphas, keys, prev_alpha)

Bases: StatisticAccumulatorFactory

LabeledLDAEstimatorAccumulatorFactory object for creating LabeledLDAEstimatorAccumulator objects.

make()

Returns an LabeledLDAEstimatorAccumulator object.

class LabeledLDAEstimator(estimators, num_alphas, suff_stat=None, pseudo_count=None, keys=(None, None), fixed_alpha=None, gamma_threshold=1.0e-8, alpha_threshold=1.0e-8, max_gamma_iter=100)

Bases: ParameterEstimator

LabeledLDAEstimator object for estimating LabeledLDADistribution objects from aggregated sufficient statistics.

accumulator_factory()

Returns an LabeledLDAEstimatorAccumulatorFactory object.

accumulatorFactory()

Deprecated alias for accumulator_factory().

estimate(nobs, suff_stat)

Estimate an LabeledLDADistribution from aggregated sufficient statistics ‘suff_stat’.

Sufficient statistics in arg ‘suff_stat’ are a Tuple containing:

suff_stat[0] (Optional[np.ndarray]): Previous alphas matrix. suff_stat[1] (LabeledLDALabelSetStats): Per-label-set expected log topic weights and counts. suff_stat[2] (Union[float, np.ndarray]): Per-label weighted document counts. suff_stat[3] (np.ndarray): Label-allocated weighted topic counts. suff_stat[4] (Sequence): Sufficient statistics for the topic distribution accumulators.

If ‘fixed_alpha’ is None, the alphas matrix is re-estimated by maximizing the coupled objective over all label rows (see ‘update_alpha_coupled()’). When every document carries exactly one label the objective decouples and the per-row fixed-point updates are used (see ‘update_alpha()’). Otherwise the alphas matrix is set to ‘fixed_alpha’.

Parameters:

• nobs (Optional[float]) – Number of observations used in estimation.

• suff_stat – See above for details.

Returns:

LabeledLDADistribution object.

class LabeledLDADataEncoder(encoder)

Bases: DataSequenceEncoder

LabeledLDADataEncoder object for encoding sequences of iid LabeledLDA observations (labeled documents).

seq_encode(x)

Encode a sequence of iid LabeledLDA observations (labeled documents) for vectorized functions.

Return value ‘rv’ is a Tuple containing:

rv[0] (int): Number of documents. rv[1] (np.ndarray): Document id for each flattened document value. rv[2] (np.ndarray): Flattened array of counts for each value in each document. rv[3] (Optional[np.ndarray]): Document gammas (defaults to None). rv[4]: Sequence encoded flattened document values. rv[5] (np.ndarray): Flattened array of label indices over all documents. rv[6] (np.ndarray): Number of labels for each document. rv[7] (np.ndarray): Document id for each flattened label index.

Parameters:

x (Sequence[Tuple[Sequence[Tuple[T, float]], Sequence[int]]]) – Sequence of labeled documents.

Returns:

See above for details.

update_alpha(current_alpha, mean_log_p, alpha_threshold)

Fixed-point update of the per-label alpha rows from mean expected log topic weights.

Iterates alpha <- digammainv(mean_log_p + digamma(sum(alpha))) row-wise until the relative change of each row falls below alpha_threshold.

Parameters:

• current_alpha (np.ndarray) – Current alphas matrix (num_alphas by num_topics).

• mean_log_p (np.ndarray) – Per-label mean expected log topic weights (num_alphas by num_topics).

• alpha_threshold (float) – Convergence threshold for the row-wise updates.

Returns:

Numpy 2-d array of updated alphas (num_alphas by num_topics).

updateAlpha(current_alpha, mean_log_p, alpha_threshold)

Deprecated alias for update_alpha().

label_set_membership(label_sets)

Returns flattened membership arrays for a sequence of label sets.

Parameters:

label_sets (Sequence[Tuple[int, ...]]) – Label-set tuples (one per distinct document label set).

Returns:

Tuple of the flattened label indices (member_label), the label-set index of each flattened entry (member_set), and the per-set sizes |S| as floats.

coupled_alpha_doc_params(alpha, label_sets)

Returns the per-label-set Dirichlet parameters a_S = mean_{l in S} alpha[l].

Parameters:

• alpha (np.ndarray) – Alphas matrix (num_alphas by num_topics).

• label_sets (Sequence[Tuple[int, ...]]) – Label-set tuples.

Returns:

Numpy 2-d array with one Dirichlet parameter row per label set.

coupled_alpha_objective(alpha, label_sets, set_counts, set_mean_logs)

Coupled multi-label alpha objective (terms independent of alpha dropped).

F(alpha) = sum_S n_S * [ log Gamma(sum_k a_Sk) - sum_k log Gamma(a_Sk) + sum_k a_Sk * mbar_Sk ], where a_S = mean_{l in S} alpha[l], n_S = set_counts[S], and mbar_S = set_mean_logs[S] are the per-set mean expected log topic weights.

Parameters:

• alpha (np.ndarray) – Alphas matrix (num_alphas by num_topics).

• label_sets (Sequence[Tuple[int, ...]]) – Label-set tuples.

• set_counts (np.ndarray) – Per-set document weights n_S.

• set_mean_logs (np.ndarray) – Per-set mean expected log topic weights mbar_S (one row per set).

Returns:

Objective value F(alpha).

coupled_alpha_gradient(alpha, label_sets, set_counts, set_mean_logs)

Gradient of the coupled multi-label alpha objective with respect to alpha.

dF/d alpha[l,k] = sum_{S contains l} (n_S/|S|) * [ psi(sum_j a_Sj) - psi(a_Sk) + mbar_Sk ], with one term per occurrence of l in S.

Parameters:

• alpha (np.ndarray) – Alphas matrix (num_alphas by num_topics).

• label_sets (Sequence[Tuple[int, ...]]) – Label-set tuples.

• set_counts (np.ndarray) – Per-set document weights n_S.

• set_mean_logs (np.ndarray) – Per-set mean expected log topic weights mbar_S (one row per set).

Returns:

Numpy 2-d array with the same shape as alpha.

update_alpha_coupled(current_alpha, label_sets, set_counts, set_mean_logs, alpha_threshold, max_its=2000)

Coupled update of the full alphas matrix for documents with multi-label sets.

Maximizes ‘coupled_alpha_objective()’ over all positive alpha entries. Since each document Dirichlet parameter a_S averages several alpha rows, the rows do not decouple; the objective is concave in alpha (a_S is linear in alpha and the Dirichlet log-partition is convex), so ascent converges to the global maximum. The ascent is run on beta = log(alpha) (keeping alpha positive) with backtracking line search and an adaptive step size, warm-started from ‘current_alpha’, until the row-wise relative change of alpha falls below alpha_threshold (matching the ‘update_alpha()’ convergence semantics).

Label rows that appear in no label set have zero gradient and are returned unchanged.

Parameters:

• current_alpha (np.ndarray) – Current alphas matrix (num_alphas by num_topics), used as warm start.

• label_sets (Sequence[Tuple[int, ...]]) – Distinct document label sets (sorted tuples).

• set_counts (np.ndarray) – Per-set document weights n_S.

• set_mean_logs (np.ndarray) – Per-set mean expected log topic weights mbar_S (one row per set).

• alpha_threshold (float) – Convergence threshold for the row-wise relative alpha changes.

• max_its (int) – Maximum number of accepted ascent steps.

Returns:

Numpy 2-d array of updated alphas (num_alphas by num_topics).

mpe_update(X, y, min_size=2)

Single minimal polynomial extrapolation (MPE) update step for a fixed-point iterate y.

Parameters:

• X (Optional[np.ndarray]) – Matrix of previous iterates (one per row), or None to start.

• y (np.ndarray) – New fixed-point iterate.

• min_size (int) – Minimum number of stored iterates before extrapolating.

Returns:

Tuple of the updated iterate matrix and the extrapolated estimate.

mpe(x0, f, eps)

Minimal polynomial extrapolation (MPE) of the fixed point of f starting from x0.

Parameters:

• x0 (np.ndarray) – Starting point of the fixed-point iteration.

• f (Callable[[np.ndarray], np.ndarray]) – Fixed-point map.

• eps (float) – Convergence threshold on the absolute change of the extrapolated estimate.

Returns:

Tuple of the extrapolated fixed point and the iteration count.

alpha_seq_lambda(meanLogP)

Returns the alpha fixed-point map for mean expected log topic weights meanLogP.

find_alpha(current_alpha, mlp, thresh)

Find the alpha fixed point for mean expected log topic weights mlp via MPE.

Parameters:

• current_alpha (np.ndarray) – Starting alpha value.

• mlp (np.ndarray) – Mean expected log topic weights.

• thresh (float) – Convergence threshold.

Returns:

Tuple of the extrapolated alpha and the iteration count.

seq_posterior(estimate, x)

Compute the variational posterior quantities for encoded labeled documents under ‘estimate’.

Runs the shared per-document mean-field gamma fixed point (see lda._lda_vi_fixed_point), passing each document’s coupled Dirichlet prior ‘alphas_loc’ – the mean of the alpha rows of the document’s label set. This is the only model difference from plain LDA, which uses a single shared alpha; the fixed-point loop is otherwise identical.

Parameters:

• estimate (LabeledLDADistribution) – LabeledLDA model used to evaluate the posterior.

• x – Encoded sequence of iid LabeledLDA observations (see LabeledLDADataEncoder.seq_encode()).

Returns:

Tuple of per-value topic responsibilities (log_density_gamma), per-document gammas (final_gammas), per-document Dirichlet parameters (alphas_loc), and per-value per-topic log-densities.

LabeledLDAAccumulator

alias of LabeledLDAEstimatorAccumulator

LabeledLDAAccumulatorFactory

alias of LabeledLDAEstimatorAccumulatorFactory