mixle.stats.latent.quantized_hidden_markov_model module

Create, estimate, and sample from a quantized hidden Markov model.

Defines the QuantizedHiddenMarkovModelDistribution and QuantizedHiddenMarkovEstimator classes for use with mixle.

Data type: Sequence[T] where T is a categorical emission value drawn from a finite set of levels.

A quantized HMM is an HMM with finite categorical emissions in which every probability in the model is a power of a single shared base theta in (0, 1):

  1. State transitions: p_mat(Z_t = j | Z_{t-1} = i) = theta^k_trans[i, j] / Z_trans[i]

  2. Emissions: P(X_t = v | Z_t = i) = theta^k_emit[i, v] / Z_emit[i]

  3. Initial states: p_mat(Z_1 = j) = theta^k_init[j] / Z_init

    (or the stationary distribution of (1), see init_mode)

where every exponent k is a non-negative integer and Z_row = sum_j theta^k[row, j] is the per-row normalizer required because integer exponents with a single shared theta cannot sum to exactly one. A negative exponent entry (-1) marks a structural zero: that transition/emission has probability 0.

Log-probabilities therefore take the form k * log(theta) - log(Z_row): a single integer multiply plus one cached per-row float, so log <-> probability conversion is fast and the maximum-likelihood parameters are naturally quantized (at theta = 1/2 the exponents are code lengths in bits).

Estimation fits within EM: the E-step is the ordinary Baum-Welch pass (reused unchanged from mixle.stats.latent.hidden_markov), while the M-step performs coordinate ascent on the expected complete-data log-likelihood, alternating between (a) quantizing the unconstrained row MLEs to integer exponents, k = round(log(p_hat) / log(theta)), and (b) a 1-d bounded maximization of

f(theta) = sum_cells N * k * log(theta) - sum_rows N_row * log(Z_row(theta))

over theta. theta is shared by the transition, emission, and (when init_mode=’quantized’) initial state blocks. Note that without a k_max cap the likelihood always improves as theta -> 1 (the quantization grid becomes arbitrarily fine), so supply k_max (or fixed_theta) when a coarse, meaningful quantization is wanted.

Because the M-step rounds probabilities to the theta^k grid, near-symmetric states produced by a random initialization can round onto (nearly) identical grid points and leave EM at an exact fixed point (dense HMM EM escapes such saddles by amplifying tiny count asymmetries, which quantization erases). The raw expected counts still carry those asymmetries, so by default the estimator checks after each M-step for state pairs whose quantized log-probabilities differ by less than split_nats everywhere and pushes them split_nats apart along the strongest raw-count asymmetry (split_collapsed=True), restoring a hill-climbing direction for the next E-step; an unwarranted split is simply rounded back by the next M-step. Random restarts (e.g. mixle.inference.estimation.best_of) remain useful for multimodality, as with any HMM.

If included, the length of the sequences is modeled through a length distribution with support on non-negative integers.

class QuantizedHiddenMarkovModelDistribution(theta, levels, transition_exponents, emission_exponents, initial_exponents=None, init_mode='quantized', k_max=None, len_dist=NullDistribution(), name=None, terminal_values=None, use_numba=False, terminal_states=None)[source]

Bases: HiddenMarkovModelDistribution

Hidden Markov model distribution with quantized observation summaries.

Parameters:
compute_declaration()[source]
classmethod left_to_right(theta, levels, transition_exponents, emission_exponents, initial_exponents=None, **kwargs)[source]

Construct a left-to-right (upper-triangular) quantized HMM.

transition_exponents must be upper triangular: every entry strictly below the diagonal is a structural zero (negative exponent), so the hidden-state path is monotone non-decreasing (a Bakis chain). This makes a sentence’s state paths exactly its monotone segmentations – only polynomially many in the length (O(L^{n-1})) rather than the n^L of a general HMM – which bounds the path/sequence ambiguity. When the per-state emission supports are additionally disjoint the model is unambiguous (one path per sentence); then the structural descending-probability seek/unrank coincides with the exact marginal order (up to quantization granularity, no path over-count), which a general HMM’s structural seek cannot.

Raises:

ValueError – if transition_exponents is not square or not upper triangular.

Parameters:
Return type:

QuantizedHiddenMarkovModelDistribution

to_fisher(**kwargs)[source]

Forward-backward Fisher view for the quantized HMM.

estimator(pseudo_count=None)[source]

Create QuantizedHiddenMarkovEstimator matching this distribution’s configuration.

Parameters:

pseudo_count (Optional[float]) – Per-cell pseudo count for the initial, transition, and emission expected counts. When None, unobserved cells become structural zeros.

Returns:

QuantizedHiddenMarkovEstimator object.

Return type:

QuantizedHiddenMarkovEstimator

enumerator()[source]

Returns an exact descending-probability enumerator over observation sequences.

For the ordinary (length-distribution) case this is the quantized-HMM-specialized enumerator, which avoids constructing per-state categorical streams and uses the cached quantized emission log-probability matrix directly. For the terminal_values stopping-time case it delegates to HiddenMarkovModelEnumerator, which implements that support (the quantized specialization only covers the length-distribution path).

determinize(max_states=1 << 16)[source]

Weighted determinization (Mohri 1997; Mohri & Riley 2002) of this terminal-value quantized HMM into a DeterminizedSequenceDistribution.

Rebuilds the machine over belief states (exact rational arithmetic) so each sequence has a single path and edge weights multiply to the exact marginal – yielding exact, duplicate-free n-best sequences (not n-best paths). Requires terminal_values. Raises EnumerationError if the belief expansion exceeds max_states (the twins property fails – not finitely determinizable; keep the original HMM’s exact O(index) enumerate-and-bin path instead).

Parameters:

max_states (int)

class QuantizedHiddenMarkovModelEnumerator(dist)[source]

Bases: DistributionEnumerator

Exact best-first enumerator specialized for QuantizedHiddenMarkovModelDistribution.

Parameters:

dist (QuantizedHiddenMarkovModelDistribution)

class QuantizedHiddenMarkovEstimator(num_states, levels=None, pseudo_count=None, k_max=None, fixed_theta=None, init_mode='quantized', len_estimator=NullEstimator(), name=None, keys=(None, None, None), use_numba=None, max_quant_its=50, split_collapsed=True, split_nats=math.log(2.0))[source]

Bases: ParameterEstimator

Parameters:

  • num_states (int)

  • levels (Sequence[Any] | None)

  • pseudo_count (float | None)

  • k_max (int | None)

  • fixed_theta (float | None)

  • init_mode (str)

  • len_estimator (ParameterEstimator | None)

  • name (str | None)

  • keys (tuple[str | None, str | None, str | None] | None)

  • use_numba (bool | None)

  • max_quant_its (int)

  • split_collapsed (bool)

  • split_nats (float)

accumulator_factory()[source]

Returns a HiddenMarkovAccumulatorFactory with categorical emission accumulators.

Return type:

HiddenMarkovAccumulatorFactory

estimate(nobs, suff_stat)[source]

Estimate a QuantizedHiddenMarkovModelDistribution from Baum-Welch expected counts.

Sufficient statistics in arg ‘suff_stat’ are the HiddenMarkovAccumulator value:

suff_stat[0] (int): Number of hidden states. suff_stat[1] (np.ndarray): Initial state counts. suff_stat[2] (np.ndarray): State counts. suff_stat[3] (np.ndarray): State transition counts. suff_stat[4] (Sequence[Dict[Any, float]]): Per-state categorical emission counts. suff_stat[5] (Optional[Any]): Optional sufficient statistics of the length distribution.

Parameters:
Returns:

QuantizedHiddenMarkovModelDistribution object.

Return type:

QuantizedHiddenMarkovModelDistribution

QuantizedHiddenMarkovModelEstimator

alias of QuantizedHiddenMarkovEstimator