mixle.enumeration.streams module

Generic stream primitives for smart enumeration.

The building blocks shared by the best-first algorithms and the combinator enumerators: freeze (canonical hashable keys for de-duplicating support values), BufferedStream (random access by rank into a lazy descending-probability stream), merge_enumerators (k-way merge of sorted streams with disjoint supports), and supports_enumeration (the capability probe). See mixle.stats.compute.pdist.DistributionEnumerator for the enumeration contract.

freeze(x)[source]

Return a canonical hashable key for x, for de-duplicating support values.

Lists/tuples freeze element-wise to tuples, dicts to frozensets of (key, value) pairs, sets to frozensets, numpy arrays to (shape, dtype string, bytes), numpy scalars to their python equivalents, and NaN to a shared sentinel (so nan == nan for dedup purposes). Raises TypeError for values that cannot be canonicalized.

Cross-type equality policy (deliberate, not incidental): type is part of a value’s identity for dedup purposes, so this never falls back to Python’s native cross-type equality/hashing.

  • ndarrays are keyed by (shape, dtype.str, bytes). Bytes alone are ambiguous – e.g. an int8 array holding -56 and a uint8 array holding 200 have identical tobytes() output but represent different numbers – so omitting dtype would silently merge genuinely distinct support values.

  • Every other value is keyed by (type(x), x) rather than bare x, after numpy scalars are unwrapped to their python equivalent via .item(). Plain x would let Python’s own cross-type equality collide unrelated values that happen to compare equal – True == 1, False == 0, 1 == 1.0 – even though a typed distribution’s support can legitimately contain both a bool and an int (or an int and a float) as distinct outcomes. Note .item() collapses numpy integer width (int8/uint8/int64 all become python int), which is safe because .item() – unlike an array’s raw .tobytes() – already applies each dtype’s own signed/unsigned interpretation, so equal post-.item() values really are the same number.

  • The one exception: every NaN float collapses to a single shared sentinel regardless of which NaN produced it, since IEEE 754 nan != nan would otherwise make each NaN a unique, never-deduplicated key.

Parameters:

x (Any)

Return type:

Hashable

supports_enumeration(dist)[source]

Return True if dist.enumerator() can be constructed.

Return type:

bool

class BufferedStream(it)[source]

Bases: object

Random access by rank into a lazy stream of (value, log_prob) pairs.

get(i) extends an internal buffer as needed and returns the i-th item, or None if the stream has fewer than i+1 items. The underlying stream is consumed at most once regardless of how many consumers share this object.

Parameters:

it (Iterator[tuple[Any, float]])

get(i)[source]

Return buffered item i, pulling from the stream if needed.

i is an absolute, 0-based rank – not a Python sequence index – so it must be an exact, non-negative integer. This is checked before the buffer is touched at all, regardless of how much has been buffered so far: without this check, get(-1) would return the last currently buffered item via ordinary Python negative indexing once anything has been pulled, but raise an incidental IndexError from indexing an empty list otherwise – making a negative rank’s observable behavior depend on prior access history instead of always meaning “invalid.” bool is rejected too despite being an int subclass, and a whole-valued float such as 2.0 is rejected as well: a rank is always exactly integral, never merely numerically equal to one.

Parameters:

i (int)

Return type:

tuple[Any, float] | None

merge_enumerators(streams, offsets)[source]

Lazy k-way merge of sorted (value, log_prob) streams with per-stream offsets.

Stream k’s log probs are shifted by offsets[k]. Correct only when the streams have pairwise disjoint supports (no de-duplication or re-scoring is performed).

Validated eagerly at call time, before any stream is touched: offsets must have exactly one entry per stream, and every offset must be finite or exactly -inf – the sentinel for “this stream contributes nothing” (e.g. a zero-weight mixture component), which excludes that stream entirely without ever opening its iterator. NaN and +inf offsets are rejected outright, since neither has a sensible reading as a log-probability shift. This function is a thin eager-validation wrapper; the actual merge runs in the generator it returns, so a malformed call raises immediately instead of on first iteration.

Each stream’s own items are validated lazily – one pull at a time, as they are actually consumed, so this is safe for infinite streams and never reads ahead: a non-finite log_prob (including -inf – a properly-formed descending stream should already exclude impossible/zero-probability items, matching the convention used elsewhere in this package, e.g. best_first.py’s if lp > -np.inf guards) raises immediately, and so does a log_prob that exceeds the previous log_prob pulled from that SAME stream – i.e. the stream failing the k-way merge’s fundamental precondition of actually being sorted descending.

Parameters:
Return type:

Iterator[tuple[Any, float]]