mixle.substrate.eig_retrieve module

Information-gain retrieval over substrate items.

Score substrate items by how much they would actually move a belief, not by how textually similar they are to the query.

retrieve() ranks by cosine/lexical similarity – a sound default, but many similar-looking items can carry the same evidence (redundant), while a single differently-worded item can be decisive. eig_retrieve() instead scores each candidate by the entropy it would actually remove from a given BeliefState if assimilated, and greedily picks the highest-gain item each round, updating the running belief before scoring what remains – so a second item redundant with the first correctly scores near zero the next round. Experiment-design workflows can reuse the same greedy EIG scorer; it is written once, here.

Expected, not realized, gain (MXR-080-0250). Most substrate items are already fully in hand – their content, and therefore their evidence, is deterministically known before scoring (evidence_fn just reads it off, e.g. a precomputed log-likelihood payload). A genuine experiment-design candidate is different: what it would show is not known until it is actually run. For those, evidence_fn returns EvidenceOutcomes – the candidate’s own predictive distribution over what its evidence could turn out to be – and the score becomes the true expected posterior-entropy reduction, averaged over that distribution, rather than the entropy change from one arbitrarily-picked realization (which can diverge sharply from the real expectation: entropy is a concave functional of the belief, so “entropy after updating on the average evidence” is not “the average of the entropy after each possible update”). A deterministic evidence value is the degenerate, single-outcome case of the same formula, so both cases share one code path. A candidate is also never picked with a non-positive expected gain: once the best remaining candidate cannot clear a strictly-positive bar, eig_retrieve() stops rather than present a harmful (entropy-increasing) or useless (zero-information) pick as if it were beneficial evidence.

Copy-on-update, narrow failure handling (MXR-080-0251). update() is documented to return a new belief, but nothing enforces that a given realization actually honors it – so every hypothetical update here runs against a fresh clone of the running belief, never the running belief itself, and the caller’s own belief argument is never mutated by this function no matter how update() is actually implemented. Only a declared evidence-incompatibility (ValueError, the same type BeliefState realizations in this codebase raise for malformed/incompatible evidence) is treated as “this candidate has no usable evidence”; any other exception is a genuine bug and propagates. Every skipped candidate is recorded, with why, on the returned EigRetrieval – never silently dropped.

class EvidenceOutcome(probability, evidence)[source]

Bases: object

One possible realization of a candidate’s evidence, weighted by its probability of occurring.

Parameters:
class EvidenceOutcomes(outcomes)[source]

Bases: object

A candidate’s predictive distribution over what its evidence could turn out to be (MXR-080-0250).

Return this from evidence_fn instead of a bare evidence value when a candidate’s true evidence is NOT yet known at scoring time – e.g. a not-yet-run experiment or a diagnostic whose result is uncertain until performed. outcomes is the finite set of what it could resolve to, each paired with its probability under the current belief’s own predictive distribution; eig_retrieve() scores the candidate by the true expectation, H[belief] - E[H[belief | outcome]], over this set, rather than the entropy change from a single realized update. Validated eagerly, the same discipline CategoricalBelief applies to its own probabilities: every probability must be finite and non-negative, and they must sum to 1 (within floating-point tolerance) – a caller that gets this wrong finds out immediately, at the point they built the outcome set, not several stack frames away inside a confusing entropy computation.

Parameters:

outcomes (Sequence[EvidenceOutcome])

class SkippedCandidate(item, reason)[source]

Bases: object

A substrate item scoring could not use, and why (MXR-080-0251) – surfaced, never just dropped.

Parameters:

  • item (SubstrateItem)

  • reason (str)

class EigRetrieval(query, items=None, scores=None, *, skipped=None, stop_reason=None)[source]

Bases: Retrieval

A Retrieval from eig_retrieve(), plus the diagnostics a similarity retrieval has no use for: which candidates scoring skipped (and why), and whether the run stopped early because no remaining candidate cleared the admissible-gain bar (MXR-080-0250/0251).

Deliberately a plain subclass with its own __init__ rather than a second @dataclass layer: Retrieval itself is @dataclass(init=False) with a hand-written constructor that builds its validated hits tuple from items/scores (MXR-080-0248, e.g. rejecting a length mismatch or a non-finite score) – re-declaring EigRetrieval as its own @dataclass would let dataclass field inheritance synthesize a DIFFERENT __init__ over the raw field list instead (query, hits=(), skipped=..., stop_reason=...), silently bypassing that validation. Delegating to Retrieval.__init__ via super() keeps construction going through the one real validation path regardless of how Retrieval represents itself internally.

Parameters:

  • query (str)

  • items (list[SubstrateItem] | None)

  • scores (list[float] | None)

  • skipped (list[SkippedCandidate] | None)

  • stop_reason (str | None)

eig_retrieve(substrate, belief, evidence_fn, *, k=8, kind=None, scope=None, seed=None)[source]

Greedily pick up to k substrate items by expected posterior-entropy reduction against belief.

evidence_fn(item) turns a candidate item into whatever belief.update(...) expects (e.g. a per-hypothesis log-likelihood vector for a CategoricalBelief`), OR – when the item’s true evidence is not yet known – an EvidenceOutcomes describing what it could turn out to be. Each round, every remaining candidate is scored by the expected entropy reduction current.entropy() - E[updated.entropy()] (MXR-080-0250; the exact realized reduction in the common deterministic-evidence case, since that is the same formula’s single-outcome degenerate case); the best-scoring item is taken – but only if its gain clears zero, otherwise eig_retrieve() stops rather than hand back a pick that would increase uncertainty (or add none) dressed as evidence – the running belief moves to its post-update state, and scoring repeats against the shrunk pool, so an item whose evidence is redundant with an already-picked item scores near zero on its next look, the direct fix for similarity retrieval pulling in near-duplicates.

belief itself is never mutated: every hypothetical update runs against a fresh clone (MXR-080-0251), so a candidate that turns out not to be chosen – or a BeliefState realization whose update() does not honor the documented “returns a new belief” contract – can never contaminate scoring for the rest of the round. evidence_fn should raise ValueError to signal “no usable evidence for this item” (mirroring what belief.update() itself raises for incompatible evidence); that is the only exception type treated as a skip – every skip is recorded, with its reason, on the returned EigRetrieval rather than silently dropped, and any other exception is a genuine bug and propagates. seed (an int or numpy.random.RandomState) is used only when a chosen candidate carries genuine outcome uncertainty, to draw which outcome is treated as realized for the next round’s baseline.

Returned as an EigRetrieval (query is a fixed marker, not a text query) so it composes with the same to_context/by_kind surface as cosine retrieval, plus .skipped and .stop_reason diagnostics cosine retrieval has no equivalent of.

Parameters:
Return type:

EigRetrieval