mixle.epistemic.loop module

One epistemic-loop step: OBSERVE -> UPDATE -> ABDUCE (on surprise) -> PREDICT -> DISCRIMINATE -> ACT.

step() is a pure function the caller drives from their own loop (interactive, scripted, or agentic – this module doesn’t prescribe which). It is the integration point: everything in discrepancy, portfolio, and likelihood is a building block; this is where they compose. There is deliberately no multi-step run_until(...) driver and no persistence beyond one EpistemicStep’s own fields here – the program plan’s “episode”/investigation-trace concept (§4.1) is training-data machinery, out of scope for this plan (see notes/epistemic-loop-integration-workplan.md §6).

ACT’s expected-information-gain scoring does not call mixle.doe.active.expected_information_gain_nmc() directly: that function’s nested-Monte-Carlo estimator is written against a continuous numpy parameter space (prior_sampler(rng, n) -> (n, k) array), while a HypothesisPortfolio is a discrete weighted set of arbitrary typed hypothesis payloads. Forcing the portfolio through that interface would mean either requiring every hypothesis payload to be a numpy vector (defeating the point of a typed portfolio) or building a lossy adapter. Instead, _portfolio_eig_nmc below is the same nested-Monte-Carlo EIG estimator (Ryan 2003), rewritten one level down against the portfolio’s own discrete weighted draws – same math, the right data shape.

ActionLikelihood

(hypothesis, action, observation) -> p(y | h, a) – the ACTION-CONDITIONED outcome law.

MXR-080-1896. ACT’s EIG = E[log p(y|h,a) - log sum_h' w_h' p(y|h',a)] is an expectation over the outcome law of the experiment ``a``, and simulate_fn(h, a, rng) already draws from that law. The matching density had nowhere to live: step()’s likelihood is a two-argument (hypothesis, observation) LikelihoodStrategy, so whenever simulate_fn’s law genuinely depended on a the estimator was pairing draws from q(y|h,a) with a density p(y|h) from a different distribution. That ratio is not a mutual information and is not even sign-constrained: mixle.task.discrepancy_invention_loop’s probe, which rejection-samples the truncated law p(y|h, |y-a| <= w), reported “EIG” values of -1.15 nats for four of five candidate actions where the true action-conditioned EIG was +0.04 – so the number being maximized was not the information gain of the experiment being commissioned.

This is an optional second density, not a replacement: action_likelihood=None keeps the existing two-argument behaviour verbatim, which stays exactly right whenever the outcome law does not depend on the action (p(y|h,a) == p(y|h)) – the common case for a simulate_fn that merely predicts from the hypothesis. Nothing here can detect which case a caller is in (a simulate_fn is opaque), so declaring it is the caller’s job and there is deliberately no guard that rejects the two-argument form.

alias of Callable[[Hypothesis, Any, Any], float]

class EpistemicStep(observation, portfolio_before, portfolio_after, surprise, next_action, next_action_eig)[source]

Bases: object

The full outcome of one loop iteration – everything EpistemicJournal logs.

Parameters:

  • observation (Any)

  • portfolio_before (HypothesisPortfolio)

  • portfolio_after (HypothesisPortfolio)

  • surprise (float)

  • next_action (Any | None)

  • next_action_eig (float | None)

step(portfolio, observation, likelihood, *, action_space=None, simulate_fn=None, action_likelihood=None, cost_fn=None, lam=1.0, surprise_threshold=None, propose_fn=None, n_outer=64, n_inner=64, rng=None)[source]

One loop iteration: reweight on observation, optionally abduce on surprise, optionally act.

UPDATE: portfolio.reweight(observation, likelihood). ABDUCE: only when surprise_threshold is set and the portfolio’s surprise_score() on observation meets or exceeds it, propose_fn(updated_portfolio) is called; a non-None return is folded in via _add_hypothesis() (program plan §3.5’s surprise trigger, at the scope this plan covers – schema-expansion / human-checkpoint semantics are not modeled here). ACT: when action_space is given, each candidate is scored by EIG(a) - lam * cost_fn(a) (program plan §2’s a* = argmax_a EIG(a) - lambda*cost(a)) via _portfolio_eig_nmc against the updated portfolio, and the argmax is returned; action_space=None is a valid “just update the belief” call and returns next_action=None. Raises ValueError if action_space is given without simulate_fn – EIG estimation needs a way to generate a predicted observation per hypothesis per action, and there’s no honest default for that.

UPDATE evaluates each likelihood exactly once (see _OnceLikelihood): surprise and the posterior are derived from the same evidence, so a stateful or stochastic likelihood can no longer make one step report two different beliefs, and an expensive likelihood is not paid for twice.

action_likelihood (MXR-080-1896, see ActionLikelihood) declares the ACTION-CONDITIONED outcome density p(y|h,a) that ACT’s own formula names. Supply it whenever simulate_fn’s outcome law depends on the action – with it left at None the two-argument likelihood is used for scoring, which is right only when it does not. UPDATE is unaffected either way: it conditions on an observation the caller has already obtained, not on a simulated one, so it keeps using likelihood.

MXR-080-1896, selection vs. final evidence: next_action_eig is NOT the score that won the argmax. max_a EIG_hat(a) over noisy Monte Carlo estimates is the winner’s curse – it is biased high for the winner precisely because an upward error is what made a candidate win (measured at +0.054 nats on a ~0.2 nat probe, a ~27% overstatement). The argmax selects; the winner’s EIG is then re-estimated once on an INDEPENDENT draw stream and that unbiased number is what gets reported and journaled. Only the reported magnitude changes – which action is chosen is still decided by the first pass, since re-scoring every candidate would just recreate the same curse.

The ACT economics are validated rather than trusted: lam and every cost_fn result must be finite and non-negative, and n_outer/n_inner must be exact positive integers. Without those checks a NaN cost or EIG made score > best_score false for every candidate, so a one-candidate action space returned “no action worth taking” instead of reporting that its evidence was invalid; a negative cost was scored as a reward, and a negative lam inverted the declared cost penalty into a bonus.

Parameters:
Return type:

EpistemicStep