mixle.doe.sequential module

Sequential design loop – the one orchestration primitive both adaptive-design experiments proved was missing, extracted as a small, composable driver instead of hand-rolled per demo.

Every adaptive-design pipeline built end-to-end (adaptive groundwater monitoring, adaptive gravity survey design) hand-wrote the same stateful loop: fit a model to the data so far -> summarize its uncertainty -> ask a controller whether the uncertainty is tight enough to stop or another sample is needed -> if continuing, use a design criterion to propose the next sample -> acquire it, append, repeat. This is that loop, parameterized. It is deliberately NOT a general workflow engine (branching DAGs, autonomous pipeline composition from a natural-language goal): that broader orchestration layer is a real but separate design question whose scope depends on intended usage, left open on purpose. This is only the part that is unambiguously right regardless of that answer, because a sequential experimental-design loop is a real thing people run either way, and both demos proved its absence forces hand-rolling.

It composes the rest of this codebase’s decision machinery rather than reinventing it:

• should_continue is any (history) -> {"keep_going": bool, "reason": str} callable – e.g. wrap mixle.analysis.real_options.voi_stopping_decision() (stop when the value of the next sample drops below its cost) or an LLM controller via mixle_mlops.core.decisions.structured_decision (a forced, un-self-contradictable STOP/CONTINUE).

• propose is any (state, history) -> action – e.g. wrap mixle.doe.active.expected_information_gain_linear() / a monitoring-network design.

• a calibration guard (mixle.inference.calibration_gate) can be dropped into summarize so a round whose posterior is miscalibrated is flagged in the record the controller sees.

The audit trail (SequentialDesignResult) is meant to be trustworthy even when a round goes wrong: summary/decision records are detached portable JSON values, and every callback, validation, and history-copy failure is recorded as an explicit failed round before the selected on_error policy returns or raises – never a silent, traceless abort.

class DesignRound(index, state, summary, decision, proposed_action=None, failed=False, failed_step=None, error=None)

Bases: object

One round of a sequential design: the fitted state, its uncertainty summary, the controller decision that round, and the proposed_action chosen next (None on the final round, where the loop stopped and proposed nothing – or on a round that failed before a proposal was made, see below).

summary and decision are the small, JSON-friendly audit fields: what sequential_design() stores is always a deep copy of what the callback returned, and what later callbacks see via history is always a deep copy of what is stored, so no callback (nor any code holding a reference to a value it once returned) can retroactively rewrite a past round’s record. state and proposed_action are shared by reference – they may be large or otherwise-uncopyable fit artifacts (a GP posterior, a torch module) that callbacks should treat as read-only by convention.

If any callback, record validation, or history-copy stage fails during this round, that failure is recorded rather than left implicit: failed is True, failed_step names the exact stage, and error is a short f"{type}: {message}" description of the exception. A failed round’s state/summary/decision reflect whatever completed before the failure.

Parameters:

• index (int)

• state (Any)

• summary (dict[str, Any])

• decision (dict[str, Any])

• proposed_action (Any)

• failed (bool)

• failed_step (str | None)

• error (str | None)

exception SequentialDesignError(message, *, result)

Bases: RuntimeError

Raised when a callback, validation, or history-copy stage fails under on_error="raise".

result is the partial audit trail up to and including the failed round (see DesignRound.failed), so a caller that wants both fail-fast propagation and the ability to inspect what happened can do:

try:
    sequential_design(..., on_error="raise")
except SequentialDesignError as exc:
    log(exc.result.rounds)  # the audit trail survives the raise
    raise

The original callback exception is available as __cause__ (standard exception chaining).

Parameters:

• message (str)

• result (SequentialDesignResult)

Return type:

None

class SequentialDesignResult(rounds=<factory>, stopped_reason='')

Bases: object

The full audit trail of a sequential design: every round in order, plus why it stopped.

Parameters:

• rounds (list[DesignRound])

• stopped_reason (str)

sequential_design(initial_data, *, fit, summarize, should_continue, propose, acquire, combine, max_rounds, on_error='raise')

Run a stateful sequential experimental-design loop and return its full audit trail.

Each round: fit(data) -> state; summarize(state, round_index) -> uq_summary; should_continue(history) -> {"keep_going": bool, "reason": str, ...}. If the controller stops (or the round budget is hit), the loop ends. Otherwise propose(state, history) -> action picks the next sample, acquire(action) -> new_data obtains it (a real simulation/survey/measurement), and combine(data, new_data) -> data folds it in for the next round.

Parameters:

• initial_data (Any) – the starting dataset (whatever fit consumes).

• fit (Callable[[Any], Any]) – data -> state (e.g. a posterior). Called once per round on the accumulated data.

• summarize (Callable[[Any, int], dict[str, Any]]) – (state, round_index) -> a JSON-friendly uncertainty summary dict. This is what the controller sees, so put the decision-relevant numbers (and any calibration flag) here.

• should_continue (Callable[[list[DesignRound]], dict[str, Any]]) – (history-so-far, including the current round’s state/summary) -> a dict with an actual Boolean "keep_going" and a nonempty string "reason". Everything else in the dict is recorded as the round’s decision. history is a copy-safe view (see DesignRound) – mutating it does not affect the returned audit trail.

• propose (Callable[[Any, list[DesignRound]], Any]) – (state, history) -> the next action/design point. Return None to stop the loop even though the controller wanted to continue (no admissible next sample) – stopped_reason is then "no_proposal". history is the same copy-safe view.

• acquire (Callable[[Any], Any]) – action -> the new observation(s) from actually taking that sample.

• combine (Callable[[Any, Any], Any]) – (data, new_data) -> the updated dataset for the next round.

• max_rounds (int) – maximum number of adaptive acquisitions after the initial fit. The audit trail therefore contains at most max_rounds + 1 fitted states: index 0 for the initial data, followed by one state per acquired sample.

• on_error (str) –

  what to do when any callback, audit-record validation, or history-copy stage raises. In every case the failure is first recorded as an explicit failed round (see DesignRound.failed) so the audit trail always shows what happened – the policy only controls what happens after that:

  • "raise" (default): re-raise as SequentialDesignError (chained to the original exception via __cause__), carrying the partial result as .result. Matches the historical behavior of letting the exception propagate, but the caller can now still recover the audit trail up to the failure by catching it.

  • "record_and_stop": swallow the exception and return the partial result normally, with stopped_reason == "callback_error" and the failed round’s failed/failed_step/ error describing what happened.

  There is deliberately no “continue past a failed round” option: a round that fails in propose/acquire/combine never obtained new data, so the next round would just re-fit on the same data – not meaningful forward progress. Retry that yourself (e.g. by calling sequential_design() again) if that is what you want.

Returns:

A SequentialDesignResult – every DesignRound in order plus stopped_reason.

Return type:

SequentialDesignResult