mixle.task.probe_policy module

Learned non-myopic probing policy versus myopic expected information gain.

The head-to-head comparison is deliberately conservative. If the learned non-myopic policy does not beat myopic EIG on solve-rate at matched oracle budget in the exploration world, the result should be treated as evidence to keep the simpler myopic policy. Myopic is often near-optimal; the learned path is useful only when delayed or combinatorial payoff makes one-step EIG miss valuable probes.

Two policies compared head-to-head, same action menu, same budget:

• myopic_eig_policy() – greedy, ONE STEP of lookahead: at every decision, picks the action (across ALL undrilled cells, survey or drill) with the highest per-cost expected information gain about that cell’s target status. A cell’s current belief uncertainty is its survey-noise; a read’s “borderline-ness” (how close to the decision boundary between target/non-target reads) governs how much resolving it is actually worth – a read far from the boundary is already confidently classified, so probing it teaches little.

• the outcome-trained decomposer’s plan model (outcome_decomposer) – trained via expert iteration against verifier-grounded reward (terminal world score, no learned reward model): propose whole action-type sequences, execute, keep the verifiably-successful ones, refit, iterate. Optimizing for a whole sequence’s terminal score (rather than one greedy step) is the non-myopic half of this comparison – it can trade an immediately-worse-looking step for a better final outcome, which pure one-step EIG structurally cannot.

  directed = train_outcome_decomposer(…) result = head_to_head_probe(directed.plan_model, held_out_seeds=range(…), …) result.non_myopic_wins

class ProbeHeadToHead(non_myopic_score, myopic_score, non_myopic_wins)

Bases: object

Held-out comparison between the non-myopic probe policy and a myopic baseline.

Parameters:

• non_myopic_score (float)

• myopic_score (float)

• non_myopic_wins (bool)

head_to_head_probe(plan_model, *, held_out_seeds, n_cells, n_targets, budget)

Compare the non-myopic (outcome-trained) plan model against the myopic EIG policy on the same held-out seeds at matched budget.

Parameters:

• plan_model (PlanModel)

• n_cells (int)

• n_targets (int)

• budget (int)

Return type:

ProbeHeadToHead

myopic_eig_policy(world)

One-step-lookahead policy, explicitly information-theoretic: EXPLOIT (drill) the most confident current target-candidate once its belief clears _DRILL_CONFIDENCE; otherwise EXPLORE (survey) the single most UNCERTAIN cell – maximum current entropy, i.e. the read closest to the decision boundary relative to its own noise, the textbook expected-information-gain target. No lookahead beyond this one step – by construction, it cannot see that an apparently-mediocre probe now sets up a better probe later.

Parameters:

world (ExplorationWorld)

Return type:

dict | None