mixle.task.plan_refine module

Outcome-trained plan refinement beyond imitation.

Imitating harvested/teacher decompositions (sft_planner()) can only reproduce known workflows. This expert-iteration step samples candidate plans from the current planner (sample_plans()), verifies each with an executable checker, and retrains the plan-writing LM on verified-successful candidates:

planner = sft_planner(teacher, requests, tools)              # imitation baseline
planner, report = outcome_refine_planner(planner, tasks, verify_fn)
report.solve_rate_before, report.solve_rate_after            # measured, not assumed

verify_fn(task, plan) -> bool must be an executable or ground-truth check, such as a VerifiableOracle for the plan-decomposition domain.

This module implements one propose-verify-retrain round on a synthetic tool-world. The full expert-iteration outer loop, DPO preference learning over plan pairs, experiment-design-as-planning, and orchestrator runtime are separate surfaces.

class RefinementReport(tasks, verified_gain_pairs, solve_rate_before, solve_rate_after)[source]

Bases: object

Measured account of one outcome-refinement round.

Parameters:

  • tasks (int)

  • verified_gain_pairs (int)

  • solve_rate_before (float)

  • solve_rate_after (float)

outcome_refine_planner(planner, tasks, verify_fn, *, k=5, temperature=0.8, epochs=15, lr=1e-3, seed=0)[source]

Run one propose-verify-retrain round and return the planner plus report.

For each task: sample k candidate plans (sample_plans()), keep the ones verify_fn accepts, and for tasks with at least one verified success – add the highest-scoring verified candidate as a new supervised-fine-tuning pair. Fine-tunes the LM on every such pair in one fit_pairs call. solve_rate_before/_after are measured on the same held-out tasks via the planner’s own single-shot try_plan (matched budget), before and after the retrain – not an aggregate over the k samples used to harvest the training signal.

Parameters:
Return type:

tuple[GenerativePlanner, RefinementReport]