mixle.inference.placement module

Placement planning for certified estimation blocks.

The estimation certificate identifies how each block is solved and whether it is eligible for offloading. Placement adds the execution decision: local or pool. Blocks stay local by default. A block is assigned to the pool only when it is pool-eligible, a pool is configured, and the estimated work clears the round-trip cost threshold.

The result is a PlacementPlan with per-block placement, an estimated cost, and a human-readable reason. With no pool configured, the plan is all-local.

class PoolSpec(available=False, cost_per_hour=0.5, flop_threshold_tflop=1.0, pool_speedup=10.0)[source]

Bases: object

What the shared GPU pool offers, and when offloading is worth it.

available: is a pool configured at all (else everything is local). cost_per_hour: the pool’s price. flop_threshold: the estimated work (in TFLOP) below which the round-trip is not worth it – small gradient blocks stay local. local_tflops / pool speedup drive the estimate.

Parameters:
available: bool = False
cost_per_hour: float = 0.5
flop_threshold_tflop: float = 1.0
pool_speedup: float = 10.0
class BlockPlacement(name, kind, placement, reason, est_tflop=0.0, est_cost=0.0)[source]

Bases: object

Where one block runs and why: ‘local’ or ‘pool’, with the priced justification.

Parameters:
name: str
kind: str
placement: str
reason: str
est_tflop: float = 0.0
est_cost: float = 0.0
class PlacementPlan(placements=<factory>, pool_spec=None)[source]

Bases: object

The full local-vs-pool plan for a model’s blocks, derived from its certificate + a PoolSpec.

Parameters:

  • placements (list[BlockPlacement])

  • pool_spec (PoolSpec | None)

placements: list[BlockPlacement]
pool_spec: PoolSpec | None = None
property pool_blocks: list[BlockPlacement]
property local_blocks: list[BlockPlacement]
property est_pool_cost: float
as_dict()[source]
Return type:

dict[str, Any]

report()[source]
Return type:

str

plan_placement(certificate, pool=None, *, telemetry=None)[source]

Decide local vs pool for every block of a certified fit (see module docstring).

Closed-form / convex / EM blocks always stay local. A gradient (pool_eligible) block goes to the pool only when pool.available and its estimated work clears pool.flop_threshold_tflop; otherwise it stays local too. Each decision carries a priced reason, and (with a pool) a placement telemetry event is emitted per block.

Parameters:

  • certificate (EstimationCertificate)

  • pool (PoolSpec | None)

  • telemetry (Any)

Return type:

PlacementPlan