mixle.utils.parallel.training_launchers module

Optional cluster launchers for distributed-gradient sessions.

Ray Train and Lightning Fabric manage worker lifecycle. They do not alter the statistical update or pretend to implement model-parallel kernels; each worker still selects torch_native or megatron and materializes its plan.

class LightningFabricLauncher

Bases: object

Create Fabric for pure DDP or FSDP jobs.

Lightning is capability-gated to data parallelism here. Model parallel dimensions remain the responsibility of the native DeviceMesh or Megatron backend rather than being accepted and discarded by Fabric.

class RayTrainLauncher

Bases: object

Launch one worker per physical plan rank with ray.train.torch.

class WorkerTopologyAttestation(rank, world_size, coordinate)

Bases: object

Runtime proof binding one launched rank to one planned mesh coordinate.

Parameters:

• rank (int)

• world_size (int)

• coordinate (tuple[tuple[str, int], ...])

attest_worker_coordinate(plan, *, actual_world_size, rank)

Validate a runtime rank and deterministically bind it to the plan mesh.

Parameters:

• plan (ParallelPlan)

• actual_world_size (int)

• rank (int)

Return type:

WorkerTopologyAttestation