mixle.experimental.ssm_hybrid module

E5 (part 2): the hybrid block – local attention + selective-scan SSM + E2’s moment-closure far field, composed in ONE ContextMechanism, with a real per-mechanism contribution receipt. See notes/designs/E5.md for the full design: why these three mechanisms (exact short-range, smooth long-range, sparse extreme-long-tail) are complementary rather than redundant, the exact fusion architecture (near+far combined by E2’s own joint softmax; that combined attention branch and the SSM branch fused by a separate learned 2-way gate), and why the contribution receipt is a real softmax-mass reading rather than a fabricated importance score.

Reuses, without reimplementing:

  • mixle.experimental.context_spine: _rope_angles/_apply_rope/SlidingWindowState (E1’s near field, exactly the code path SlidingWindowSpine.step uses).

  • mixle.experimental.moment_closure_attention: ClusterBank/_empty_cluster_bank/ mgf_cluster_attention/cluster_responsibilities/update_cluster_bank/birth_and_merge (E2’s far field, verbatim).

  • mixle.experimental.selective_scan: _scan_layer/_s4d_real_a_log_init/_dt_bias_init (E5 part 1’s S6 recurrence and its verified init, verbatim – the ONE scan implementation, not a second one).

class HybridState(near, banks=<factory>, ssm_h=<factory>)[source]

Bases: object

Per-layer carried state: E1’s near-field cache, E2’s far-field ClusterBank, and E5 part 1’s SSM hidden state – one list per mechanism, indexed by layer, matching E1/E2’s existing per-layer-list convention so nothing about the state shape is new.

Parameters:
class HybridBlock(vocab, *, d_model=32, n_layer=2, n_head=2, window=16, d_state=16, ssm_expand=2, max_clusters=4, birth_threshold=-2.0, merge_threshold=None)[source]

Bases: Module

ContextMechanism (E1 protocol): per layer, per position, combines (a) E1-style windowed exact attention, (b) E2’s far-field ClusterBank mixture attention – (a)+(b) joined by ONE softmax, per E2.md section 3.3 – and (c) a selective-scan SSM branch (E5 part 1’s _scan_layer), fused with the combined attention output via a learned per-position 2-way softmax gate (notes/ designs/E5.md section 2). report() exposes the real per-mechanism contribution receipt after a step() call (section 3), an instance-level side channel populated by step the same way MomentClosureAttention.last_misfit/last_receipts are.

Parameters:

  • vocab (int)

  • d_model (int)

  • n_layer (int)

  • n_head (int)

  • window (int)

  • d_state (int)

  • ssm_expand (int)

  • max_clusters (int)

  • birth_threshold (float)

  • merge_threshold (float | None)

report()[source]

The per-mechanism contribution receipt from the most recent step() call: fractional share of the fused output attributable to each of (local, far_field, ssm), summing to 1.0 by construction (notes/designs/E5.md section 3) – a real reading of the learned gate’s and joint softmax’s own weights, not a fabricated importance score.

Return type:

dict[str, float]

log_density(x, y)[source]

x, y: (n, T) long tensors. Returns -mean_per_position_nll per row, each scored independently (fresh state per row) via one init_state + step call, exactly the SelectiveScan.log_density convention (see notes/designs/E5.md section 4 for the caveat this inherits from E2: cluster birth/merge is only independent across rows because each row is scored with its own fresh, unbatched stream, not scored together).

Parameters:
Return type:

Any