The all-to-all in expert-parallel MoE dispatch is one of those things everyone treats as load-bearing. The harness didn’t, and it was right.
The proposal
Given the routing statistics of the actual workload, most tokens reach a small, stable set of experts. The rewrite replaced the global exchange with a locality-aware two-stage dispatch and a fallback path for the tail.
Why we didn’t believe it
A 34% decode latency cut at EP=32 is the kind of number that usually means a broken benchmark. It took two days of re-running with different routing seeds, different batch shapes and a numerics audit before we accepted it.