Every frontier model release starts the same 26-day clock: the public record shows it took the community about that long to bring MI355X from first-light to peak on DeepSeek-V4-Pro, across 31 merged PRs (InferenceX). Our claim is that most of those 26 days are search, not insight — and search is what the harness is for. V4 Flash was the first release since our Ascend run where we could test that claim on a clock we didn’t control.
The setup: one node, 8× MI355X, ROCm 7.2, our runtime, no human commits after the run started. Workload is 8K/1K chat-style, the same shape InferenceX uses, so the numbers are comparable. Target metric is throughput per GPU at a fixed interactivity floor of 8 tok/s/user.
Note: All results in this post are reproducible with
hoid optimize --model deepseek-v4-flash --target mi355x --workload 8k1kon runtime 0.4.2. Configs, traces, and the full run log are in the companion repo.
Hour 0: first light
First light — the model running correctly, no optimization — came 41 minutes after download finished, at 340 tok/s/GPU. Most of that time was the correctness oracle building its reference activations at FP32 on CPU, which we require before the agent is allowed to touch anything. Slow, deliberate, non-negotiable: an agent optimizing for tok/s will happily break numerics to get there, and every candidate it proposes for the rest of the run is diffed against these references with per-layer tolerance budgets.
The initial profile told a familiar story. V4 Flash keeps the sparse attention indexer from V4-Pro but pairs it with a smaller, wider expert layout — 384 experts, 8 active. On an unoptimized run the indexer alone was 43% of decode wall clock.
Hours 1–19: the search
From here the loop is the same one we’ve described before: profile, hypothesize, generate candidates, verify, benchmark, keep or discard. Overnight the harness evaluated 4,100 kernel candidates across 61 hypotheses. Three landings did most of the work:
- TileLang sparse-MLA rewrite (h6). The stock indexer materializes the top-k mask before the attention gather. The winning candidate fuses selection into the gather and keeps the mask in registers — 2.4× on the indexer, worth 31% end-to-end.
- FP4 MoE fusion (h11). Grouped GEMM with dequant fused into the epilogue of the routing gather. On CDNA4 this is mostly a memory-layout problem; the agent found a swizzle we would not have tried by hand.
- Fused RoPE + Hadamard (h17). Small individually, but it removed the last elementwise kernel between two big GEMMs, which unlocked a graph-level fusion the scheduler had been proposing and rejecting for nine hours.
A representative slice of the run log, at the moment the MoE fusion landed:
# run 8842f3 · hypothesis 11 · candidate 2,317/4,100
[h11.c214] verify PASS max_err 3.1e-3 (budget 5e-3, layer 61/61)
[h11.c214] bench 1,412 → 1,688 tok/s/GPU (+19.5%)
[h11.c214] decision KEEP promoted to trunk
[sched] replan moe_dispatch now off critical path
[sched] next retire hypothesis h09 (superseded), spawn h17, h18
The two dead ends
We publish these because the failures describe the system better than the wins do.
Dead end #1: the wide-EP mirage (4 hours)
The harness spent four hours pursuing expert-parallel layouts copied from its V4-Pro experience. On V4-Pro, wide EP is the whole game. On Flash’s 8-active-of-384 layout, the all-to-all cost exceeds the compute it saves at single-node scale — every EP candidate verified correct and benchmarked worse. The interesting failure is that prior experience acted as a bad prior; the fix (landed since) is discounting cross-model priors when the expert activation ratio differs by more than 2×.
Dead end #2: the fast, wrong kernel (2 hours)
Candidate h13.c88 hit 2,340 tok/s/GPU at hour 14 — a 12% jump — and failed
verification by a hair: max error 6.2e-3 against a 5e-3 budget, on 3 of 61
layers. The agent spent two hours trying to rescue it before the scheduler cut
losses. A human would have been tempted to widen the tolerance. The budget is not
the agent’s to negotiate, and eval runs later confirmed the caution: the rescued
variant we tested manually dropped 1.9 points on GPQA. Speed you can’t trust is
not speed.
Results
| Checkpoint | Time | tok/s/GPU | vs first light |
|---|---|---|---|
| First light | h0 | 340 | 1.0× |
| Sparse-MLA rewrite | h6 | 890 | 2.6× |
| FP4 MoE fusion | h11 | 1,688 | 5.0× |
| RoPE/Hadamard + graph fusion | h17 | 2,410 | 7.1× |
| End of run | h34 | 2,910 | 8.6× |
| Hand-tuned reference branch | day 26* | 2,655 | 7.8× |
* The reference branch is the community V4-Pro recipe rebased onto Flash by us, not an independent day-26 effort — Flash is two days old. It is the strongest human-tuned baseline available today, not the strongest possible. We’ll rerun this comparison when the community branch matures, and we’ll publish that result whichever way it goes.
At the 8 tok/s/user floor, 2,910 tok/s/GPU works out to $0.11 per million output tokens at current MI355X cloud pricing — in FP4, with every layer inside tolerance budgets and a <0.3-point average delta across our eval suite. Full Pareto curves against B200 are in the companion repo.
What we think this means
One run is one run — this section is opinion, not measurement.
The 26-day clock is a coordination cost, not a knowledge cost. Nothing the harness landed this weekend was undiscoverable by humans; the sparse-MLA fusion resembles work that shipped for V4-Pro, and the MoE swizzle is exotic but not deep. What the community spends 26 days on is the search: trying things, benchmarking honestly, abandoning bad branches without sunk-cost bias. Machines are better at exactly that, around the clock, in parallel.
We also think the dead ends argue for something we keep repeating: verification is the product. The 12% we refused at hour 14 is the entire difference between an optimizer you can put in production and a benchmark-chasing toy.
Appendix A — Run configuration
- Hardware: 1 node, 8× AMD MI355X (288 GB HBM3E), ROCm 7.2
- Model: DeepSeek V4 Flash, FP4 weights (MXFP4), FP8 KV cache
- Workload: 8K input / 1K output, interactivity floor 8 tok/s/user
- Harness budget: 36 h wall clock, 4,100 candidates evaluated, 61 hypotheses
- Verification: per-layer FP32 reference diff, tolerance budget 5e-3, 61/61 layers
Appendix B — Sources
- InferenceX — MI355X DeepSeek-V4-Pro: 110.5× in 26 days
- InferenceX — DeepSeek V4 1.6T, Day 0 to Day 43
- Hoid companion repo — configs, traces, run log
— Momcilo & Vladimir
Notes from the gap between models and hardware. If you want your model at its peak by Monday, book a call or write to contact@runhoid.com.