[Kernel][Perf] Tune fused_moe FP8 config for Qwen3-Next-80B tp=4 on H100 (+25% at batch 96-512)

[qyYue1389]

Purpose

Add a tuned Triton fused_moe config for Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 on a single NVIDIA H100 80GB HBM3 with tp=4 and FP8 w8a8 blockwise quantization (block_shape=[128, 128])

Qwen3-Next-80B has num_experts=512, num_experts_per_tok=10, moe_intermediate_size=512. With tp=4 this gives the kernel shape E=512, N=128. The current get_default_config in fused_moe.py is ~25% slower than the tuned configuration for the batch_size range that dominates production serving

The file contains 5 batch-size keys (96, 128, 256, 512, 1024) for which the tuned config beats the default by >=5%. Other batch sizes fall back to get_default_config via the existing vllm fallback path

Test plan

Tuned with benchmark_moe.py --tune on a single H100 80GB HBM3 (precompiled vLLM cu128 + torch 2.11.0+cu128 + triton 3.6.0)

Test Result

Tuning walked 10,368 valid configurations across 18 batch sizes ({1,2,4,8,16,24,32,48,64,96,128,256,512,1024,1536,2048,3072,4096}), total wall time 77 minutes.

After tuning, the file was copied into the configs/ directory and the benchmark re-run for all 18 batch sizes. Per-batch-size median kernel time:

batch_size	baseline (us)	tuned (us)	speedup
1	29.45	29.26	+0.6%
2	30.35	30.13	+0.7%
4	36.38	35.24	+3.2%
8	54.81	54.93	-0.2%
16	80.74	79.29	+1.8%
24	95.90	96.65	-0.8%
32	111.83	109.90	+1.8%
48	140.51	140.38	+0.1%
64	152.18	151.30	+0.6%
96	215.09	171.56	+25.4%
128	229.10	183.06	+25.1%
256	248.41	199.60	+24.5%
512	262.77	217.58	+20.8%
1024	290.12	268.59	+8.0%
1536	318.21	317.86	+0.1%
2048	345.23	344.86	+0.1%
3072	429.06	429.53	-0.1%
4096	568.93	567.50	+0.3%

The JSON ships only the bolded 5 keys (96, 128, 256, 512, 1024). Batch sizes outside this range fall back to get_default_config, which my benchmark shows is within 1% of the tuned config

The wins on the kept keys are primarily driven by GROUP_SIZE_M=64 (vs get_default_config, GROUP_SIZE_M=32), which gives a more L2-cache friendly tile traversal order for E=512, N=128, FP8 workload

The closest H100 PR is #35808 (E=256,N=512 on H100), which is in different shape. Recent tuned-config PRs #44273 / #44152 / #44553 all target H20, not H100. No open or merged PR addresses
E=512,N=128 FP8 blockwise on H100.

Reproduction commands

# Baseline (move the new JSON aside first to force the default config)
mv vllm/model_executor/layers/fused_moe/configs/E=512,N=128,device_name=NVIDIA_H100_80GB_HBM3,dtype=fp8_w8a8,block_shape=[128,128].json /tmp/
.venv/bin/python benchmarks/kernels/benchmark_moe.py \
    --model Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 \
    -tp 4 --dtype fp8_w8a8 \
    --batch-size 1 2 4 8 16 24 32 48 64 96 128 256 512 1024 1536 2048 3072 4096

# Tuned (restore the JSON)
mv /tmp/E=512,N=128,device_name=NVIDIA_H100_80GB_HBM3,dtype=fp8_w8a8,block_shape=[128,128].json \
   vllm/model_executor/layers/fused_moe/configs/
.venv/bin/python benchmarks/kernels/benchmark_moe.py \
    --model Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 \
    -tp 4 --dtype fp8_w8a8 \
    --batch-size 1 2 4 8 16 24 32 48 64 96 128 256 512 1024 1536 2048 3072 4096

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.