[Kernel] Fuse Q/K RMSNorm and RoPE by bobboli · Pull Request #5990 · vllm-project/vllm-omni

bobboli · 2026-08-10T10:30:09Z

Summary

add a shared Q/K RMSNorm + packed RoPE custom-op boundary
fuse normalization and rotation into one Triton pass per Q/K tensor
avoid materializing normalized Q/K and rotary-product intermediates
retain an eager fallback for unsupported devices, dtypes, and geometries

The CUDA fast path is integrated into the diffusion attention path used by MiniMax H3 and currently specializes its BF16 head_dim=128, rotary_dim=96 geometry. The public layer API and fallback are model-independent.

Performance

Minimax H3 on 4x NVIDIA B300 SXM6 with TP1, Ulysses4, Ring1, dense BF16 TRTLLM_ATTN, 1344x768 output, 243 frames, 50 configured inference steps, and seed 0, steady denoising latency decreases from 109.687 s to 106.205 s (-3.18%, 1.033x). Each revision used one warmup followed by one measured request.

Nsight Systems

Unfused:

Fused:

Testing

pytest -c /dev/null -q tests/diffusion/layers/test_fused_qk_norm_rope.py (3 passed)
ruff check on changed Python files
ruff format --check on changed Python files
git diff --check

Related to #5700.

Expose a shared Q/K custom-op boundary while keeping normalization and rotation in single-pass Triton kernels. Avoid normalized Q/K and rotary-product intermediates. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>

chatgpt-codex-connector · 2026-08-10T10:43:13Z

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

vllm-omni-review-bot · 2026-08-10T10:44:56Z

This PR appears to belong to: docs/design/module/diffusion/diffusion_model_integration.md.

Module owners: @Isotr0py @wtomin @ZJY0516

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

lishunyang12 · 2026-08-11T06:20:03Z

Can we show metrics such as SSIM or PSNR to illustrate video precision is within acceptable range.

bobboli · 2026-08-11T10:14:08Z

The unit test in tests/diffusion/layers/test_fused_qk_norm_rope.py compares the fused operator against the BF16 reference. It reports a maximum absolute error of 0.0625 and a mean absolute error of 0.00072–0.00077 across the tested sequence lengths (1, 257, and 1024). The end-to-end generated video is nearly identical to the baseline.

Signed-off-by: lishunyang12 <lishunyang12@163.com>

…oject#6129) Signed-off-by: lishunyang12 <lishunyang12@163.com>

Signed-off-by: lishunyang12 <lishunyang12@163.com>

A CUDA-event profile of the float16 generator, taken after the projections were packed, showed the bottleneck had moved off the GEMMs: RoPE 14.9% and per-head QK norm 13.5% of the forward, with the GQA repeat_interleave at 4.5% and the transposes into SDPA's layout at 2.7% behind them. All of it is cheap arithmetic over the whole activation, so what it really costs is round-trips through memory -- Q and K are written by the norm, read back by RoPE, written again, and K and V are then read and rewritten whole by the broadcast and the transpose. fused_qkv_rope.py does the entire prologue in a single Triton pass: split the packed QKV, RMSNorm Q and K per head, rotate both, broadcast K and V across their query groups, and write all three straight into SDPA's [batch, heads, positions, head_dim] layout. Each program owns one tile of heads of one token, reads that tile once, and stores the finished values from registers. RoPE's rotate-half partner is re-read from the same row instead of being materialized as a concatenation, and the RMS is a per-head scalar so the partner needs no second reduction -- the same technique vllm_omni/diffusion/layers/fused_qk_norm_rope.py (vllm-project#5990) uses for the norm and rotation alone. Eight launches per layer become one. Nothing in the op is OmniVoice-specific beyond its calling convention, but OmniVoice is its only consumer today: every other model here with a packed QKV projection runs on vLLM's attention stack, whose backends already handle GQA and RoPE, so none of them performs the sequence this replaces. It therefore sits beside its consumer rather than in models/common/, which is also where vllm-project#5990 put an op its own description calls model-independent. Lift it out when a second model wants it. An eager reference covers non-CUDA tensors, absent Triton, and geometries the tiling cannot express; float32, float16 and bfloat16 are all tested against it. The same commit threads the residual across block boundaries. The block used to end with a bare `residual + hidden_states` and the next block began with a standalone input RMSNorm; those are the same read of the same tensor, so both now happen inside the fused add+RMSNorm kernel this file already had. Only the first block, which has no pending residual, still pays for a standalone norm. Measured on one A800 (torch 2.13.0+cu130), 5 runs after 2 warmups, same seed and texts, mean +/- stddev, float16, for 1.44 / 4.16 / 14.60 s of generated audio: projections packed only 153.8 / 196.4 / 388.4 ms this commit 96.4 / 130.3 / 285.5 ms 1.60x / 1.51x / 1.36x Against unmodified main in float32 that is 6.95x / 6.99x / 7.30x end to end, and RTF on the 4.16 s utterance goes 4.57x to 31.93x. The float32 audio stays bit-identical to unmodified main at the same seed. Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>

The generator's 28 layers run 32 unmasking steps per request, and a CUDA profile of one 4.16 s utterance showed 66% of the forward sitting in the projections: 22.5% in q_proj/k_proj/v_proj and 43.9% in the MLP. Those are sibling GEMMs over the same activation at 2x44 rows, where three narrow matmuls cost meaningfully more than one wide one. Pack q/k/v into a single qkv_proj and gate/up into a single gate_up_proj, and let the SwiGLU kernel read both halves straight out of the packed activation -- splitting it first would hand the kernel two strided views and cost a full copy of each half, per layer, per step. The checkpoint still stores the five tensors separately, so load_weights packs them explicitly. That step cannot be left to the generic name-based loader: q_proj and gate_proj no longer exist as modules, so it would find no destination, log a warning, and leave the fused parameters at their random initialization -- a model that loads "successfully" and emits noise. A missing, mis-shaped, or partial set of shards now raises. Measured on one A800 (torch 2.13.0+cu130), 5 runs after 2 warmups, same seed and texts throughout, mean +/- stddev: float32 1.44 s audio 668.5 -> 625.9 ms 1.07x 4.16 s audio 903.7 -> 841.0 ms 1.07x 14.60 s audio 2074.4 -> 2048.3 ms 1.01x With the same seed the fused build reproduces unfused main's audio bit-for-bit (max abs diff 0.0 over 99840 samples), so the packing is numerically exact and not just close. Also records a measured negative result at the SDPA call: enable_gqa would remove the repeat_interleave copy, but on torch 2.13 no fused kernel accepts mismatched head counts, so it drops to the math backend and runs 2.6-5.0x slower. The copy stays. Signed-off-by: MrlixiangWE <mrdanaer@gmail.com> A CUDA-event profile of the float16 generator, taken after the projections were packed, showed the bottleneck had moved off the GEMMs: RoPE 14.9% and per-head QK norm 13.5% of the forward, with the GQA repeat_interleave at 4.5% and the transposes into SDPA's layout at 2.7% behind them. All of it is cheap arithmetic over the whole activation, so what it really costs is round-trips through memory -- Q and K are written by the norm, read back by RoPE, written again, and K and V are then read and rewritten whole by the broadcast and the transpose. fused_qkv_rope.py does the entire prologue in a single Triton pass: split the packed QKV, RMSNorm Q and K per head, rotate both, broadcast K and V across their query groups, and write all three straight into SDPA's [batch, heads, positions, head_dim] layout. Each program owns one tile of heads of one token, reads that tile once, and stores the finished values from registers. RoPE's rotate-half partner is re-read from the same row instead of being materialized as a concatenation, and the RMS is a per-head scalar so the partner needs no second reduction -- the same technique vllm_omni/diffusion/layers/fused_qk_norm_rope.py (vllm-project#5990) uses for the norm and rotation alone. Eight launches per layer become one. Nothing in the op is OmniVoice-specific beyond its calling convention, but OmniVoice is its only consumer today: every other model here with a packed QKV projection runs on vLLM's attention stack, whose backends already handle GQA and RoPE, so none of them performs the sequence this replaces. It therefore sits beside its consumer rather than in models/common/, which is also where vllm-project#5990 put an op its own description calls model-independent. Lift it out when a second model wants it. An eager reference covers non-CUDA tensors, absent Triton, and geometries the tiling cannot express; float32, float16 and bfloat16 are all tested against it. The same commit threads the residual across block boundaries. The block used to end with a bare `residual + hidden_states` and the next block began with a standalone input RMSNorm; those are the same read of the same tensor, so both now happen inside the fused add+RMSNorm kernel this file already had. Only the first block, which has no pending residual, still pays for a standalone norm. Measured on one A800 (torch 2.13.0+cu130), 5 runs after 2 warmups, same seed and texts, mean +/- stddev, float16, for 1.44 / 4.16 / 14.60 s of generated audio: projections packed only 153.8 / 196.4 / 388.4 ms this commit 96.4 / 130.3 / 285.5 ms 1.60x / 1.51x / 1.36x Against unmodified main in float32 that is 6.95x / 6.99x / 7.30x end to end, and RTF on the 4.16 s utterance goes 4.57x to 31.93x. The float32 audio stays bit-identical to unmodified main at the same seed. Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>

A CUDA-event profile of one 4.16 s utterance put 66% of the generator's forward in the projections: 22.5% in q_proj/k_proj/v_proj and 43.9% in the MLP. Those are sibling GEMMs over the same activation at 2x44 rows, where three narrow matmuls cost more than one wide one. q/k/v become a single qkv_proj and gate/up a single gate_up_proj, and the SwiGLU kernel reads both halves out of the packed activation instead of a split -- splitting first would hand it two strided views and cost a full copy of each half, per layer, per step. The checkpoint still stores those five tensors separately, so load_weights packs them explicitly and raises on a missing, wrong-shaped or partial shard set. It cannot be left to the generic name-based loader: q_proj and gate_proj no longer exist as modules, so it would find no destination, log a warning, and leave the fused parameters at their random initialization -- a model that loads "successfully" and emits noise. Re-profiling in float16 after that showed the bottleneck had moved off the GEMMs: RoPE 14.9% and per-head QK norm 13.5% of the forward, with the GQA repeat_interleave at 4.5% and the transposes into SDPA's layout at 2.7% behind them. All of it is cheap arithmetic over the whole activation, so what it costs is round-trips through memory. fused_qkv_rope.py does that entire prologue in one Triton pass: split the packed QKV, RMSNorm Q and K per head, rotate both, broadcast K and V across their query groups, and write all three straight into SDPA's [batch, heads, positions, head_dim] layout. Each program owns one tile of heads of one token, reads it once, and stores the finished values from registers; RoPE's rotate-half partner is re-read from the same row instead of being materialized as a concatenation, and the RMS is a per-head scalar so the partner needs no second reduction. Eight launches per layer become one. vllm_omni/diffusion/layers/fused_qk_norm_rope.py (vllm-project#5990) uses the same technique for the norm and rotation alone; the single launch, the V path, the broadcast and the layout are what this adds. The op sits beside OmniVoice rather than in models/common/ because it has one consumer: every other model here with a packed QKV runs on vLLM's attention stack, whose backends already handle GQA and RoPE, so none of them performs the sequence this replaces. Worth lifting out when a second model wants it. An eager reference covers non-CUDA tensors, absent Triton, and geometries the tiling cannot express; float32, float16 and bfloat16 are all tested against it. Last, the residual is threaded across block boundaries. The block used to end with a bare `residual + hidden_states` and the next block began with a standalone input RMSNorm; those are the same read of the same tensor, so both now happen inside the fused add+RMSNorm kernel this file already had. Measured on one A800 (torch 2.13.0+cu130), 5 runs after 2 warmups, same seed and texts, every stage checked out from its own commit, float16, for 1.44 / 4.16 / 14.60 s of generated audio: dtype fix only 159.7 / 201.9 / 397.8 ms + packed projections 154.0 / 196.9 / 385.7 ms 1.04x / 1.03x / 1.03x + fused prologue 98.9 / 132.9 / 288.5 ms 1.56x / 1.48x / 1.34x + residual threading 97.3 / 130.8 / 287.1 ms 1.02x / 1.02x / 1.00x Against unmodified main in float32 (670.1 / 905.8 / 2074.8 ms) that is 6.89x / 6.93x / 7.23x end to end, and RTF on the 4.16 s utterance goes 4.59x to 31.8x. In float32 the fusions alone are 1.17x / 1.16x / 1.07x, and the generated audio there stays bit-identical to unmodified main at the same seed. Also records a measured negative result at the SDPA call: enable_gqa would remove the repeat_interleave copy, but on torch 2.13 no fused kernel accepts mismatched head counts, so it drops to the math backend and runs 2.6-5.0x slower. The copy stays, folded into the kernel's store. Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>

A CUDA-event profile of one 4.16 s utterance put 66% of the generator's forward in the projections: 22.5% in q_proj/k_proj/v_proj and 43.9% in the MLP. Those are sibling GEMMs over the same activation at 2x44 rows, where three narrow matmuls cost more than one wide one. q/k/v become a single qkv_proj and gate/up a single gate_up_proj, and the SwiGLU kernel reads both halves out of the packed activation instead of a split -- splitting first would hand it two strided views and cost a full copy of each half, per layer, per step. The checkpoint still stores those five tensors separately, so load_weights packs them explicitly and raises on a missing, wrong-shaped or partial shard set. It cannot be left to the generic name-based loader: q_proj and gate_proj no longer exist as modules, so it would find no destination, log a warning, and leave the fused parameters at their random initialization -- a model that loads "successfully" and emits noise. Re-profiling in float16 after that showed the bottleneck had moved off the GEMMs: RoPE 14.9% and per-head QK norm 13.5% of the forward, with the GQA repeat_interleave at 4.5% and the transposes into SDPA's layout at 2.7% behind them. All of it is cheap arithmetic over the whole activation, so what it costs is round-trips through memory. fused_qkv_rope.py does that entire prologue in one Triton pass: split the packed QKV, RMSNorm Q and K per head, rotate both, broadcast K and V across their query groups, and write all three straight into SDPA's [batch, heads, positions, head_dim] layout. Each program owns one tile of heads of one token, reads it once, and stores the finished values from registers; RoPE's rotate-half partner is re-read from the same row instead of being materialized as a concatenation, and the RMS is a per-head scalar so the partner needs no second reduction. Eight launches per layer become one. vllm_omni/diffusion/layers/fused_qk_norm_rope.py (vllm-project#5990) uses the same technique for the norm and rotation alone; the single launch, the V path, the broadcast and the layout are what this adds. The op sits beside OmniVoice rather than in models/common/ because it has one consumer: every other model here with a packed QKV runs on vLLM's attention stack, whose backends already handle GQA and RoPE, so none of them performs the sequence this replaces. Worth lifting out when a second model wants it. An eager reference covers non-CUDA tensors, absent Triton, and geometries the tiling cannot express; float32, float16 and bfloat16 are all tested against it. Last, the residual is threaded across block boundaries. The block used to end with a bare `residual + hidden_states` and the next block began with a standalone input RMSNorm; those are the same read of the same tensor, so both now happen inside the fused add+RMSNorm kernel this file already had. Measured on one A800 (torch 2.13.0+cu130), 5 runs after 2 warmups, same seed and texts, every stage checked out from its own commit, float16, for 1.44 / 4.16 / 14.60 s of generated audio: dtype fix only 159.7 / 201.9 / 397.8 ms + packed projections 154.0 / 196.9 / 385.7 ms 1.04x / 1.03x / 1.03x + fused prologue 98.9 / 132.9 / 288.5 ms 1.56x / 1.48x / 1.34x + residual threading 97.3 / 130.8 / 287.1 ms 1.02x / 1.02x / 1.00x Against unmodified main in float32 (670.1 / 905.8 / 2074.8 ms) that is 6.89x / 6.93x / 7.23x end to end, and RTF on the 4.16 s utterance goes 4.59x to 31.8x. In float32 the fusions alone are 1.17x / 1.16x / 1.07x, and the generated audio there stays bit-identical to unmodified main at the same seed. Also records a measured negative result at the SDPA call: enable_gqa would remove the repeat_interleave copy, but on torch 2.13 no fused kernel accepts mismatched head counts, so it drops to the math backend and runs 2.0-4.9x slower. The copy stays, folded into the kernel's store. Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>

This reverts commit 596c16a.

…k_norm_rope Extend the shared fused_qk_norm_rope op (vllm-project#5990) beyond MiniMax-H3 and wire Boogu-Image's attention front to it: - interleaved=True mode: adjacent-pair rotation (Boogu's apply_rotary_emb semantics), shipped as a combined Q+K single-launch Triton kernel with register pair-exchange (tl.split); 29.4 us for Q[4139,28,120] + K[4139,7,120] vs the 145 us eager chain measured in situ (4.9x), 52% of HBM roof. - General geometry on the CUDA fast path: any even rotary_dim <= head_dim <= 256 (power-of-two lane padding); fp32 rope tables accepted alongside the activation dtype. The MiniMax-H3 path (128/96, half-split) keeps its original kernel and launch config - old-vs-new output verified bit-identical at the H3 geometry, its test unchanged; the general half-split geometry is covered by a new test at head_dim 120. - Boogu wiring: one packed [cos|sin] fp32 table per rotary embedding built once per forward (including the rebuilt per-reference-image batch tuple, so ti2i RefImgRefiner sites fuse too); BooguImageSelfAttention and BooguImageJointAttention call the op in place of norm_q/norm_k + apply_rotary_emb x2 + casts. Kill-switch additional_config {"boogu_fused_qk_norm_rope": false} or VLLM_OMNI_BOOGU_FUSED_QK_NORM_ROPE=0; the eager chain remains the fallback and is bit-identical to the previous code path when disabled. Measured on one H200 at 1024x1024, 28 steps, cfg 4.0, concurrency 1 (dfx protocol, 10 prompts, 4 warmups): latency mean 8.233 s -> 7.949 s (-283.9 ms, -3.45%); fusion OFF vs clean base +12.3 ms (noise). nsys: 2,912 fused launches replace 10,976 norm/rotate kernels (406 -> 79 ms GPU per request); no new graph breaks (17 = 17). Image parity at 28 steps: SSIM 0.965 / PSNR 35.5 dB vs the eager chain - single-step PSNR is 51.1 dB and the divergence grows with step count, i.e. in-loop trajectory amplification of <=1-ulp rounding differences (the fused norm rounds once in fp32 like F.rms_norm; vLLM's RMSNorm rounds before the weight multiply), consistent with vllm-project#6571's precedent; images visually identical. The ti2i (reference-image) path carries the same fused call but has no end-to-end harness here; its unit coverage is the same operand-ulp bound. Tests: interleaved statistical bound + rotation-only + validation + half-split general-geometry cases in tests/diffusion/layers/test_fused_qk_norm_rope.py; packed-table layout, kill-switch resolution and fused-vs-eager operand-ulp bound in tests/diffusion/models/boogu_image/test_boogu_fused_qk_norm_rope.py. Signed-off-by: Qihan Kang <rollykanggg@gmail.com>

…k_norm_rope Extend the shared fused_qk_norm_rope op (vllm-project#5990) beyond MiniMax-H3 and wire Boogu-Image's attention front to it: - interleaved=True mode: adjacent-pair rotation (Boogu's apply_rotary_emb semantics) as a per-tensor Triton kernel mirroring the existing op form (launched once for Q, once for K), with the pair value exchanged in registers (tl.split/tl.join) instead of a second global load. Kernel time 36.0 us for Q[4139,28,120] + K[4139,7,120] vs the 145 us eager chain measured in situ (4.0x). - General geometry on the CUDA fast path: any even rotary_dim <= head_dim <= 256 (power-of-two lane padding); fp32 rope tables accepted alongside the activation dtype. The MiniMax-H3 path (128/96, half-split) keeps its original kernel and launch config - old-vs-new output verified bit-identical at the H3 geometry, its test unchanged; the general half-split geometry is covered by a new test at head_dim 120. - Boogu wiring: one packed [cos|sin] fp32 table per rotary embedding built once per forward (including the rebuilt per-reference-image batch tuple, so ti2i RefImgRefiner sites fuse too); BooguImageSelfAttention and BooguImageJointAttention call the op in place of norm_q/norm_k + apply_rotary_emb x2 + casts. Kill-switch additional_config {"boogu_fused_qk_norm_rope": false} or VLLM_OMNI_BOOGU_FUSED_QK_NORM_ROPE=0; the eager chain remains the fallback and is bit-identical to the previous code path when disabled. Measured on one H200 at 1024x1024, 28 steps, cfg 4.0, concurrency 1 (dfx protocol, 10 prompts, 4 warmups): latency mean 8.233 s -> 8.059 s (-174.0 ms, -2.11%); fusion OFF vs clean base +12.3 ms (noise). 2 x 2,912 fused launches replace 10,976 norm/rotate kernels; no new graph breaks (17 = 17). Image parity at 28 steps: SSIM 0.965 / PSNR 35.5 dB vs the eager chain - single-step PSNR is 51.1 dB, 50-step 38.6 dB: in-loop trajectory amplification of <=1-ulp rounding differences that peaks near 28 steps and saturates (the fused norm rounds once in fp32 like F.rms_norm; vLLM's RMSNorm rounds before the weight multiply), consistent with vllm-project#6571's precedent; images visually identical. The ti2i (reference-image) path carries the same fused call but has no end-to-end harness here; its unit coverage is the same operand-ulp bound. Tests: interleaved statistical bound + rotation-only + validation + half-split general-geometry cases in tests/diffusion/layers/test_fused_qk_norm_rope.py; packed-table layout, kill-switch resolution and fused-vs-eager operand-ulp bound in tests/diffusion/models/boogu_image/test_boogu_fused_qk_norm_rope.py. Signed-off-by: Qihan Kang <rollykanggg@gmail.com>

Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> Signed-off-by: lishunyang12 <lishunyang12@163.com> Signed-off-by: Mengjie Zhao <zmj0129@gmail.com> Co-authored-by: lishunyang12 <lishunyang12@163.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

perf(diffusion): fuse QK norm and RoPE

761f8fe

Expose a shared Q/K custom-op boundary while keeping normalization and rotation in single-pass Triton kernels. Avoid normalized Q/K and rotary-product intermediates. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>

bobboli marked this pull request as ready for review August 10, 2026 10:43

bobboli requested review from Isotr0py, NickCao, RuixiangMa, SamitHuang, ZJY0516, david6666666, princepride, wtomin, xuechendi and yenuo26 as code owners August 10, 2026 10:43

hsliuustc0106 added Kernel optimization Codes related to optimize kernel execution to improve hardware utilization diffusion codes related to diffusion models labels Aug 11, 2026

lishunyang12 added the ready label to trigger buildkite CI label Aug 11, 2026

david6666666 mentioned this pull request Aug 11, 2026

[Roadmap] MiniMax-H3 follow-up: accuracy/performance CI, quantization, and optimizations #5700

Open

fix(diffusion): sync H3 TeaCache RoPE table

e076580

Signed-off-by: lishunyang12 <lishunyang12@163.com>

lishunyang12 force-pushed the perf/minimax-h3-fused-qk-norm-rope branch from ef55f3b to e076580 Compare August 13, 2026 07:55

lishunyang12 approved these changes Aug 13, 2026

View reviewed changes

lishunyang12 added ready label to trigger buildkite CI diffusion-x2v-test label to trigger buildkite x2video series of diffusion models test in nightly CI and removed ready label to trigger buildkite CI labels Aug 13, 2026

lishunyang12 requested review from Gaohan123, hsliuustc0106 and ywang96 as code owners August 13, 2026 07:58

lishunyang12 force-pushed the perf/minimax-h3-fused-qk-norm-rope branch from 187307f to 0001fdb Compare August 13, 2026 07:59

lishunyang12 enabled auto-merge (squash) August 13, 2026 08:18

[CI/Build] Fix ReadTheDocs build under RTD's seeded pip 23.1 (vllm-pr…

529014e

…oject#6129) Signed-off-by: lishunyang12 <lishunyang12@163.com>

lishunyang12 force-pushed the perf/minimax-h3-fused-qk-norm-rope branch from 0001fdb to 529014e Compare August 13, 2026 08:19

Merge branch 'main' into perf/minimax-h3-fused-qk-norm-rope

99cc9e8

lishunyang12 added ready label to trigger buildkite CI and removed ready label to trigger buildkite CI labels Aug 13, 2026

mo-ke-ke mentioned this pull request Aug 13, 2026

[Model][Performance] Optimize MiniMax-H3 strict Ulysses boundaries #6173

Merged

lishunyang12 added the merge-test label to trigger buildkite merge test CI label Aug 13, 2026

test(diffusion): align LTX output unpacking

0d0d42f

Signed-off-by: lishunyang12 <lishunyang12@163.com>

lishunyang12 added ready label to trigger buildkite CI merge-test label to trigger buildkite merge test CI and removed ready label to trigger buildkite CI merge-test label to trigger buildkite merge test CI labels Aug 13, 2026

lishunyang12 merged commit 596c16a into vllm-project:main Aug 14, 2026
6 of 9 checks passed

MrlixiangWE mentioned this pull request Aug 18, 2026

[Perf][Bugfix][OmniVoice] Restore float16 serving, and fuse the generator hot loop #6317

Merged

brandneway pushed a commit to brandneway/vllm-omni that referenced this pull request Sep 1, 2026

Revert "[Kernel] Fuse Q/K RMSNorm and RoPE (vllm-project#5990)"

a56af07

This reverts commit 596c16a.

Holworth mentioned this pull request Sep 4, 2026

[Kernel][Boogu-Image] Fuse Q/K RMSNorm + interleaved RoPE via fused_qk_norm_rope #6982

Merged

This was referenced Sep 4, 2026

[Survey] Kernel 优化全仓梳理：通用 vs 模型绑定 + 归档方案（2026-09-04） Dong1017/vllm-omni#1

Open

[Diffusion][Perf] Add Qwen-Image QK RoPE Triton path #5931

Merged

[RFC]: Kernel Reuse and Hardware Integration in vLLM-Omni #7382

Open

THUqliu mentioned this pull request Sep 26, 2026

[Doc] Add QK/RoPE contract detail and qualification status to shared ops README bugkeep/vllm-omni#1

Merged

Navigation Menu

[Kernel] Fuse Q/K RMSNorm and RoPE - #5990

[Kernel] Fuse Q/K RMSNorm and RoPE#5990
lishunyang12 merged 5 commits into
vllm-project:mainfrom
bobboli:perf/minimax-h3-fused-qk-norm-rope

bobboli commented Aug 10, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Aug 10, 2026

vllm-omni-review-bot commented Aug 10, 2026

lishunyang12 commented Aug 11, 2026

bobboli commented Aug 11, 2026 •

edited

Loading

Uh oh!

Labels

4 participants

Conversation

bobboli commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance

Nsight Systems

Testing

chatgpt-codex-connector Bot commented Aug 10, 2026

vllm-omni-review-bot commented Aug 10, 2026

lishunyang12 commented Aug 11, 2026

bobboli commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Labels

4 participants

bobboli commented Aug 10, 2026 •

edited

Loading

bobboli commented Aug 11, 2026 •

edited

Loading