[Perf] fuse qk rmsnorm rope gate for qwen3.5

[ZJY0516]

Purpose

combine split + QK-RMSNorm + partial RoPE + gate copy into one kernel launch
Ref: lightseekorg/tokenspeed#228

Test Plan

vllm bench throughput \
 --model Qwen/Qwen3.5-27B \
 --dataset-name random --language-model-only

Test Result

main

Throughput: 14.35 requests/s, 16535.51 total tokens/s, 1837.28 output tokens/s

PR

Throughput: 14.65 requests/s, 16877.84 total tokens/s, 1875.32 output tokens/s

Accuracy

main

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.6702	±	0.0129
		strict-match	5	exact_match	↑	0.6710	±	0.0129

PR

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.6619	±	0.013
		strict-match	5	exact_match	↑	0.6603	±	0.013

GPQA

main: 0.8485

PR: 0.8485

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 410c9f80da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[dllehr-amd]

This looks like one of the first PRs to tackle the new RFC for putting fusions directly in model files. Given the size of the kernel, and how it affects the overall LoC's in the qwen_next.py. Would it be worth having a fusion, or kernel file that can hold these instances?

I'm looking at this from a ROCm/Aiter perspective, where we have "aiter_ops" to track a lot of these kernels, and keep the model file focused more on the overall flow.

What are people's thoughts here @SageMoore @robertgshaw2-redhat @WoosukKwon ?

[vadiklyutiy]

LGTM
But only one thing. I'd propose to move triton kernel and its python interface to another file

[ZJY0516]

LGTM But only one thing. I'd propose to move triton kernel and its python interface to another file

ok, I'll refactor it. cc @dllehr-amd

[mergify]

Hi @ZJY0516, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.