Skip to content

[MRV2] Make FP32 Gumbel sampling more accurate#45996

Merged
WoosukKwon merged 4 commits into
mainfrom
woosuk/gumbel-fix
Jun 18, 2026
Merged

[MRV2] Make FP32 Gumbel sampling more accurate#45996
WoosukKwon merged 4 commits into
mainfrom
woosuk/gumbel-fix

Conversation

@WoosukKwon

@WoosukKwon WoosukKwon commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

The current FP32 Gumbel sampling kernel is not accurate enough. This PR improves the accuracy with a simple trick: use log1p(-u) instead of log(u), enabling higher resolution in determining the argmax winner.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
@mergify mergify Bot added the v1 label Jun 18, 2026
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
@WoosukKwon WoosukKwon added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 18, 2026
@mergify

mergify Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Hi @WoosukKwon, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
@WoosukKwon WoosukKwon enabled auto-merge (squash) June 18, 2026 19:16
@WoosukKwon WoosukKwon merged commit 16908e1 into main Jun 18, 2026
82 checks passed
@WoosukKwon WoosukKwon deleted the woosuk/gumbel-fix branch June 18, 2026 19:42
divineearthly pushed a commit to divineearthly/vllm that referenced this pull request Jun 19, 2026
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: divineearthly <divineearthly@gmail.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Jun 21, 2026
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
tunglinwood pushed a commit to tunglinwood/vllm that referenced this pull request Jun 22, 2026
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
nkzhenhua pushed a commit to nkzhenhua/vllm that referenced this pull request Jun 24, 2026
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed v1

2 participants