Conversation
The coordinator passes needs_kv_cache_zeroing to every single-type manager since vllm-project#44490, but SinkFullAttentionManager spelled out its parent's old signature and raised TypeError at startup for models using StaticSinkAttention (openPangu). Forward kwargs like the other managers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Contributor
Author
|
This is not a urgent PR and I don't have a plan to merge it soon. Just wanted to leave it here so people and agents can refer to this if see any issues with SinkFullAttentionManager |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Since #44490 the KV cache coordinator passes
needs_kv_cache_zeroingto every single-type manager, butSinkFullAttentionManagerstill spelled out its parent's old__init__signature. Any model usingSinkFullAttentionSpec(openPangu withStaticSinkAttention) fails at KV cache manager init withTypeError: SinkFullAttentionManager.__init__() got an unexpected keyword argument 'needs_kv_cache_zeroing', whether or not zeroing is enabled.This forwards
**kwargsto the parent, matching the other manager subclasses (RSWA, SWA, chunked-local, Mamba, HiSparse), and adds a regression test that constructs the manager withneeds_kv_cache_zeroing=Trueand checks its new blocks are recorded for zeroing.Not a duplicate: no open PR touches this constructor (searched
SinkFullAttentionManagerandneeds_kv_cache_zeroing; #47271 and #43374 modify nearby sink code but not this).Test Plan
Test Result
The new test fails on main with the
TypeErrorabove and passes with the fix; both files pass (81 passed). No model eval: the change only fixes a startup crash and does not affect outputs of models that already started.AI assistance (Claude Code) was used; I reviewed every changed line and ran the tests above.
🤖 Generated with Claude Code