[Perf][KVConnector][Mooncake] Compact chunk-hash keys and zero-copy lookup wire format

[ivanium]

Purpose

Two related optimizations to the MooncakeStoreConnector prefix-lookup path. Both reduce the cost of moving block hashes around, which becomes acute when the model's block_size is much larger than the connector's hash_block_size.

1. Compact chunk-hash keys

When block_size > hash_block_size, a single block_size chunk previously keyed Mooncake by concatenating all of its fine-grained sub-hashes (BlockHashListWithBlockSize joined every sub-hash). The Mooncake key therefore grew linearly with the block_size / hash_block_size ratio.

This is especially costly for DeepSeek-V4-style configs, which run block_size=256 with hash_block_size=4 — a ratio of 64, so every Mooncake key became 64 hash digests concatenated together.

Because the engine chains block hashes (each block hash folds in its predecessor), a chunk's last sub-hash already uniquely identifies the whole chunk and its prefix. The new chunk_hashes_for_block_size / _CompactChunkHashList key each chunk by that single trailing digest, so the Mooncake key stays one digest regardless of the ratio (64x smaller keys for the DSv4 config above, and proportionally smaller anywhere block_size != hash_block_size).

2. Zero-copy lookup wire format

The lookup RPC previously msgpack-encoded a list[str] of hex digests — hex doubles the byte count and msgpack adds per-element framing. The protocol now sends a hash_len frame (u16) followed by the raw fixed-size hashes concatenated back-to-back. The server reconstructs them through BlobBlockHashes, a lazy Sequence[BlockHash] view over the flat buffer, so it never materializes the full hash list upfront. This removes the hex inflation, the msgpack framing, and the eager allocation on the hot lookup path.

Test Plan

.venv/bin/python -m pytest \
  tests/v1/kv_connector/unit/test_mooncake_store_coordinator.py \
  tests/v1/kv_connector/unit/test_mooncake_store_worker.py \
  tests/v1/kv_connector/unit/test_mooncake_store_hma_e2e.py

Tests are updated to assert the compact last-sub-hash keying and the new wire format.

Test Result

85 passed.

Notes

This is not a duplicate of any open PR. No other open PR touches the chunk-hash key construction or the lookup serialization format; the only adjacent work is my own #45659 (async lookup), which changes the lookup call site but not its payload encoding. This change is rebased to stand alone on main.

AI assistance was used in preparing this change; the author has reviewed every changed line.

🤖 Generated with Claude Code

[ivanium]

cc @wzhao18 and @Dao007forever for comments

[njhill]

nit: use tuple

Suggested change

	all_frames = [
	LOOKUP_MSG,
	token_len.to_bytes(4, byteorder="big"),
	hash_len.to_bytes(2, byteorder="big"),
	b"".join(block_hashes),
	]
	all_frames = (
	LOOKUP_MSG,
	token_len.to_bytes(4, byteorder="big"),
	hash_len.to_bytes(2, byteorder="big"),
	b"".join(block_hashes),
	)

[njhill]

Suggested change

	blob = bytes(all_frames[3])
	blob = all_frames[3].buffer

This will make a copy otherwise.

[njhill]

combined with other suggestion

Suggested change

	def __init__(self, blob: bytes, hash_len: int):
	def __init__(self, blob: memoryview, hash_len: int):

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ivanium.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ivanium]

@njhill Thanks for the comments! I updated the PR with the suggestions.

[njhill]

Thanks @ivanium lgtm

[mergify]

Hi @ivanium, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.