[KV Events] Switch event structs from array to map encoding

[sagearc]

Context

KV cache events are published over ZMQ as msgpack-encoded batches consumed by external subscribers (routing daemons, cache managers, etc.). The encoding used a positional array format — every field identified purely by its index — which makes every schema addition a potential break for subscribers that haven't been updated at the same time. This was first flagged by @njhill in #27577, and the schema has been extended four times since:

• block_hashes, parent_block_hash, token_ids, block_size, lora_id — #16750
• medium — #19737
• lora_name — #27577
• extra_keys — #33304
• group_idx — #37688
• kv_cache_spec_kind, kv_cache_spec_sliding_window — #40984

This PR removes array_like=True from the event structs, switching to named-key (map) encoding.

Example

Before — the entire message is a positional array. Inserting a field anywhere but the end shifts every subsequent position and breaks deserialization at an unrelated field.

[
  1.0,                             // ts
  [
    [
      "BlockStored",               // tag      (pos 0)
      [4291203, 1837291, 9104857], // block_hashes   (pos 1)
      null,                        // parent_block_hash (pos 2)
      [0, 1, 2, 3],                // token_ids  (pos 3)
      64,                          // block_size (pos 4)
      null,                        // lora_id    (pos 5)
      "GPU",                       // medium     (pos 6)
      null,                        // lora_name  (pos 7)
      [["mm_hash1", 50]]           // extra_keys (pos 8)
    ]
  ]
]

After — fields are matched by name. A subscriber that doesn't know about extra_keys ignores it. An older subscriber still decodes correctly as long as the fields it knows about are present.

{
  "ts": 1.0,
  "events": [
    {
      "type": "BlockStored",
      "block_hashes": [4291203, 1837291, 9104857],
      "parent_block_hash": null,
      "token_ids": [0, 1, 2, 3],
      "block_size": 64,
      "lora_id": null,
      "medium": "GPU",
      "lora_name": null,
      "extra_keys": [["mm_hash1", 50]]
    }
  ]
}

This is a one-time breaking change. Existing subscribers must drop array_like=True from their local struct definitions (the updated kv_events_subscriber.py example reflects this). After this, the schema can grow without further breaks.

Performance

Measured on 2048-tokens per request, 32-block event with 3 multimodal features (block_size=64):

	Array	Map	Delta
Payload size	6196 B	6294 B	+1.6%
Encode (200×)	1.4 ms	1.4 ms	within noise
Decode (200×)	5.4 ms	5.5 ms	within noise
ZMQ throughput	~25k msg/s	~27k msg/s	within noise

The overhead is a fixed ~98 bytes per message — immeasurable at realistic KV event rates.

CC

@njhill @NickLucche @vMaroon @orozery

[mergify]

Documentation preview: https://vllm--42892.org.readthedocs.build/en/42892/

[gemini-code-assist]

Code Review

This pull request removes the "array_like=True" parameter from the msgspec.Struct definitions for EventBatch and KVCacheEvent in both the core implementation and the example subscriber. I have no feedback to provide as there were no review comments.

[vMaroon]

Thanks @sagearc - I think backwards compatibility will have to be implemented either here or inside consumers. I think the first makes more sense, following standard deprecation protocols.

Looping @PeaBrane for Dynamo input on the change and the above point.

[PeaBrane]

Thanks for pushing this. Moving BlockStored / BlockRemoved off array_like=True makes sense; those structs keep growing and positional fields are a fragile ABI for external consumers.

One question: do we also need to change EventBatch in this PR? The batch envelope seems much more stable (ts, events, data_parallel_rank), and changing it means consumers need to handle a new outer shape too:

[ts, events, data_parallel_rank]

to:

{"ts": ts, "events": events, "data_parallel_rank": data_parallel_rank}

On the Dynamo side, we already tolerate array/map forms for individual events, but would still need a top-level batch parser update if EventBatch changes. If there’s a strong reason to switch the batch level too, we’re open to patching Dynamo to support both formats during the transition.

[sagearc]

I think backwards compatibility will have to be implemented either here or inside consumers. I think the first makes more sense, following standard deprecation protocols.

@vMaroon We could add a flag to keep the old format during a transition. Worth noting the schema has grown 5 times without compat protocol so far, and the consumer update is one line. Happy to go either way, just want to flag the tradeoff of keeping two formats alive.

One question: do we also need to change EventBatch in this PR? The batch envelope seems much more stable (ts, events, data_parallel_rank), and changing it means consumers need to handle a new outer shape too

@PeaBrane You're right, scoped it out — EventBatch reverted to array_like=True. Thanks!

[NickLucche]

Thanks for following up with this @sagearc .
I would personally list this as a breaking change and just change the KVCacheEvent format. Clients/consumers would have to handle backward compatibility if aiming to support multiple vllm versions. From this change onward, retro-compat is ensured by this format.
Otherwise with current code we're anyway breaking compatibility on each new addition.

Do you have a different opinion on this @njhill @markmc ?

[NickLucche]

LGTM but would appreciate another stamp here