[llm][serve] Materialize chat completion message content in sanitizer

[kouroshHakha]

Summary

_sanitize_chat_completion_request (in ray/llm/_internal/serve/core/ingress/ingress.py) only materialized tool_calls on assistant messages, leaving content as a Pydantic ValidatorIterator whenever the request matched the Iterable[ContentPart] arm of the Union[str, Iterable[...], None] field type that OpenAI's TypedDicts use on every message variant. Cloudpickle then failed when the ingress forwarded the request to the LLM replica via model_handle.chat.remote(...), returning 500s for any payload that sends content as a list of content parts (e.g. [{"text": "...", "type": "text"}]).

Plain-string content requests were unaffected, which is why this slipped through earlier.

This PR extends the sanitizer to also materialize content to a list on every message variant (system, user, assistant, tool). The existing tool_calls handling is unchanged.

It also drops the optimistic TODO(seiji): Remove when we update to Pydantic v2.11+ — the Iterable[...] Union-arm bug is still reproducible on Pydantic 2.12, so this workaround is required regardless of Pydantic version.

Repro (before)

from ray.llm._internal.serve.core.configs.openai_api_models import ChatCompletionRequest
import pickle

body = ChatCompletionRequest.model_validate({
    "model": "qwen35-9b",
    "messages": [
        {"role": "user", "content": [{"text": "hi", "type": "text"}]},
        {"role": "assistant", "content": [{"text": "ok", "type": "text"}]},
    ],
})
pickle.dumps(body)
# TypeError: cannot pickle 'pydantic_core._pydantic_core.ValidatorIterator' object

After this PR (running through _sanitize_chat_completion_request), the request pickles successfully.

Related

• vLLM PR: [BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used vllm-project/vllm#9951
• Pydantic Issue: Accessing a TypedDict field has ValidationIterator instead of the original value pydantic/pydantic#9467
• Related Issue: Attributes declared as iterables are replaced in the instances by pydantic-core ValidatorIterator instance pydantic/pydantic#9541
• Original Ray fix (assistant tool_calls only): [serve.llm] Serialize ChatCompletionRequest tool_calls ValidatorIterator object #55538 / [llm] fails to serialize assistant message with tool_calls #55294

Test plan

• New unit test test_serializes_content_iterator exercises a payload with list-of-content-parts on every role and asserts the result pickles.
• Existing tests test_serializes_tool_calls_iterator and test_handles_no_tool_calls still pass.
• bash ci/lint/lint.sh pre_commit clean.
• CI green.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request updates the _sanitize_chat_completion_request function to address a Pydantic serialization bug where content fields in chat messages are stored as non-picklable ValidatorIterator objects. The changes ensure these fields are materialized into lists across all message roles, and a new test case has been added to verify picklability. Feedback suggests a more idiomatic approach to updating the message dictionary by utilizing the existing reference.