[llm] lazy-load batch stage and processor submodules

[kouroshHakha]

Why

ray.llm._internal.batch.stages and ray.llm._internal.batch.processor previously imported every stage / engine submodule eagerly from their __init__.py. Several of those submodules pull in heavy optional dependencies (transformers, vllm, sglang, mistral_common, huggingface_hub, ...).

As a result, even importing a lightweight piece like HttpRequestProcessorConfig — which only needs an aiohttp client and a few pydantic models — was loading the entire ML stack. On a typical machine that meant ~7s of import latency, a multi-hundred-MB process footprint, and a hard ImportError whenever any of those optional deps were not installed.

Closes #52632

Concretely, before this change:

import sys, time
t = time.perf_counter()
import ray.llm._internal.batch.processor.http_request_proc  # noqa
print(f"{time.perf_counter() - t:.2f}s")
print("transformers loaded:", "transformers" in sys.modules)
print("vllm.transformers_utils loaded:", "vllm.transformers_utils" in sys.modules)
print("sglang loaded:", "sglang" in sys.modules)
print("mistral_common loaded:", "mistral_common" in sys.modules)

prints something like:

7.46s
transformers loaded: True
vllm.transformers_utils loaded: True
sglang loaded: True
mistral_common loaded: True

After this change the same script prints ~1.1s and all four flags are False.

What

Convert both __init__.py files to PEP 562 __getattr__ lazy re-exports:

• Cheap symbols (StatefulStage, Processor, ProcessorBuilder, ProcessorConfig, the various *StageConfig classes) stay eagerly imported because they have no heavy transitive deps and are used everywhere.
• Engine-specific stage / processor classes are listed in a small _LAZY_ATTRS map and resolved on first attribute access via __getattr__. The result is then cached in globals() so subsequent lookups are free.
• __dir__ is overridden to keep tab-completion / introspection working.
• A TYPE_CHECKING block re-exports the lazy names statically so type checkers / IDEs continue to see them.

Side-effect note

Each *_proc.py calls ProcessorBuilder.register(...) at import time. With this change the registration happens the first time the corresponding config is accessed via the package, which is exactly when a user constructs that config and then calls ProcessorBuilder.build, so the registry is populated in time for every realistic use. This is exercised in the tests — see test_lazy_imports.py and the existing ProcessorBuilder.build(HttpRequestProcessorConfig(...)) path.

Test plan

• New regression test file python/ray/llm/tests/batch/cpu/processor/test_lazy_imports.py (17 cases, all green) pinning the new behavior:
  • importing HttpRequestProcessorConfig must not pull transformers, vllm, sglang, mistral_common, tokenizers, huggingface_hub or any non-HTTP stage / processor submodule into sys.modules (verified in a clean subprocess).
  • importing HttpRequestStage from the stages package must only load http_request_stage.py and not any other stage submodule.
  • every lazy attr resolves to the right class from the right submodule.
  • unknown attributes raise AttributeError (so hasattr etc. behave correctly).
  • dir(pkg) lists all lazy attrs.
• Pre-existing python/ray/llm/tests/batch/cpu/processor/ and python/ray/llm/tests/batch/cpu/stages/ tests pass with this change identically to baseline (73 passed, same 13 pre-existing env-skew failures unrelated to this change).
• Sanity-checked the public API end-to-end: from ray.llm._internal.batch import HttpRequestProcessorConfig, then ProcessorBuilder.build(cfg) builds an HTTP processor with the expected stages.
• pre-commit passes on all changed files (black, ruff, pydoclint, import order, etc.).

Made with Cursor

[gemini-code-assist]

Code Review

This pull request implements lazy loading for batch processors and stages using PEP 562 getattr to avoid unnecessary loading of heavy ML dependencies. It also includes regression tests to ensure that importing lightweight components does not trigger heavy imports and that attribute resolution works correctly. I have no feedback to provide.

[jeffreywang88]

This is quite similar to stages/_init_.py's _get_attr_. Consider refactoring to a shared utility.