[ZenCPU] Add zencpu Platform Runtime Logging and Docs

[amd-lalithnc]

Change-Id: I6049a7950a6c312c02892a740576411e886271a7

Purpose

Add AMD Zen CPU runtime visibility to the public fork so it is easy to confirm, from live logs, that:

• ZenCpuPlatform was activated
• CPU unquantized GEMM dispatch went through the zentorch path

This PR also updates unit tests - to verify the ZenCPU platform verification logs - and installation/verification docs for the Zen CPU flow.

Test Plan

1. Run the focused unit tests for the new logging coverage:

   python -m pytest tests/model_executor/test_cpu_unquantized_gemm_dispatch.py -q

2. Install the public fork in the validation env and run a real CPU throughput benchmark:
   python -m pip install . --no-build-isolation --no-deps --force-reinstall

env -i HOME="$HOME" PATH="$PATH" TERM="${TERM:-xterm}" USER="${USER}" bash --noprofile --norc -lc '
source /proj/zendnn/lalithnc/anaconda3/etc/profile.d/conda.sh &&
conda activate vllm-zentorch-0.20.2 &&
cd /tmp &&
export VLLM_CPU_KVCACHE_SPACE=40 &&
export VLLM_CPU_OMP_THREADS_BIND="0-55" &&
vllm bench throughput \
  --model /proj/ai_models/vllm/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659 \
  --random-input-len 128 \
  --random-output-len 128 \
  --num-prompts 8 \
  --max-num-seqs 8

Verify the runtime output contains:

• ZenCpuPlatform activated
• CPU unquantized GEMM dispatch: using zentorch_linear_unary

Test Result

• Focused unit tests passed:

4 passed, 17 warnings in 1.91s

• Real benchmark on the public fork emitted the expected runtime logs:

ZenCpuPlatform activated | zentorch=5.2.1 | VLLM_ZENTORCH_WEIGHT_PREPACK=1 | AVX-512=True | AVX-512_BF16=True
CPU unquantized GEMM dispatch: using zentorch_linear_unary (prepacked=True)

• Benchmark completed successfully with:

Throughput: 0.65 requests/s, 167.36 total tokens/s, 83.68 output tokens/s
Total num prompt tokens:  1024
Total num output tokens:  1024
exit_code: 0

Docs were updated to reflect Zen CPU install/verification flow in the public fork.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

cc: @tlrmchlsmth @ProExpertProg

[mergify]

Documentation preview: https://vllm--42726.org.readthedocs.build/en/42726/

[gemini-code-assist]

Code Review

This pull request introduces documentation and logging for AMD Zen CPU optimizations. It adds detailed installation instructions for the zentorch plugin, explains the auto-activation of ZenCpuPlatform on AMD Zen 4/5 hardware, and implements informative logging to identify which GEMM dispatch kernel is being utilized. Unit tests were also included to verify the new logging and activation logic. I have no feedback to provide.

[mergify]

Hi @amd-lalithnc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[amd-lalithnc]

hello @tlrmchlsmth @ProExpertProg - can we enable CI checks on this PR? please do give this a review when you get a chance, these logs help us identify if the relevant platform and kernels are being selected. Thanks!

[amukho]

hi @AndreasKaratzas can you please review and help merge this PR? This is adding the logging and documentation support for the ZenCPU platform plugin already upstreamed in vLLM

[AndreasKaratzas]

I would put this above the "How to decide VLLM_CPU_OMP_THREADS_BIND"

[amd-lalithnc]

done

[AndreasKaratzas]

No reason for line diff here.

[amd-lalithnc]

done

[AndreasKaratzas]

No reason for new lines here.

[amd-lalithnc]

done

[AndreasKaratzas]

Again check this file, no need for some of the new lines.

[amd-lalithnc]

done

[AndreasKaratzas]

Should this be skipped if platform is not zen_cpu?

[amd-lalithnc]

No, this is intentional. The test is a unit test for the dispatcher logic.

It uses the _mock_zentorch_linear_unary fixture to register a fake zentorch_linear_unary op via torch.library and monkeypatches current_platform.is_zen_cpu to True. This makes the test hardware-independent by design so the dispatch path is exercised on any CI runner.

Skipping on non-Zen hardware would mean the new logging code is never validated in CI. The pattern matches the two existing tests above it (test_dispatch_cpu_unquantized_gemm_uses_zentorch_on_zen, test_dispatch_cpu_unquantized_gemm_zen_remove_weight) that were accepted in PR #35970, and the four test_zen_cpu_platform_detection.py tests (also from #35970) that mock os.path.exists + open to make /proc/cpuinfo reads hardware-independent. Verified passing locally: 4/4 in this file + 4/4 in test_zen_cpu_platform_detection.py = 8/8 PASSED.

[amd-lalithnc]

although, would love to get your thoughts on adding UT for logging

[AndreasKaratzas]

same

[amd-lalithnc]

same answer as the previous one

[AndreasKaratzas]

The first log is borderline OK since it happens on the zentorch path alone, but the rest of them I am not so sure. Can you explore the option of installing logs in your dispatcher? (platforms/zen_cpu.py)

Or these are important logs?

[amd-lalithnc]

Dropped the three non-Zen logger.info_once calls in dispatch_cpu_unquantized_gemm (sgl-kernel, oneDNN, fallback)

[tlrmchlsmth]

check_and_update_config is not the right place to print info banners. We already have logging that the zen platform is being used here - perhaps upgrade that logging level to info instead?

vllm/vllm/platforms/__init__.py

Lines 190 to 192 in 02606b0

	logger.debug(
	"AMD Zen CPU detected with zentorch installed, using ZenCpuPlatform."
	)

[tlrmchlsmth]

Could you change this to debug_once and then add a log for all of the backends, not just zentorch? Thanks

[AndreasKaratzas]

Gonna add the ready label, comments have been addressed
cc @tlrmchlsmth for final stamping

[mergify]

Hi @amd-lalithnc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

Hi @amd-lalithnc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[amd-lalithnc]

hi @tlrmchlsmth @AndreasKaratzas - CI failing due to a 503 error, can we retrigger?

[tlrmchlsmth]

A couple comments in the readme that I didn't catch before

[tlrmchlsmth]

Let's update the model to something more modern. (Qwen3 0.6b? In that case you won't have to set --dtype bf16) We also shouldn't need to set VLLM_LOGGING_LEVEL=INFO since this is the default

[amd-lalithnc]

changed the model

[amd-lalithnc]

hi @tlrmchlsmth - can we go ahead and merge this PR - all review comments have been addressed, thanks!

[tlrmchlsmth]

accident

[amd-lalithnc]

hi @tlrmchlsmth - looks like the CI failures are unrelated to our PR - please check and let me know if there are any concerns