[Bugfix][CPU] Honor cgroup memory limit when computing KV cache size

[maobaolong]

Purpose

When vLLM runs inside a container (Docker / Kubernetes pod) with a memory
cgroup limit, the CPU worker still reads NUMA MemTotal from
/sys/devices/system/node/node*/meminfo, which reports host-wide
numbers and ignores the cgroup limit. As a result, gpu_memory_utilization
is applied to the host's total RAM rather than the container's quota,
which leads to KV-cache allocation failures such as:

ValueError: Available memory on node 0 (149.11/692.65 GiB) on kv cache allocation is less than requested memory for kv (206.54/207.8 GiB). Reduce CPU memory used by other processes.

Here 692.65 GiB is the host total, but the pod is only granted ~150 GiB.

This PR makes get_memory_node_info() aware of the cgroup memory limit
(both v2 unified /sys/fs/cgroup/memory.max and v1
/sys/fs/cgroup/memory/memory.limit_in_bytes), and clamps the reported
total_memory / available_memory to the cgroup quota when it is
smaller than the NUMA node total. Bare-metal / unconstrained deployments
are unaffected (the function short-circuits when no limit is found or the
v1 sentinel value indicates "unlimited").

Test Plan

Verify that get_memory_node_info() honors the cgroup memory limit
when running inside a constrained container (the original bug):

1. Bare-metal / no cgroup limit → behavior unchanged (NUMA numbers
   returned as-is).
2. cgroup v1 container with a small memory.limit_in_bytes → reported
   total_memory / available_memory are clamped to the cgroup
   limit, not the host NUMA total.
3. cgroup v2 container with memory.max → same clamping behavior via
   the v2 path.
4. cgroup v1 with the "unlimited" sentinel (>= 2^62) → treated as no
   limit, NUMA numbers preserved.

Test Result

Reproduced inside a real cgroup-v1 Docker container (host has 30.83
GiB on node0, container limited to 4 GiB):

cgroup limit = 4294967296 (4.00 GiB)
cgroup usage = 8429568 (0.01 GiB)
NUMA node0 total = 30.83 GiB
NUMA node0 available = 28.44 GiB 
---- after cgroup clamp ---- 
total = 4.00 GiB available = 3.99 GiB

Without this patch get_memory_node_info() would return the host
NUMA values (30.83 / 28.44 GiB), causing
requested_cpu_memory = host_total * gpu_memory_utilization to vastly
exceed what the pod can actually allocate — exactly the failure mode
in the original report
(Available memory on node 0 (149.11/692.65 GiB) ... is less than requested memory for kv (206.54/207.8 GiB)).

pre-commit run --files vllm/utils/cpu_resource_utils.py passes
clean (ruff, ruff-format, mypy, SPDX).

[bigPYJ1151]

Thanks for the fix :)
get_memory_node_info returns per-numa node memory status. So maybe it's better to use cgroup/memory.numa_stat?

[maobaolong]

Thanks for the fix :) get_memory_node_info returns per-numa node memory status. So maybe it's better to use cgroup/memory.numa_stat?

@bigPYJ1151 Good point — memory.numa_stat is a much better fit. Updated to parse it, and use the per-node usage to clamp available_memory while distributing the cgroup headroom across nodes. Falls back to the previous global-limit path when memory.numa_stat is not available. PTAL.

[bigPYJ1151]

Hmmm, looks like cgroup doesn't provide per-numa node memory restriction and free size. This is a bit tricky...

[maobaolong]

Hmmm, looks like cgroup doesn't provide per-numa node memory restriction and free size. This is a bit tricky...

@bigPYJ1151 You're right that cgroup doesn't expose per-node limits/free, so any
per-node split has to be an approximation. What this PR does:

• total_memory = min(host_node_total, cgroup_limit)
  Per-node cap. Slightly over-estimates on multi-NUMA pods (sum can
  exceed the global limit), but vLLM's CPU worker only consumes the
  node it's bound to, so it never actually allocates beyond cgroup_limit.

• available_memory = min(host_node_free - cgroup_node_used,
  cgroup_limit - sum(cgroup_node_used))
  The first term is a per-node estimate from memory.numa_stat
  (anon+file). The second term is a hard pod-wide ceiling — even if
  the per-node estimate is off, the sum across nodes can never exceed
  what the pod really has left.

• If memory.numa_stat is unavailable, falls back to the pod-wide
  limit/usage path, same as before.

So the per-node numbers are best-effort, but the pod-wide invariants
(no node exceeds cgroup_limit, sum of available <= cgroup headroom)
always hold.

[bigPYJ1151]

@maobaolong Ok, let's ignore the NUMA memory status in this PR, it needs further thoughts. Just clamp the total memory size when cgroup has memory restriction.

[maobaolong]

@bigPYJ1151 Sounds good, dropped the numa_stat parsing. Now it just clamps total_memory / available_memory against the cgroup limit when one is set. PTAL.

[maobaolong]

@bigPYJ1151 Thanks for the help of this PR. The failed ci check seems flaky test, would you like to help me to trigger the failed ci retry. Thanks!