[Security] Fix info disclosure via int32 truncation in GGUF dequantize kernels

[jperezdealgaba]

Purpose

Fix an information disclosure vulnerability caused by integer truncation in GGUF dequantize kernels (csrc/libtorch_stable/quantization/gguf/).

The to_cuda_ggml_t function pointer typedef declares its element count parameter k as int (32-bit). When a GGUF model has weight tensor dimensions whose product exceeds INT_MAX (e.g. a 65536x65536 matrix), the int64_t product m * n is silently truncated to int at the call site. The CUDA kernel then processes only the truncated number of elements. Since output tensors are allocated with torch::empty (uninitialized memory), the unfilled portion retains stale GPU memory which — in multi-tenant inference deployments — may contain tensor data from other users' requests.

Changes:

• Widen k from int to int64_t in the to_cuda_ggml_t typedef (ggml-common.h) and all 18 dequantize function signatures in dequantize.cuh
• Widen the dequantize_block kernel's index arithmetic to int64_t to match
• Widen col, batch, vecs, and padded local variables in gguf_kernel.cu from int to int64_t to prevent the same class of truncation in the matmul and MoE paths
• Defense-in-depth: zero-initialize all output tensors via torch::stable::fill_(Y, 0.0) in ggml_dequantize, ggml_mul_mat_vec_a8, ggml_mul_mat_a8, and ggml_moe_a8, so that even if a future truncation bug occurs, stale GPU memory is never exposed

Test Plan

# Build and verify compilation succeeds
python setup.py build_ext --inplace
# Run existing GGUF tests to verify no regression
python -m pytest tests/ -k "gguf" -v

The type change from int to int64_t is ABI-compatible and all downstream call sites already pass int64_t values (e.g. m * n in ggml_dequantize). The fix eliminates the silent truncation at the function pointer boundary.

Test Result

• All existing GGUF kernel call sites pass int64_t values — the widened signatures eliminate the implicit narrowing conversion
• clang-format passes on all changed files
• ggml_moe_a8_vec already used fill_(Y, 0.0) — the other four functions now match

MR was created with the assisstance of: opus-4.6-high