Anyscale GPU Health Observability at Ray Summit

👉𝐓𝐡𝐫𝐞𝐚𝐝𝐫𝐢𝐩𝐩𝐞𝐫 𝐇𝐚𝐥𝐨 𝐒𝐭𝐚𝐭𝐢𝐨𝐧 🟰 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥 𝐒𝐮𝐩𝐞𝐫𝐜𝐨𝐦𝐩𝐮𝐭𝐞𝐫 It brings datacenter-class AI compute into a single deskside workstation. It pairs an AMD Ryzen Threadripper PRO processor with Instinct-class accelerators, delivering the memory capacity and bandwidth needed to train, fine-tune and run large models locally as well as run intensive agentic workflows—without a server room, a cloud queue or shared tenancy. It helps keep sensitive data and IP on-premise. #Workstation #AI #Onprem #Supercomputer #LLM #SharedTenancy #AgenticWorkflows #AIDevelopers AMD

AMD’s Threadripper Halo Station redefines local compute by combining 96 PRO cores with MI350P accelerators. The ability to run trillion-parameter AI models locally—without relying on the cloud—is a massive leap forward for developer workflows and enterprise data privacy.

NVIDIA at Hot Chips: Vera, Rubin, Groq 3 LPX, Spectrum-X, BlueField-4 — one stack built for agentic workloads at rack scale. AMD: Helios + MI455X in the data center, and Strix Halo on the desk with 128GB unified memory for local 70B–120B models. https://lnkd.in/p/embyp9A2 Agents will live in both places. Fast tokens in the factory. Private inference at the edge. Which layer matters more for your work over the next 12 months — rack-scale throughput, or a box you can actually own?

Came across this on LinkedIn — a great image and probably one of the clearest explanations I’ve seen of the role of the SuperNIC in AI infrastructure. What I also like is how closely it connects to what we’re doing at Netris. The NIC/SuperNIC accelerates the GPU’s connection into the network; Netris works across the fabric layer and right into the host, helping configure and maintain NVIDIA BlueField-3, ConnectX-7 and ConnectX-8 SuperNICs alongside the network itself. We also manage BlueField DPUs as part of the EVPN/VXLAN fabric, extending automation and hardware-enforced multi-tenancy all the way into the server. Different layers of the stack, but the same objective: remove complexity and bottlenecks between the GPU and the fabric, and ultimately accelerate time to token.

GLM-5.3 from Z.ai. is live on Telnyx. A 753B-parameter reasoning model with 1M token context joins our roster of leading open-source models hosted on Telnyx infrastructure. GLM-5.3 has comparable intelligence to Kimi-K3, but it beats K3 on price. It's roughly half the cost of Kimi K3 on input and 67% cheaper on output, perfect for devs looking to scale agentic use without the pricetag. GLM-5.3 is running on telnyx-owned GPUs, on the same private backbone as your voice, messaging, and compute traffic. Test it today in the portal or via API https://lnkd.in/gFZZzw_k

NVIDIA GB300 NVL72 IS 7X BETTER 💰💰 PERF PER DOLLAR 💰💰 THAN H200 on long context agentic workloads due to disagg prefill and wide expert parallelism optimizations which take advantage of NVL72 copper backplane.

In this roundtable with Supermicro and Vultr, our CEO Tenry Fu explains where the local-first inferencing inside AMD Instinct™ Coder comes from and the amazing ROI customers are seeing, with token savings on one side and fast hardware payback on the other. The conversation also covered why AI coding costs have climbed all the way to CFO conversations, the data sovereignty concerns that come with them, and how the partnership between Supermicro and Vultr grew from small CPU projects into large-scale GPU deployments. Hear more about the partnership behind the box and the hosted option for teams that don't want to run hardware 👉 https://okt.to/pSZnl4

Edge Impulse is a good product. It also requires an account, uploads your sensor data, and ships a C++ runtime that is comfortable on a Cortex M4 and far too heavy for the small RISC-V parts I work with. Oh and its now owned by Qualcomm. So I built the workflow into Rovari Studio and it runs entirely on your machine. In this video I demonstrate TinkerStream, I train a gesture classifier on a CH32V307 with an MPU6050. I capture labelled accelerometer data live off the serial port, extract windowed statistics, train a model and export a single C99 header. It's all offline and requires no account or cloud connection. https://lnkd.in/emY5dWC4 #RISCV #EmbeddedSystems #TinyML #EdgeAI

MiniMax M3 is a 23B-active-parameter model that can still demand an 8-GPU deployment. That sounds contradictory until you look at what is actually happening under the hood. M3 has 428B total parameters, with ~23B active per token. But the full model weights still have to live somewhere, putting the memory requirement at roughly: • 856GB VRAM at BF16 • 440GB at FP8/MXFP8 • 8-GPU tensor parallelism in the documented deployment setup The interesting part is that M3's architecture is doing something different on the attention side too. Its MiniMax Sparse Attention (MSA) limits each query to 2,048 KV tokens, even with a context window reaching 1M tokens. That is what makes the long context practical without paying the full attention cost of a standard transformer. Our new guide breaks down the actual deployment math, the vLLM configuration, the --block-size 128 requirement, and the licensing conditions to check before deploying M3 commercially. MiniMax M3 Self-Hosting Guide: GPU Requirements and Serving Setup https://lnkd.in/gdXiWGE8 #LLM #GPU #AIInfrastructure