Amin Vahdat

🚀 PagedAttention: Borrowing from OS Design to Solve LLM Memory Bottlenecks 🎇🎇Just revisited the vLLM paper that revolutionized how we serve large language models in production. The core insight? Treat KV cache memory management like virtual memory in operating systems. ✈️The Problem: When serving LLMs, the key-value cache can consume up to 30% of GPU memory. Traditional systems suffer from fragmentation and can't share memory across requests, leading to massive underutilization. 🚕The Solution - PagedAttention: Stores KV cache in non-contiguous memory blocks (like OS paging) Enables near-zero waste memory management Allows memory sharing across parallel sequences (beam search, multi-queries) Achieves 24x higher throughput than HuggingFace Transformers 🎤🎤Why This Matters for Manufacturing AI: Most of Industry on-premises deployments , memory efficiency directly impacts: 🚕Cost per inference for quality inspection systems 🎇Latency in real-time fault diagnosis 🎇Number of concurrent requests we can handle on limited edge hardware The paper elegantly shows how classical CS concepts (virtual memory, paging) can solve modern AI infrastructure challenges. For anyone deploying LLMs in production - whether cloud or edge - understanding PagedAttention's memory management is essential. Paper: https://lnkd.in/gbvvy5xf #LLM #AI #MachineLearning #MLOps #ManufacturingAI #IndustrialAI #vLLM #PyTorch

11

As enterprise workflows move toward a wide range of autonomous AI agents, how do we maintain control and data sovereignty? Based on my experience, the key lies in an age-old paradigm: Hunters vs. Farmers. In my latest article, I explore why a strict Separation of Duties (SoD) is essential for the long-term safety of Agentic AI. We look at: 🏹 Hunter AI (The Edge): Proactive executors deployed in end-user environments to fetch privileged data and execute tasks. 🚜 Farmer AI (The Core): Strategic orchestrators that distill information, manage bias, and interface with humans. I also break down the "Maturity Curve"—how the role of Human-in-the-Loop evolves from approving every small adjustment to managing only high-risk, high-cost actions. Read the full article here: 🔗 Medium: https://lnkd.in/gfC_5pbh 🔗 Personal Blog: https://lnkd.in/gQdA2FvG #AI #AgenticAI #EnterpriseArchitecture #AISafety #MachineLearning #TechLeadership

11

Excited to share the release of ALCHEMI Toolkit-Ops v0.3.0! This release includes: + Molecular dynamics integrators and geometry optimization + JAX bindings + DSF electrostatics + Differentiable virial for Ewald and PME + GPU-side batch rebuild detection and selective skip This release builds on existing features (e.g., batched neighbor lists, DFT-D3 corrections, Ewald and PME) from v0.2.0. Learn more and get started: https://lnkd.in/g6UpX7mz

97

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58

🚀 2026 IS THE YEAR AI STOPS BEING A TREND AND STARTS BEING INFRASTRUCTURE. The global AI boom is reshaping our world—and the pace is accelerating faster than we anticipated. Here's what just happened in the AI world: ✅ OpenAI + Cerebras: $10B partnership to supercharge inference speed ✅ DeepSeek's R1: Open-weight models proving innovation doesn't require massive budgets ✅ Apple × Google: Siri + Gemini marking a major tech alliance ✅ Agent Memory Systems: HINDSIGHT & frameworks reducing token overhead by 84.7% ✅ MedGemma 1.5: AI understanding medical imaging (CT/MRI) with human-level accuracy ✅ Physical AI Boom: Tesla's Optimus, Alpamayo 1, autonomous systems in production ✅ Global Economic Impact: IMF revising growth forecasts upward due to AI productivity gains 🧠 THE INFRASTRUCTURE LAYER IS SOLIDIFYING We're moving from "Can AI do this?" to "How do we scale efficiently?" Token optimization, on-device AI, and agent memory systems = competitive advantages. 🎯 FOR ARCHITECTS & ENGINEERS: The companies building intelligent, distributed systems with: → Context caching → Memory management → Agentic workflows ...will own the next decade. #AI #AgenticAI #LLM #TechArchitecture #AIInfrastructure #DeepLearning #OpenAI #Innovation #Future

1

Memory management is one of the biggest challenges AI engineers face today. LLMs are stateless by design, which means every intelligent agent needs a robust memory layer to truly perform in real-world environments. With the introduction of the Redis Agent Memory Server, Redis is tackling this challenge head-on. The open-source solution brings a two-tier memory architecture (working + long-term memory), semantic retrieval, metadata filtering, and asynchronous memory extraction — all built for production scale and speed. It’s a powerful example of how Redis continues to evolve beyond caching into a core infrastructure layer for AI-native applications. If you're building AI agents and thinking about context engineering, persistent memory, and intelligent retrieval, you should explore this! Thank you to Andrew Brookins, Principal Applied AI Engineer, and Raphael De Lio, Developer Advocate at Redis, for sharing this article with us! ➡️ Read the full article on our magazine: https://lnkd.in/db3-fhZT #AI #Redis #AgenticAI #LLMs #DeveloperTools

5