Activity
7K followers
-
Keerti Melkote shared thisRobert Nishihara says “Inference is a subroutine of larger more complex AI pipelines”. This is a very succinct way to understand what is happening in AI right now. AI projects are graduating from custom inference to custom models. The business imperative is shifting from simply lower costs to owning a moat. The moat is the data and the AI learning loop. Learning loops require complex orchestration of rollouts, data, evals, policy updates and more across a heterogeneous compute estate of GPUs and CPUs. Inference is a subroutine in this context. It’s still critical. But a part of a whole that is more complex. For this new era of AI, composability becomes a key aspect without giving up on performance. Ray is the backbone for this era with Ray Serve as the most ergonomic way for developers to compose model serving as a part of the AI learning loop. But that is not an excuse for lower performance. Performance still matters in this context. This is why we have focused on improving Ray Serve performance 4.4x for prefill and 28x for decode stages. We are excited for what this does to unify the disparate parts of the AI learning loop into a single cohesive AI backbone for all your varied workload needs. Read more about the performance optimizations in this blog: https://lnkd.in/gVdsg7cj Try it out in Ray 2.56 or easier still on Anyscale, and join us on the Ray Slack to share feedback!High Performance Distributed Inference with Ray Serve LLM | AnyscaleHigh Performance Distributed Inference with Ray Serve LLM | Anyscale
-
Keerti Melkote shared thisOne of the biggest shifts happening in AI right now isn't about model intelligence. It's about control. Over the last two years, APIs have been the fastest path from idea to application. They helped organizations experiment quickly and prove value. But as AI moves deeper into production, many enterprises are running into two realities: (1) The capabilities available to them are increasingly defined by model providers. Recent launches have shown how new limits are being put in place to build differentiated AI systems. (2) The economics become harder to predict. As model providers look to monetize beyond infrastructure costs, organizations face rising per-token costs. For many enterprises, the question is shifting from "How do we use AI?" to “How do we own our AI?” The shift from renting to owning intelligence means controlling where models run, where multimodal data is processed, and how costs scale. Ultimately with the goal of creating a competitive advantage. This is exactly why we're excited about the Anyscale on Azure announcement at Microsoft Build. We're already seeing this transition with companies like Xoople and Wayve. These AI-native organizations were among the first Azure customers to move beyond experimentation and build AI platforms that give them full control over their data, models, and infrastructure. Today, more enterprises are following the same path. Anyscale on Azure, now available to all Azure users is the foundation to that path: https://lnkd.in/gCggHp3EAnyscale Launches on Microsoft Azure as a Native Integration for Enterprises to Build Sovereign AI and Take Control of Variable API CostsAnyscale Launches on Microsoft Azure as a Native Integration for Enterprises to Build Sovereign AI and Take Control of Variable API Costs
-
Keerti Melkote reposted thisKeerti Melkote reposted thisWe just published a deep dive on Anyscale on Azure, a new Azure Native integration now in public preview. The way enterprises consume AI is changing. The first wave was about calling externally hosted APIs to run inference. The next is about building AI systems on your own proprietary data, inside infrastructure you control — what's increasingly called sovereign AI. The post covers: - How Anyscale runs as an Azure Native integration, co-engineered with Microsoft, governed by the same Azure RBAC, Entra SSO, and Policy your platform team already uses - Why the full AI lifecycle (data prep, training, and inference) belongs on one compute foundation instead of stitched-together tools - How Wayve and Xoople are already running production AI on Anyscale on Azure, from autonomous driving to planetary-scale satellite imagery Bringing this to public preview took deep work from the engineering team. Special thanks to Aashutosh Khandelwal, Adhip Gupta, Allen Yin, Chris Fellowes, Chris Sivanich, Daniel Arrizza, Dwaipayan Mukhopadhyay, Douglas Strodtman, Elizabeth Hu, Gopal Agarwal, Lanbo Chen, Matt Eshelman, Naila K., Omar Shorbaji, Pei Yang, Sanjeeb Bhanja, Tim Dwyer, Tim You, and Toji George — and many others who helped get it shipped. Read it here: https://lnkd.in/gPD8UsK9
-
Keerti Melkote reposted thisKeerti Melkote reposted thisToday at BUILD we officially announced Anyscale on Azure! This new Azure Native integration gives enterprises a new way to build and operate AI with more sovereignty, efficiency, and reliability. As companies move from AI experimentation to production, many are discovering that relying solely on external AI APIs creates growing cost, governance, and operational challenges. Anyscale on Azure is purpose-built for this shift, giving enterprise AI and platform teams a unified compute foundation for the entire AI lifecycle, not just one stage of it. With Anyscale on Azure, enterprises can run the full AI lifecycle, keep proprietary data, models, and pipelines inside their own Azure environment, and replace unpredictable per-token API costs with governed compute infrastructure. Special thanks to Robert Nishihara, Ion Stoica, Philipp Moritz, Keerti Melkote, Pradeep Iyer, Lanbo Chen, Jooree Na, Julian Forero, Katarina Stanley, Lou Serlenga, Chad Carlisle, Anirudhya (Arnie) Dasgupta, and the entire Anyscale and Microsoft teams for the deep collaboration in empowering the next wave of enterprise AI. Learn more here: https://lnkd.in/gHhwnNsd. And if you’re at Microsoft Build, come connect with us at Booth G201!
-
Keerti Melkote reposted thisKeerti Melkote reposted thisCursor just released a frontier coding model with 4x faster generation. They will be speaking at Ray Summit about their journey building a frontier coding model. - Training on 1000s of GPUs - Scaling 100,000s of sandboxed coding environments - Custom training infrastructure with PyTorch and Ray - Custom MoE kernels, expert parallelism, hybrid sharded data parallelism They'll be speaking in much more detail next week at Ray Summit: https://lnkd.in/gE8pn3sv
-
Keerti Melkote reposted thisKeerti Melkote reposted thisRay Summit is going to be excellent. Can't wait to hear from xAI, Perplexity, Cursor, Thinking Machines Lab, Physical Intelligence, Applied Intuition, Prime Intellect, vLLM, SGLang (sgl-project), and so many others. Some major themes this year that come up over and over - Reinforcement learning infra - Multimodal data (lots of video) - Distributed inference - Scalable agent infrastructure - Robotics / autonomy https://lnkd.in/gE8pn3sv
-
Keerti Melkote reposted thisKeerti Melkote reposted thisThe Ray Summit 2025 agenda just dropped! 🔥 80+ deep technical sessions from Al researchers and builders across every industry from autonomous vehicles to finance and media. 👉 Check out the agenda → https://lnkd.in/e-_Sm2E3 This year’s focus: AI in production Hear from builders at Meta, Netflix, Apple, Cursor, Bridgewater Associates, J.P. Morgan, Adobe, Perplexity, Roblox, Anthropic, NVIDIA, Microsoft, Amazon, Google, Zoox, ByteDance, Coinbase, DataRobot, Autodesk, Grab, Pinterest, Character AI, Physical Intelligence, Applied Intuition and many more as they share how they’re building and scaling the next generation of distributed AI systems. 🗓 November 3–5 • San Francisco We hope to see you there!
-
Keerti Melkote shared thisIts #RaySummit time again Nov 3-5. And the lineup is looking fantastic. Especially the tech breakouts. These aren���t just “how we thought about it” talks. They’re “here’s the stack, here’s what broke, and here’s what we’d do differently” talks. Some I can’t wait for: 🔹 Character.ai on Scaling LLM Post-Training 🔹 The State of vLLM in 2025 🔹 Roblox on Training 3D Foundation Models with Ray 🔹 xAI on Scaling Image + Video Processing 🔹 Zoox on Reliable, High-Velocity Model Serving 🔹 Perplexity on RDMA P2P for KvCache + Weight Transfer + MoE Join us at #RaySummit 2025. Register using the link below. https://lnkd.in/gqKCqwGm
-
Keerti Melkote reposted thisKeerti Melkote reposted thisThe Ray Summit CfP is closing on July 14! We'll be back in San Francisco from Nov 3-5 for Ray Summit and want to showcase your work. Whether it's scaling smoother, building GenAI workflows, or creating complex ML systems - if you've built it with Ray, we want to hear about it. Talk formats: • Lightning Talks (15 min): Quick demos, focused ideas, real code • Breakouts (30 min): In-depth stories, real-world wins, technical takeaways Key tracks: Ray Ecosystem • Generative AI • Multimodal • vLLM • RL • Post-training • and more! Submit now: https://lnkd.in/gjvKdvFF
-
Keerti Melkote liked thisKeerti Melkote liked thisRay on the Road made its final city stop at Ray Day: London, with a keynote from Anyscale co-founder and CTO Philipp Moritz and four user talks on running AI at production scale. 🔆 Thank you to our speakers. Marcell Ferencz opened the user talks with how Xoople scales geospatial foundation model inference on Ray and Anyscale. Martin Iglesias and Maxime Battello followed with the Ray-powered foundational model behind ML at Adyen. Paul Coursaux broke down multimodal training and inference at Criteo, and Thomas Riedl closed with BMW Group's AI gateway and its expansion from chat to video on Ray and vLLM. Catch highlights from every talk in the recap 👉 https://lnkd.in/gYM5wWd2 The road leads to Ray Summit: August 24–26 in San Francisco, with three days of keynotes, breakouts, and hands-on training. Request your invite → https://lnkd.in/g---W7X3
-
Keerti Melkote liked thisKeerti Melkote liked this𝗘𝘅𝗰𝗶𝘁𝗲𝗱 𝘁𝗼 𝘀𝗵𝗮𝗿𝗲 𝘁𝗵𝗮𝘁 𝘄𝗲’𝘃𝗲 𝗹𝗮𝘂𝗻𝗰𝗵𝗲𝗱 𝗔𝗻𝘆𝘀𝗰𝗮𝗹𝗲 𝗼𝗻 𝗔𝘇𝘂𝗿𝗲, bringing the open-standard Ray runtime into Azure as a native service to help enterprises run end-to-end AI as a single system. What we’re seeing across customers: 𝗔𝗜 𝗶𝘀𝗻’𝘁 𝗳𝗮𝗶𝗹𝗶𝗻𝗴 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗼𝗳 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗜𝘁’𝘀 𝗮 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝗽𝗿𝗼𝗯𝗹𝗲𝗺. Fragmented stacks often lead to: → 30–40% GPU utilization → Rising infrastructure costs → Too much glue code across data, training, and inference Leading AI teams are moving to unified runtimes to drive: → 80%+ GPU utilization → 40–60% lower infrastructure spend → Faster paths from experimentation to production With 𝗔𝗻𝘆𝘀𝗰𝗮𝗹𝗲 𝗼𝗻 𝗔𝘇𝘂𝗿𝗲, customers can: → Run data prep → training → inference in one runtime → Keep data, models, and governance inside their Azure tenant → Operate under Entra ID and Azure-native controls end to end We’ll also have a few launch posts and blogs going live that go deeper into the product, architecture, and customer scenarios behind Anyscale on Azure. I’ll share those as they become available. 𝗜𝗳 𝘆𝗼𝘂’𝗿𝗲 𝗮𝘁 𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁 𝗕𝘂𝗶𝗹𝗱, 𝗰𝗼𝗺𝗲 𝗺𝗲𝗲𝘁 𝗺𝗲 𝗮𝘁 𝗺𝘆 𝘀𝗲𝘀𝘀𝗶𝗼𝗻. I’ll be joining the table talk 𝗦𝗰𝗮𝗹𝗲 𝗣𝘆𝘁𝗵𝗼𝗻 𝗔𝗜 𝗙𝗮𝘀𝘁𝗲𝗿 𝘄𝗶𝘁𝗵 𝗔𝗻𝘆𝘀𝗰𝗮𝗹𝗲 𝗼𝗻 𝗔𝘇𝘂𝗿𝗲, where we’ll discuss how teams can move from notebooks to large-scale distributed AI workloads using Ray, Anyscale, AKS, and Azure. 𝗥𝗲𝗴𝗶𝘀𝘁𝗲𝗿 𝗳𝗼𝗿 𝗺𝘆 𝘀𝗲𝘀𝘀𝗶𝗼𝗻: Option 1: https://lnkd.in/g6__8ZjV Option 2: https://lnkd.in/geYCnKsa You can also see the full Anyscale at Microsoft Build agenda here: https://lnkd.in/gM6WT4Rw Kudos to the team who helped get us here: Brendan Burns Robert Nishihara Kenneth K. Kaysie Yu Lachlan Evenson Pradipa Karbhari Anson Qian Lou Serlenga Matt Eshelman Pradeep Iyer Daniel Arrizza Naila K. Chad Carlisle Julian Forero Katarina Stanley Sanjeeb Bhanja Adhip Gupta Evan Hissey Thomas Yip Minu Iyer Pls feel free to DM me or grab time at Build. Happy to dive deep. #MicrosoftBuild #Azure #Anyscale #Ray #AKS #AIInfrastructure #GenerativeAI #CloudNative #MLOps #EnterpriseAI
-
Keerti Melkote liked thisKeerti Melkote liked thisDeeply honored to be recognized for this milestone. This journey would not have been possible without my mentors Dipankar Raychaudhuri Partha Narasimhan, Pradeep Iyer and Keerti Melkote and my collaborators Stuart Walker Strickland Chuck Lukaszewski Eldad Perahia Peter Thornycroft Gaurav Patwardhan Shahnawaz Siraj Andre Beaudin Qiang Z. HAO LU Karthik Srinivasa Murthy Farhan Hasnain GOPALAKRISHNA RAMAN Dan Harkins Omar El Ferkouss Rajini Balay Abhiruchi Dakshinkar Nitin Changlani
-
Keerti Melkote liked thisJoin us!Keerti Melkote liked thisThe Ray Core and Ray Data teams at Anyscale are actively hiring system engineers in Bengaluru! Ray Core serves as the cornerstone of the entire Ray ecosystem and powers libraries like Ray Train, Ray Data, and Ray Serve — quickly adopted by companies like OpenAI, DeepSeek, Spotify, Uber, DoorDash, Pinterest, Apple, and many more. Ray Data takes this further as the scalable data processing engine behind the training and inference pipelines of some of the largest AI models in the world. In this role, you will play a pivotal part in shaping the future of Ray and Anyscale. Particularly in the context of the growing importance of opensource and LLMs, you will be a crucial contributor to our strategic goal of establishing ourselves as the compute substrate of this unprecedented AI wave. If the prospect of tackling challenges like: → Scaling clusters to 10K+ nodes → Optimizing network transfer speed for petabyte-scale workloads → Building the data and execution infrastructure behind large multi-modal models → Pushing the limits of distributed training and inference …excites you, I encourage you to apply or message me directly if you have any questions. 📍 Bengaluru 🔗 Open roles: https://lnkd.in/ga6R6bvJ #Hiring #Bengaluru #DistributedSystems #RayCore #RayData #OpenSource #LLM #MLInfra
-
Keerti Melkote liked thisKeerti Melkote liked this🔆 The co-founder of Kubernetes and the co-creator of Ray, in one conversation about the future of enterprise AI. We sat down with Brendan Burns (Corporate Vice President, Microsoft & co-founder of Kubernetes) and Robert Nishihara (co-founder, Anyscale & co-creator of Ray) to talk through Anyscale on Azure – and why running AI on Azure Kubernetes Service matters. The throughline: Kubernetes became the natural foundation for AI applications. Ray adds the AI-native compute layer on top, and Anyscale brings the production layer – a performance-enhanced runtime, developer tooling, and managed operations – so teams stay focused on building instead of managing clusters. Delivered Azure Native on AKS, teams are seeing 4x faster development and 50% higher GPU utilization. Worth a watch 👇
-
Keerti Melkote liked thisCongratulations to NVIDIA on the release! Super exciting to see two models trained with Ray back to back (MAI-Thinking-1 and Nemotron 3 Ultra).Keerti Melkote liked thisToday we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.
-
Keerti Melkote liked thisKeerti Melkote liked thisTwo years in a row! Honored to be named to the 2026 #100WomenInAI list alongside an incredible group of women building the future of AI. Thank you to XFactor Ventures and Flybridge for the recognition, and congratulations to my fellow honorees Daniela Amodei Lisa Su Fei-Fei Li Mira Murati Tekedra N. Mawakana Daphne Koller Daniela Rus @Yejin Choi Niki Parmar Joelle Pineau Sarah Friar Anca Dragan @Helen Toner Lila Ibrahim MeeLan Lee Sara Achour Yasmin Razavi @Alondra Nelson Anima Anandkumar @Jennifer Tour Chayes Cynthia Breazeal Sonya Huang May Habib Ann Miura-Ko Pelonomi Moiloa Sanja Fidler @Kara Swisher Sarah Bird Talia Goldberg Grace Brown Marian Croak Maya Ackerman, PhD. Kakul Srivastava Elham Tabassi @Kate Crawford @Sara Hooker @Denay Solis @Grace Isford @Kari Briski @Karen Hao @Dannie Herzberg @Maryam Ashoori @Bianca Anghelina @Rudina Seseri @Minna Song @Sneha Revanur @Garima Kapoor @Esha Joshi @Rana el Kaliouby @Cecilia Ziniti @Aparna Ramani @Kiara Nirghin @Amy Wu Martin @Liz Bacelar @Aditi Maliwal @Sonia Kastner @Annie Lu @Aparna Dhinakaran @Stephanie Godin @Gigi Yuen-Reed @Rachel Schutt @Anita Modi @Katherine Kostereva @Sruthi Viswanathan @Karine Mellata @Dipanwita Das @Eunice Wu @Sunita Verma @Amanda Kahlow @Sara Buchner @Phaedra Boinodiris @Ece Kamar @Juliet Shen @Sharon Goldman @Caitlin Leksana @Ania Musial @Erika Bondereva @Isa Watson @Rebecca Hu @Ayesha Khanna @Radha Jain @Shannon Kay @Elena Ikonomovska @Hafeezah Muhammad @Krittika D’Silva @Lake Dai @Genevieve Smith @Angela McNeal @Mayada Gonimah @Parisa Zehtabi @Ainsley MacLean @Eliza Kosoy @Carmin Marin de Leon @Danielle Regis @Olivia Kane @Victoria Westerhoff @Gina Aquilano @Samira Rahimi @Taesen Chavis @Laura Montoya @Latoya Montoya Sancus Ventures Carnegie Mellon University Carnegie Mellon University - Integrated Innovation Institute CMU Silicon Valley Carnegie Mellon University's Information Networking Institute
Other similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content