Arun Sharma

Last week at International Symposium on Microarchitecture (MICRO) 2025, we presented our work PyTorchSim, a fast and cycle-accurate NPU simulation framework with comprehensive feature support. We would like to share it here as well. Highlights of PyTorchSim: - Integrated with PyTorch 2, it can simulate existing PyTorch models by simply designating a simulated NPU as the target device. - Provides an NPU-specific compiler backend based on MLIR and LLVM, enabling compiler optimizations and supporting the simulation of both training and inference. - Supports multi-core NPU and multi-model tenancy with detailed interconnect and DRAM models (Booksim and Ramulator 2). - Can model data-dependent timing behavior, such as that of mixture-of-experts models. - Implements a custom RISC-V–based ISA with a rich instruction set to express various operations in AI models. - Employs a Tile-Level Simulation technique, which enables fast simulation without loss of accuracy. - Validated against Google TPU v3, showing a mean absolute error (MAE) of 11.5%. PyTorchSim is available on GitHub. Please refer to our paper for more details. GitHub: https://lnkd.in/gmVDNCFD Paper: https://lnkd.in/gSKJVMwk MICRO’25 slide deck: https://lnkd.in/g6PjgFvm We will continue to enhance PyTorchSim’s capabilities to make it more useful for NPU architecture and compiler research. We would like to thank Prof. Jongse Park (KAIST) and Dr. Jeehoon Kang (Furiosa AI) for their valuable advice on this work. This project was led by our students Wonhyuk Yang, Yunseon Shin, and Okkyun Woo.

91

1 Comment

I'm excited to share my new article, "Multi-Turn Tool Use with RL." Link: https://lnkd.in/ge6hT6Vp tl;dr: In this post, I explore how we can train Large Language Models to move beyond simple, single-turn responses and effectively solve complex problems using external tools. The key challenge isn't just teaching a model to call a tool (like a Python interpreter), but to engage in a multi-turn loop of reasoning, acting, and observing. I did SFT to teach the ritual and then GRPO-RL to learn the strategy. My work documents an experiment training a 1.5B parameter model to master this Think → Act → Observe → Revise → Answer cycle. This is a small, concrete step, but it's foundational for building agents that can tackle long-horizon, verifiable tasks. I also detail some of the implementation challenges, including the need for a custom trainer to handle multi-turn interactions. #LargeLanguageModels #ReinforcementLearning #AI #MachineLearning #Agents #LLM #DeepLearning #RL #FineTuning

107

Love to see this! I've been saying this for a couple of years, researchers working with 3D data and simulation should really be looking at OpenUSD. 1. Why split your data among multiple file formats (OBJ, PLY, CSV, JSON, etc.) when USD can encode it all and you can keep everything in context? 2. Even if you want to keep the legacy formats, USD can aggregate everything without changing your canonical source. 3. Datasets in OpenUSD can be shared and built upon non-destructively. Reference someone else's dataset and just add your unique contributions on top. Tooling like this is a great step towards democratizing OpenUSD for more researchers. #OpenUSD #Robotics

46

3 Comments

I came across an interesting role: Sr. Applied Scientist – SLAM (Amazon Industrial Robotics), and it’s a good signal of where industrial robotics is heading. A few things stood out to me: • SLAM + Spatial AI is treated as a core module, not just a research topic—focused on real-time localization and continuous map updates in dynamic environments. • The scope clearly spans camera/LiDAR perception + semantic understanding, tying mapping/localization to obstacle detection and scene interpretation. • It’s very “end-to-end”: experiments → prototypes → production-quality C++/Python, with ownership that extends into deployed systems. • Strong emphasis on cross-functional execution (especially tight collaboration with hardware teams from prototyping to production) and representing the work via publications/presentations. Overall, it’s a nice example of how robotics orgs at scale are blending classical SLAM, modern ML, and production engineering into one integrated stack. #Robotics #SLAM #SpatialAI #Perception #LiDAR #ComputerVision #Autonomy #AmazonIndustrialRobotics https://lnkd.in/gCAnNXev

32

New LARA Method Enables Modular and Composable Behaviors for Frozen LLMs 📌 Researchers have introduced LARA, a post-training method that applies low-rank corrections to an LLM's residual stream. This approach allows for a Mixture of Behaviors architecture using a single frozen base model. 🔗 Read more: https://lnkd.in/dApGBHAR #lara #llm #machinelearning #post-training #residualstream

2

CDS-affiliated Assistant Professor of Computer Science Saining Xie, CDS founding director Yann LeCun, and others recently released a new paper, titled “Beyond Language Modeling: An Exploration of Multimodal Pretraining.” The paper investigates new methods for training models across multiple data types, and focuses on moving past the limitations of standard language modeling to create more versatile systems. https://lnkd.in/gMz-DCM2 #MultimodalPretraining #DataScience #MachineLearning #CDS

308

6 Comments

In this 20-minute talk, PolyMage Labs founder Uday Reddy Bondhugula talks about why and how one should build effective compilers for AI programming frameworks: https://lnkd.in/gWUQNUsF

81