About
Personal academic website:
www.cs.princeton.edu/~fheide
Find me on Google…
www.cs.princeton.edu/~fheide
Find me on Google…
Activity
11K followers
-
Felix Heide shared thisWe built an efficient training infrastructure to support self-play at scale 🚀 (and end-to-end from sensor data!) that hinges on heterogeneous compute 😶🌫️! Check out my recent talk at #RaySummit on how we build our stack and train it at scale: https://lnkd.in/eywXSCwQ
-
Felix Heide shared thisSuper excited to share this unedited fully autonomous truck ride! We're seeing strong zero-shot planning capabilities emerge from self- and mixed-play RL training at scale in complex situations. Our stack has not been trained on scenarios from this route, and the generalization capabilities are tremendous! Proud of the Torc Robotics team for their work on this frontier architecture for autonomous driving.
-
Felix Heide shared thisCome check out WorldFlow3D at #ECCV 2026 in Malmö this week! We reformulate 3D generation as flowing through sequentially finer 3D distributions. Vectorized layouts provide control for indoor and outdoor scene generation at unbounded scale! Meet Julian O. and Amogh Joshi at the poster session in ExHall #51 on September 12th, 3:00 PM – 5:00 PM, or check out the paper and our interactive demo at https://lnkd.in/g8nA3xrA .
-
Felix Heide shared thisSuper excited to have Mike McAuliffe on board!Felix Heide shared thisExcited to join Cephia as CEO, working with the mission-driven founders - Ethan Tseng, Howard Dong and Felix Heide - and team to lead our next phase of growth in AI-native sensor engines for the world of Physical AI machine vision. Privileged to work with renowned deeptech investors and collaborators including Incharge Capital, MetaVC Partners, NRM Partners, Radiant Opto-Electronics, SOSV and Princeton University. The clue is (buried!) in our name. Superhuman multimodal vision platform > ultra-compact form factors> photon efficiency > redefining sensing. Will be fun. www.cephia.ai #machineVision #physicalAI #multimodalSensing #robotics #drones https://lnkd.in/gqwAcYu7
-
Felix Heide shared thisCongratulations to the NVIDIA on today's Alpamayo 2 Super launch. Excited to see the open innovation that this enables in robotics!Felix Heide shared thisToday, we’re launching Alpamayo 2 Super, our frontier open reasoning model for autonomous vehicles. Beyond seeing, Alpamayo understands and reasons through the complex world - thinks before it acts. It’s a powerful backbone for robotaxis, trucks, shuttles, delivery vans, tractors and the long tail of mobile robots—billions of autonomous machines someday. We’re releasing it for commercial use under OpenMDW-1.1 so teams can inspect it, fine-tune it and deploy it—open models advance safety and security. The next wave of AI is robotics—and it starts with autonomous vehicles. Great work, Alpamayo team! https://lnkd.in/g7urPEjp
-
Felix Heide shared thisScenarioControl (ECCV 2026) enables end-to-end controllable generative simulation 🚗✨for autonomous driving, allowing users to generate diverse traffic scenarios with fine-grained actor control while preserving realistic scene dynamics. We have released the code, and we’re looking forward to seeing what the community builds with it! 🔗 Check out our project page for the paper & code: https://lnkd.in/erXQ8THs 👏 Thanks to the first authors Lili Gao, Yanbo XU, William Koch #ECCV2026 #ComputerVision #AutonomousDriving #GenerativeAI #OpenSource
-
Felix Heide shared thisExcited to share that #TruckDrive 🚛 — our long-range autonomous driving dataset presented at #CVPR2026 — is now available to download on 🤗 Hugging Face! Built specifically for long-range truck autonomy, TruckDrive includes: 📊 475K multimodal samples 🏷️ 165K densely annotated frames 🎯 Detection benchmarks reaching up to 1,000m in 2D and 400m in 3D. 📷A purpose-built sensor suite with cameras, LiDARs, and 4D radars. Whether you're working on 👁️ perception, 🎯 tracking, 🛣️ planning, or 🤖 end-to-end driving, go check it out, and help shape the next generation of architectures for long-range autonomy! 🚀 🔗 Project website: https://lnkd.in/g3e5qtK6 🤗 Hugging Face: https://lnkd.in/g8YcVKHX Torc Robotics #AutonomousDriving #ComputerVision #Robotics #HuggingFace
-
Felix Heide shared thisEnd-to-end Self-play from Pixels! Excited to share Gigapixel: a high-throughput driving simulator that enables scalable self-play training of end-to-end driving models! Super fun work from the amazing Luke Rowe! We extended PufferDrive with a GPU-accelerated perspective renderer, enabling self-play training from pixel observations at 50k SPS on 1 GPU, with throughput scaling linearly with the number of GPUs. We also introduce self-play DAgger training, a sample-efficient alternative to self-play RL that produces robust driving policies in ~3000x fewer steps. Planners trained in Gigapixel transfer to photorealistic observations via lightweight perception adaptation, achieving SOTA on closed-loop driving benchmarks, all without human trajectory supervision. Moreover, scaling self-play yields proportional gains in policy performance, establishing it as a scalable strategy for training end-to-end models. Website: https://lnkd.in/gfGqMQ6E GitHub: https://lnkd.in/geR_JYJj Really cool collaboration between Mila - Quebec Artificial Intelligence Institute, New York University, Princeton University and Torc Robotics: Roger Girgis, Rodrigue de Schaetzen, Daphne Cornelisse, Alaap Grandhi, Eugene Vinitsky, Chris Pal, and Liam Paull.
-
Felix Heide shared thisHeading to Denver this week for #CVPR 2026. We're also presenting TruckDrive 🛻 at the conference — a long-range highway-scale multimodal dataset built specifically for long-range truck perception and planning. TruckDrive extend ranges to 1,000m for 2D detection and 400m for 3D, using FMCW LiDAR, 4D radar, and high-res surround cameras mounted on a production semi-truck. State-of-the-art models drop between 31–99% on 3D perception tasks beyond 150m. That gap is what we're building toward. 📰 https://lnkd.in/eUZJTnnd I'll be presenting at two workshops on Thursday, June 4: ➡️ 9:20am MDT — UG2+ WorkshopRobot Learning with a 1000m Horizon: Ultra Long-Range Scene Understanding and Planning for Autonomous Driving https://lnkd.in/eqBSpMpf ➡️ 11:00am MDT — URVIS Workshop Unified Robotic Vision with Cross-Modal Sensing and Alignmenthttps://lnkd.in/eSr_-xZV If you're at CVPR stop by Booth #449 to meet the Torc Robotics AI team too!
Felix Heide replied to a comment
2w
Shantanusinh Parmar we run a multimodal driving stack based on camera (top row), LiDAR and radar.
Felix Heide replied to a comment
2mo
Patrick Denny Yes, we are looking at this! It's well over 100TB, though, so we are evaluating a reduced set of scenes to keep it manageable.
-
Felix Heide liked thisFelix Heide liked thisI left Berlin feeling like a failure… My goal was to run 26.2 miles in under 5 hours. I trained. I planned. I did the long runs. I ate the carbs. I obsessed over the details. Race morning came and the conditions were basically perfect…and I didn’t hit it. The thought of failing crushed me. I ran nearly 30 minutes faster than my last marathon just 6 months ago. A huge personal best…and I still felt like I failed. I almost didn’t celebrate it because it wasn’t the number I decided “success” was supposed to be. That got me thinking about how often we do this in life and at work. We get the opportunity but not the title. We grow…but not as quickly as we planned. We make progress…but miss the deadline we gave ourselves. We accomplish something that an earlier version of us would be so proud of…and immediately move the goalpost. I’m still trying to figure out why we do this. Instead of celebrating our wins…we bury them in comparison to other’s “wins” and our own manufactured failure. Berlin gave me a huge PR, my first major marathon finish, memories I’ll have forever and a very humbling reminder that I am capable of more than I give myself credit for. Since Sunday… my thoughts kept landing on something I think we can all benefit from. Don’t become so focused on the thing that didn’t happen that you forget to thank God and celebrate everything that did. Berlin, thank you. 🇩🇪🏃🏾♀️✨ .
-
Felix Heide liked thisFelix Heide liked thisA team led by our partner Dr. Aris Miro Marinello, including Niels Schwaiger and Sebastian Pollok advised Kembara VC on its investment in energy tech company Reverion. Congratulations to Pierre Festal, Robert Trezona and team! Read more here: https://lnkd.in/emiEgMse
-
Felix Heide liked thisFelix Heide liked thisWorld Labs is joining AMD. This is a huge moment for World Labs, our team, and for me. I wanted to take a moment to share what this means and why I’m so excited for this next chapter – read more in my Substack linked below. The research and technical breakthroughs we have achieved since our founding in 2024 have given us a clear vision for AI’s potential to solve problems in the spatial and physical world. To accelerate into this future requires scaling our efforts, scaling our reach, and getting closer to the hardware. We began a deep technical partnership with AMD last year, starting with model training and inference optimization on AMD GPUs. As our teams worked together, we realized it would be a natural fit to bring together our AI ecosystem of software and hardware, foundation models, and applications. I will join AMD as an Executive Vice President and Chief Scientist, working directly with CEO Dr. Lisa Su, Justin Johnson and Ben Mildenhall to continue leading the World Labs team as it joins AMD to form a world leading frontier research organization. Together, we are committed to building out an end-to-end open AI ecosystem spanning hardware, software, platforms, and widely accessible open models. https://lnkd.in/g8gQ54aj
-
Felix Heide liked thisFelix Heide liked this🏭 🌱 Unternehmen sind wie Pflanzen. Mein Artikel aus der NZZaS vom letzten Sonntag
-
Felix Heide liked thisFelix Heide liked thisFelix Heide gave a fantastic keynote talk at the #GCPRVMV 2026 on optical nanosurfaces for imaging and optical computing, as well as on the design of application specific micro-cameras. Moreover, he explained and showed long-time experiments of his current approach to autonomous truck driving. #GCPR26 #VMV26 Center for Sensor Systems
-
Felix Heide liked thisFelix Heide liked thisCelebrating 6 Months at Torc Robotics! Over the past six months, I’ve had the opportunity to learn more about autonomous vehicle technology, develop my skills, and grow a lot more confident in my work. It has been really rewarding to look back and see how much I’ve learned since my first day. I’m grateful for the amazing team I get to work with, and I’m excited to continue learning and growing with the team. Here’s to the next 6 months at TORC! 🚀 #WorkAnniversary #CareerGrowth #AutonomousVehicles #Robotics #ProfessionalDevelopment #Teamwork #LearningAndGrowing #TORCRobotics
-
Felix Heide liked thisFelix Heide liked thisToday marks an important milestone for DTCP: we are very proud that DTCP Defence has successfully completed the first closing of its dedicated 𝗗𝗲𝗳𝗲𝗻𝗰𝗲 𝗮𝗻𝗱 𝗥𝗲𝘀𝗶𝗹𝗶𝗲𝗻𝗰𝗲 𝗙𝘂𝗻𝗱, 𝘀𝗲𝗰𝘂𝗿𝗶𝗻𝗴 €𝟰𝟱𝟱 𝗺𝗶𝗹𝗹𝗶𝗼𝗻 in commitments and establishing a specialist investment team. With the support of Deutsche Telekom, Porsche Automobil Holding SE, EIFO, Danica, SmartCap and other investors, the fund brings together long-term European capital to invest in technology companies strengthening Europe’s defence, security and resilience infrastructure. "𝘗𝘳𝘪𝘷𝘢𝘵𝘦 𝘤𝘢𝘱𝘪𝘵𝘢𝘭 𝘩𝘢𝘴 𝘢 𝘤𝘳𝘪𝘵𝘪𝘤𝘢𝘭 𝘳𝘰𝘭𝘦 𝘵𝘰 𝘱𝘭𝘢𝘺 𝘪𝘯 𝘴𝘵𝘳𝘦𝘯𝘨𝘵𝘩𝘦𝘯𝘪𝘯𝘨 𝘌𝘶𝘳𝘰𝘱𝘦’𝘴 𝘥𝘦𝘧𝘦𝘯𝘤𝘦 𝘳𝘦𝘢𝘥𝘪𝘯𝘦𝘴𝘴. 𝘞𝘦 𝘯𝘦𝘦𝘥 𝘵𝘰 𝘴𝘩𝘰𝘳𝘵𝘦𝘯 𝘪𝘯𝘯𝘰𝘷𝘢𝘵𝘪𝘰𝘯 𝘤𝘺𝘤𝘭𝘦𝘴, 𝘥𝘦-𝘳𝘪𝘴𝘬 𝘮𝘪𝘴𝘴𝘪𝘰𝘯-𝘤𝘳𝘪𝘵𝘪𝘤𝘢𝘭 𝘵𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘺 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯𝘴 𝘢𝘯𝘥 𝘩𝘦𝘭𝘱 𝘣𝘶𝘪𝘭𝘥 𝘦𝘯𝘵𝘦𝘳𝘱𝘳𝘪𝘴𝘦-𝘨𝘳𝘢𝘥𝘦 𝘥𝘦𝘧𝘦𝘯𝘤𝘦 𝘢𝘯𝘥 𝘳𝘦𝘴𝘪𝘭𝘪𝘦𝘯𝘤𝘦 𝘵𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘪𝘦𝘴 𝘢𝘵 𝘴𝘤𝘢𝘭𝘦. 𝘞𝘪𝘵𝘩 𝘰𝘶𝘳 𝘯𝘦𝘸 𝘥𝘦𝘥𝘪𝘤𝘢𝘵𝘦𝘥 𝘋𝘦𝘧𝘦𝘯𝘤𝘦 𝘍𝘶𝘯𝘥 𝘢𝘯𝘥 𝘴𝘱𝘦𝘤𝘪𝘢𝘭𝘪𝘴𝘵 𝘪𝘯𝘷𝘦𝘴𝘵𝘮𝘦𝘯𝘵 𝘵𝘦𝘢𝘮, 𝘸𝘦 𝘢𝘳𝘦 𝘱𝘳𝘰𝘶𝘥 𝘵𝘰 𝘤𝘰𝘯𝘵𝘳𝘪𝘣𝘶𝘵𝘦 𝘵𝘰 𝘰𝘯𝘦 𝘰𝘧 𝘌𝘶𝘳𝘰𝘱𝘦’𝘴 𝘮𝘰𝘴𝘵 𝘪𝘮𝘱𝘰𝘳𝘵𝘢𝘯𝘵 𝘤𝘩𝘢𝘭𝘭𝘦𝘯𝘨𝘦𝘴: 𝘴𝘵𝘳𝘦𝘯𝘨𝘵𝘩𝘦𝘯𝘪𝘯𝘨 𝘰𝘶𝘳 𝘪𝘯𝘥𝘶𝘴𝘵𝘳𝘪𝘢𝘭 𝘢𝘯𝘥 𝘵𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘪𝘤𝘢𝘭 𝘣𝘢𝘴𝘦 𝘢𝘯𝘥 𝘢𝘤𝘤𝘦𝘭𝘦𝘳𝘢𝘵𝘪𝘯𝘨 𝘥𝘦𝘧𝘦𝘯𝘤𝘦 𝘳𝘦𝘢𝘥𝘪𝘯𝘦𝘴𝘴,” says our Managing Partner & Chief Investment Officer of DTCP Growth, Thomas Preuss The fund has already completed its first investments in Six Robotics and Kraken Technology Group, with a third investment underway. The dedicated DTCP Defence investment team focuses on mission-critical technologies across AI, autonomous systems, cyber defence, secure communications and space technologies. Read more: https://lnkd.in/eVk-eRGb #DefenceTechnology #Security #EuropeanTechnology #GrowthEquity #DTCP Thomas Preuss, Ole Aguirre, Georgia Watson, Claudius Laskawy, Joel-Oskar Raisanen, Jonas Byok, Filippo Vidoni, Vicente Vento
-
Felix Heide liked thisFelix Heide liked thisTwo posters, two pretty great places to talk about our work. This summer I had the chance to present our work on the TruckDrive dataset at CVPR 2026 in Denver, and then bring it to the ICVSS Summer School in Sicily together with our automatic labeling approach for long-range autonomous driving. Really enjoyed the conversations, feedback, and meeting people working on similar problems; and Sicily wasn’t a bad place for a poster session either. 🤗 Truckdrive Dataset: https://lnkd.in/e4Hfu3JQ 📖 UniLiPs: https://lnkd.in/eMzzcgH8
Experience & Education
Publications
-
FlexISP: A Flexible Camera Image Processing Framework
SIGGRAPH Asia 2014
Conventional pipelines for capturing, displaying, and storing images are usually defined as a series of cascaded modules, each responsible for addressing a particular problem. While this divide-and-conquer approach offers many benefits, it also introduces a cumulative error, as each step in the pipeline only considers the output of the previous step, not the original sensor data. We propose an end-to-end system that is aware of the camera and image model, enforces natural-image priors, while…
Conventional pipelines for capturing, displaying, and storing images are usually defined as a series of cascaded modules, each responsible for addressing a particular problem. While this divide-and-conquer approach offers many benefits, it also introduces a cumulative error, as each step in the pipeline only considers the output of the previous step, not the original sensor data. We propose an end-to-end system that is aware of the camera and image model, enforces natural-image priors, while jointly accounting for common image processing steps like demosaicking, denoising, deconvolution, and so forth, all directly in a given output representation (e.g., YUV, DCT). Our system is flexible and we demonstrate it on regular Bayer images as well as images from custom sensors. In all cases, we achieve large improvements in image quality and signal reconstruction compared to state-of-the-art techniques. Finally, we show that our approach is capable of very efficiently handling high-resolution images, making even mobile implementations feasible.
Other authorsSee publication
Honors & Awards
-
NVIDIA Graduate Fellowship
NVIDIA
One of 5 fellowship holders. Funding in the amount of $25,000.
https://research.nvidia.com/content/2015-grad-fellows
Other similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content