Zeyu Ning

Happy to share that our paper "Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX" has been accepted to ICLR 2026 🇧🇷! Octax is a high-performance JAX-based framework that transforms CHIP-8 arcade games into fully vectorized RL environments. CHIP-8, a cousin to Atari 2600, offers a simpler yet rich set of arcade games that are nice for RL benchmarking. We have implemented a complete CHIP-8 emulator entirely in JAX that can load ROM files directly, creating the first end-to-end GPU-accelerated alternative to traditional CPU-bound arcade environments like Atari. Why it matters: - We achieve a 14x speed improvement over Envpool-accelerated Atari environments (the current standard), with orders-of-magnitude gains over traditional CPU emulators. This democratizes RL research on arcade games: researchers with low-end GPUs or small clusters can now run experiments in reasonable timeframes and conduct statistically rigorous evaluations with many more seeds than the traditional 5-seed standard, improving research reliability. - Unlike existing solutions that remain CPU-bound, Octax runs entirely on GPU, eliminating the computational bottleneck that has limited large-scale RL experimentation with image-based environments. - Extensible and modular design: The framework makes it easy to add new games or modify existing ones, providing flexibility for custom research needs. - As a proof-of-concept, we demonstrate that LLMs can generate entirely new, playable CHIP-8 games directly from prompts. This capability paves the way for exciting research directions in open-ended learning, procedural content generation, and automatic curriculum design. We are currently polishing the camera-ready version and developing comprehensive documentation to make Octax accessible to the broader RL community. Our goal is to provide the JAX ecosystem with a powerful, scalable tool for large-scale RL research. Huge thanks to my brilliant co-authors Thomas Michel and Hector Piteau for making this collaboration such a rewarding experience! 📄 Paper: https://lnkd.in/eYDz8M7Y 💻 GitHub: https://lnkd.in/eXKUpfd8 #Jax #ICLR2026 #OpenSource

150

6 Comments

Meta just shared their blueprint for high-performance recommendation inference in PyTorch. They walk through the full workflow from trained model to production serving: graph capture, splitting, fusion, quantization, compilation, plus a C++ runtime for high QPS. Key optimizations: • GPU inference with model splitting • Distributed patterns (CPU-GPU, embedding-dense) • Request coalescing + table batched embeddings • Quantization (bf16/int8 down to int4 for embeddings) • TorchRec primitives for massive sharded embeddings (the sparsity backbone) All powering Meta’s Feed, Ads, Reels, etc., on a real-world scale. If you're working on large-scale personalized models (healthcare or otherwise), this is a solid reference for squeezing more performance out of PyTorch. Anyone already using TorchRec or similar for rec systems? Full post: https://lnkd.in/gf6bezBP #PyTorch #RecommendationSystems #AIInference #MachineLearning

1

Are we limiting LLM potential by sticking to old alignment methods? This new paper, "Inverse Reinforcement Learning Meets LLM Post-Training: Basics, Advances, and Opportunities" argues and offers a powerful alternative. While many teams lean on imitation fine-tuning and preference PPO, this show how this path often stalls due to the lack of a shared underlying reward. Inverse Reinforcement Learning (IRL) flips the script entirely. Instead of pushing a policy, IRL watches expert traces or pairwise votes to learn a neural function that scores any output. This learned reward signal is a game-changer. The bottom line is clear: Learn the reward, not just the policy. This fundamental shift is critical for building more robust, aligned, and capable Large Language Models. Dive into the paper to explore the basics, advances, and exciting opportunities IRL presents for LLM post-training. #LLMs #AIAlignment #InverseReinforcementLearning #MachineLearning #DeepLearning #AIResearch #ReinforcementLearning #FutureOfAI

6

1 Comment

I’m happy to share our new paper, “KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning” now available on arXiv: https://lnkd.in/eYkRTRfJ   This work builds on the LeJEPA framework by reinterpreting its regularization objective through the lens of kernel discrepancies. Doing so allows us to place LeJEPA within a broader family of self-supervised methods and to design more flexible objectives beyond Gaussian assumptions.   Key contributions: •  We show how LeJEPA can be viewed as a special case of a more general kernel discrepancy framework. •  We introduce KerJEPAs, which allow for richer regularization via alternative kernels. •  We derive closed-form high-dimensional limits for sliced MMD / KSD, providing both theoretical insight and practical guidance. This project was a collaboration with Harley Wiltzer, Justin Szeto, David Alvarez Melis, and Lester Mackey. I’m grateful for the many discussions that shaped this work. Happy to discuss with anyone interested in representation learning, self-supervision, or kernel methods.

174

8 Comments

Why Are Modern LLMs Mostly Decoder-Only? If you zoom out, Transformer models come in three main flavors: Encoder (BERT-style) Reads the full input bidirectionally excellent at understanding text. Used for classification, embeddings, retrieval, and NLI. Decoder (GPT-style) Generates text token by token (autoregressive). Optimized for generation, reasoning, and long-form outputs. Encoder–Decoder (T5 / BART) Encoder understands the input, decoder generates conditioned output. Ideal for sequence-to-sequence tasks like translation and summarization. So why did decoder-only models dominate? • One unified objective Everything is just next-token prediction simple, scalable, elegant. • Scaling works beautifully As models grow, reasoning, planning, and multi-step thinking emerge naturally. • Prompting becomes the interface Few-shot learning, chain-of-thought, tool use — all via plain text, no architectural changes. • Infrastructure wins Autoregressive pipelines are heavily optimized for GPUs, training, and serving. • Multimodal flexibility Text, code, images, audio → treated as tokens in one sequence. In short, decoder-only models are universal text continuation engines and that turns out to be the most powerful abstraction we’ve found so far.

9

3 Comments

Read Chaim Rand's practical walkthrough for optimizing PyTorch inference. Learn about batched inference, memory formats, IPEX, and graph compilation on Intel Corporation Xeon.

5