Ronghui Gu

Explore how the structure of AI and ML ecosystems shapes the distribution of value and risk, and determines system-level properties like safety and fairness, at Monday’s Computer Science and DSI joint colloquium! Featuring Benjamin Laufer, PhD candidate at Cornell Tech https://lnkd.in/gYuAW2Nd

16

1 Comment

🏥 After the COVID-era surge pushed its infrastructure to its limits, Carnegie Mellon University's Delphi Group, co-founded by ML professors Ryan Tibshirani and Roni Rosenfeld, re-architected its ML and data infrastructure with Astro — shifting from crisis-built systems to scalable, crisis-ready public health infrastructure. Delphi has played a leading role in shaping epidemic forecasting technology in the United States. The group has been recognized by the Centers for Disease Control and Prevention as a National Center for Epidemic Forecasting and is a perennial winner in CDC forecasting challenges. Learn how they moved from a fragile, custom Python orchestration system to Astro and achieved a 90%+ reduction in data latency, 3x faster ML training pipelines, and 10x faster backfill run times. Link below.

27

4 Comments

Excited to share insights from a deep-dive talk by Chenyang Zhao on a critical problem: efficient, large-scale Reinforcement Learning (RL)! RL isn't just about algorithms; it's a massive systems challenge. Chenyang detailed the complex, two-stage pipeline of training and inference, where frequent "context switching" creates major bottlenecks. Here are some of the key efficiency challenges and clever solutions discussed: Challenge 1: The 'Long-Tail' Effect in Multi-Turn RL. In complex, interactive rollouts, the system spends a huge amount of time (often >50%!) just waiting for the slowest 20% of requests to finish. Solution: Partial Roll-out (APRIL). A smart scheduling fix. Instead of waiting, set a threshold for finished requests, cache the unfinished "long-tail" requests in a buffer, and prioritize them in the next round. This gives a significant throughput boost. 📈 Challenge 2: Training-Inference Mismatch. Using different engines for training (e.g., Megatron) and inference leads to different log probabilities due to "non-associative float point arithmetic." This introduces policy bias and can even cause training collapse! Solution: Importance Sampling. An elegant algorithmic fix to correct this distribution shift, treating the inference engine as the behavior policy and the training engine as the target policy. Challenge 3: Slow Context Switching. When placing training and inference on the same GPU, you must offload and reload models. The biggest bottleneck? Rebuilding the CUDA graph, which takes a long time. Solution: CUDA Graph-Aware Model Wake-up. A fantastic system trick. They use a "torch memory saver" to "pause" memory—releasing the physical memory while preserving the virtual memory address. This keeps the CUDA graph intact for a near-instant wake-up. Challenge 4: Model 'Refit' Latency. The "refit" process (copying new weights from the trainer to the inferencer) is slow. Solution: System Co-design. They optimized this from 60 seconds down to 7 seconds using a pipeline of zero-copy CUDA IPC, tensor bucketing (reducing HTTPS overhead), and tensor flatting (reducing IPC handle creation). This is fantastic work bridging the gap between complex RL algorithms and high-performance systems. Solving these efficiency challenges is key to making large-scale RL practical and scalable. Great talk, Chenyang! You can watch the full talk here: https://lnkd.in/gMy4qf8S #AI #ReinforcementLearning #SystemsForAI #EfficientAI #DeepLearning #LLM #GPU #Optimization

16

Last week, we hosted NYU Tandon School of Engineering alum Dr. Yu Hu for his webinar 'AI Agent 101 and Vibe Coding Demo.' Dr. Hu, now a Senior Software Engineer at Comcast, introduced the core concepts behind modern AI agents—from LLMs and memory to tool integration, reusable agent skills, and autonomous workflows. He also included a live “vibe coding” demo to show how AI agents can orchestrate tools and skills to accelerate development, automate data workflows, and support transportation analytics and decision-making. Watch the webinar and follow along with the demo here: https://lnkd.in/eqaUMBtq

3

Check out "Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering," by Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, Xin Xia in the 2025 featured article of Proceedings of the ACM on Software Engineering (PACMSE): https://lnkd.in/eVpKanqF

6

𝑴𝒊𝒏𝑲𝒚𝒆𝒐𝒎 𝑲𝒊𝒎 𝑾𝒊𝒏𝒔 𝟐𝟎𝟐𝟓 𝑰𝒏𝒕𝒆𝒓𝒏𝒂𝒕𝒊𝒐𝒏𝒂𝒍 𝑸𝒖𝒂𝒏𝒕 𝑪𝒉𝒂𝒎𝒑𝒊𝒐𝒏 MinKyeom Kim, an undergraduate student in the Department of Industrial Engineering at UNIST, has made history as the first Korean to win the prestigious International Quant Championship (IQC), a global competition hosted by the leading quantitative asset management firm, WorldQuant. The victory underscores Korea’s rising prominence in the field of data-driven finance and quantitative research. 🔗Read more: https://lnkd.in/gnXDwrk2 #ThisisUNIST #GlobalChampion #IQC #WorldQuant #DataScience

236

5 Comments

At IslandoraCon 2025, our own Akanksha Singh shared how institutions can leverage Islandora’s flexibility, workflow tools, and standards-based architecture to build stronger, more scalable institutional repositories (IRs). From supporting ETDs and research outputs to enabling custom submission workflows and metadata-driven discovery, Islandora continues to stand out as a powerful, open-source IR platform built for academic libraries. In our latest post, we recap the highlights from her session: 1. Why institutions choose Islandora for their IR platforms 2. How customizable editorial workflows streamline ETD review and approvals 3. Persistent Identifiers minted automatically at publication 4. Community feedback and input If you’re exploring IR modernization or considering alternatives to legacy systems, this is a great place to start. https://lnkd.in/edR-ZWXt PS. You can even view her full slide presentation in the link. #Islandora #InstitutionalRepository #OpenSource #Drupal #DigitalRepositories

13