Priunsh Syen

Why are Gaussian distributions so normal? Gaussian distributions show up everywhere in science - measurement errors, experimental noise, and biological variation. They show up so often that they’re also called normal distributions. What’s most surprising to me is that they even appear when measuring something that is itself NOT normally distributed. Let’s imagine that you count the number of machine malfunctions in a year. That distribution will likely be Poisson. But if you track the number of malfunctions over many years and plot the average number of malfunctions per year, you will approach a normal distribution. Or, more formally: if yearly malfunctions are independent Poisson random variables, then as you take averages over an increasing number of years, the distribution of those sample means converges toward a normal distribution. And it’s the same in chemistry. A single reaction run is rarely well-behaved. Yields jump, impurities appear or disappear, and “identical” experiments don’t result in the same outcome. Errors could be distributed in any number of ways. Yet, when you plot the means, a Gaussian distribution appears. It’s also why replicate experiments are essential in high-throughput work. Errors don’t arrive labeled. They accumulate quietly across weighing, dosing, temperature control, mixing, and workup. Only repetition lets you tell chemistry apart from noise.

40

1 Comment

Why are medical imaging teams switching to MONAI over custom pipelines? What went wrong with old approaches: • Reinventing data loaders for DICOM/NIfTI every project • Inconsistent augmentations across teams • No built-in support for multi-modal or 3D • Debugging 3D convolutions from scratch What MONAI got right: • Ready-made loaders for medical formats • Standardized, reproducible augmentations • Built-in 3D/ multi-modal support • Bundles for common tasks (segmentation, classification) Great tools win when they remove boilerplate and let you focus on the model. In clinical AI, that means faster, more reproducible research. If you want to try it: https:https://lnkd.in/gSbV3kNp #PyTorch #MONAI #MedicalImaging #HealthcareAI

3

𝐓𝐡𝐞 𝐁𝐫𝐞𝐚𝐤𝐭𝐡𝐫𝐨𝐮𝐠𝐡 𝐢𝐧 𝐒𝐞𝐥𝐟-𝐃𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧 Current RL methods for LLMs often rely on a simple binary signal: Success or Failure. But in complex domains like math and coding, a "No" doesn't tell the model why it failed. This creates a massive credit-assignment bottleneck. At SciPulse, we just finished a deep dive into a revolutionary approach: Self-Distillation Policy Optimization (SDPO). The Core Shift: Instead of a simple scalar reward, SDPO leverages Rich Textual Feedback (like compiler errors or judge evaluations). It essentially teaches the model to become its own tutor, distilling its ability to retrospectively identify mistakes back into its policy. Why this matters for the industry: - 3x Fewer Attempts: Achieving the same discovery probability as best-of-k sampling with significantly less compute. - Sample Efficiency: Outperforms standard RLVR baselines across scientific reasoning and competitive programming (LiveCodeBench v6). - Beyond Code: The methodology suggests a future where AI can learn from any "rich" environment without needing an external teacher or explicit reward model. The transition from binary "right/wrong" signals to dense, feedback-driven learning is a fundamental step toward more autonomous, reasoning-capable AI. 🎥 𝐏𝐨𝐝𝐜𝐚𝐬𝐭: https://lnkd.in/eU9gXnKK 🎥 𝐕𝐢𝐝𝐞𝐨: https://lnkd.in/e8eR4FSP 📄 𝐎𝐫𝐢𝐠𝐢𝐧𝐚𝐥 𝐏𝐚𝐩𝐞𝐫: https://lnkd.in/eubMwsfp #AIResearch #MachineLearning #ReinforcementLearning #LLMs #ComputerScience #SelfDistillation #SciPulse #TechInnovation

We’re excited to welcome Mooncake to the PyTorch Ecosystem! Mooncake is designed to solve the “memory wall” in LLM serving. By integrating Mooncake’s high performance KVCache transfer and storage capabilities with PyTorch native inference engines like SGLang, vLLM, and TensorRT-LLM, it unlocks new levels of throughput and scalability for large language model deployments. Mooncake enables prefill decode disaggregation, global KVCache reuse, elastic expert parallelism, and serves as a fault-tolerant PyTorch distributed backend. 🔗 https://lnkd.in/g3Dm4mN6 #PyTorch #OpenSourceAI #LLM #AIInfrastructure

101

I was anyway deep into Quantum Mechanics by now (my PhD topic was going to be homogenization for nonlinear optics) so good time to pivot to non-classical computations. Q) use RoPE on a Bloch sphere to get a Quantum computer to perform next token prediction A) remember “show me the prompt, or it ain’t real” import numpy as np from qiskit import QuantumCircuit from qiskit.quantum_info import Statevector from qiskit.visualization import plot_bloch_multigraph # --- 1. Token Vocabulary Mapping --- # Let's map a tiny vocabulary to coordinates on the Bloch sphere: # |0> (North Pole) = "AI" # |1> (South Pole) = "Quantum" # |+> (Equator, X-axis) = "Computers" # --- 2. Initialize Prompt State --- # Start with the token "AI" represented by the state |0> qc = QuantumCircuit(1) # --- 3. Context Processing (Transition Model) --- # Suppose our trained "quantum model weights" dictate that: # Context: "AI" -> Next Token Transition: 70% "Quantum", 30% "AI" # We calculate the required theta angle for this probability: # Prob(|1>) = sin(theta/2)^2 = 0.70 -> theta = 2 * arcsin(sqrt(0.70)) theta_transition = 2 * np.arcsin(np.sqrt(0.70)) # Apply parametric Ry rotation gate around the Y-axis to shift the vector qc.ry(theta_transition, 0) # --- 4. Get Final State Representation --- # Extract the final state vector representing the next-token distribution next_token_state = Statevector.from_instruction(qc) print("--- Next Token Quantum Representation ---") print(f"State Vector: {next_token_state.data}") print(f"Probability of 'AI' (|0>): {np.abs(next_token_state.data[0])**2:.2f}") print(f"Probability of 'Quantum' (|1>): {np.abs(next_token_state.data[1])**2:.2f}") # --- 5. Add Measurement for Sampling --- # In a real run, this collapses the sphere to pick a concrete token qc.measure_all()

12

Shoutout to Keenan Johnson for all the work he's put in helping us . . . thanks to Keenan, we have a fully functional auto-encoder able to process raw ROV survey imagery at scale (the auto-encoder is linked below, as is the Seattle Aquarium repo where we're tracking project development in the Issues). It's been amazing to see this work unfold, and as it's all open-source, other researchers / entities with similar imagery can make use of Keenan's framework to (1) train their own auto-encoder with raw and processed image pairs, then (2) apply the model to process raw imagery at scale. Having seen that accessing relatively inexpensive, robust, and consistent compute is challenging for researchers at small organizations, he's spinning up the GPU hardware necessary to easily enable access to high-performance hardware . . . yay for helping out the smaller folks out there wanting to dabble in the high-performance compute space! https://lnkd.in/ene6cXKT https://lnkd.in/eXsmfFNK

29

9 Comments

Copied verbatim from Karpathy’s x post. I agree 100 % as the AI coding tools are so amazing. Cc Philip Bettenburg who is also a huge advocate of the below. “ I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind. “

10

Most of the breakthroughs in modern LLMs came from rethinking "Attention" computations and optimizing it. Vanilla self-attention scales quadratically with sequence length. That’s fine at small context sizes, but it becomes the main bottleneck as models push toward 32k, 128k, and beyond. In Part 2 of my Modern LLM Architecture series, I break down how today’s models scale attention efficiently: • Why Multi-Query (MQA) and Grouped-Query Attention (GQA) drastically reduce KV cache memory • How Multi-Head Latent Attention (MLA) compresses keys and values without losing capacity • How Sliding Window Attention trades global context for predictable efficiency • How DeepSeek Sparse Attention (DSA) works • How Gated SDPA and Gated DeltaNet improve stability and enable linear or near-linear scaling These mechanisms are the reason modern LLMs can serve long contexts without exploding memory or latency. Link to the post: https://lnkd.in/e9jCsHc9 Part 3 will cover MoE, Low-level Hardware related KV cache Reduction, and Muon Optimizer.

17

1 Comment