RJ Honicky

Enterprise AI has a equipment problem, not a compute problem. I contributed to a new 3-part series on the Intel community blog: "From Gold Rush to Factory: How to Think About TCO for Enterprise AI." The core idea: most organizations are still in gold-rush mode — buying GPUs before understanding the work. But the enterprises getting real ROI from AI are thinking like factory operators. Match the equipment to the job. Build repeatable workflows. Optimize the CPU-to-GPU ratio by workload, not by hype. One story from the piece that still gets me: a genomics company deployed high-end GPUs to meet a seven-day SLA. Got results in under a minute. Impressive — until someone asked if a two-hour completion on CPUs would suffice. It did. The GPUs were decommissioned. That's not an edge case. It's closer to the norm than most want to admit. The constraint isn't compute. It's the lack of a system for matching the job to the right equipment. Part 1 is live now. Parts 2 and 3 are coming. https://lnkd.in/gQX_pe_2 cc: Rahul Awasthy Lynn Comp

6

1 Comment

Today's article in my AI column – at Google Bay View, a panel discussed wafer-scale chips, data center and fabrication bottlenecks, costly movement of model weights, and the risk of overheated investment. Despite those challenges, the panelists expect AI infrastructure to underpin economic change. https://lnkd.in/g9CNCasG

38

1 Comment

OpenAI’s OSS-120B is the premier open-source model. How many of the five fastest inference providers serve it on GPUs?   One. And they’re in fifth place.   Performance data (OpenRouter, 10/21/25):   - Cerebras: 2,838 tokens/sec per user - Groq: 915 tokens/sec per user - SambaNova: 851 tokens/sec per user - Google Vertex: 516 tokens/sec per user - Fireworks: 494 tokens/sec per user (GPU-based)   Speed tells the story. The fastest inference in the world doesn't run on GPUs.   Check out the uptime data as well. Cerebras and Google get perfect uptime scores.

130

4 Comments

MLCommons unveiled a new benchmark that tests how well storage vendors handle KV cache and vector database workloads, two important components of running AI inference at scale. https://ow.ly/mN1s50ZLApL

6

 Executive Thesis — Earth as the Ultimate Compute Substrate AI compute demand is rising at a rate that outpaces the physical limits of surface-level data centers. Thermal density, grid constraints, water scarcity, and land-use conflicts are converging into a single bottleneck: the inability to dissipate heat at the rate AI generates it. Some propose escaping Earth’s constraints entirely — most notably Elon Musk’s long-term vision of orbital compute platforms. While elegant in theory, orbital compute is economically and operationally non-viable for at least the next half-century. https://lnkd.in/eHgNB6cZ #SupercriticalCO2 #sCO2 #ThermalManagement #DataCenterCooling #AdvancedCooling #SubsurfaceDataCenters #UndergroundDataCenters #AIInfrastructure #ComputeInfrastructure #HighPerformanceComputing #HPC #EnergyEfficiency #SustainableInfrastructure #DigitalInfrastructure #DataCenterInnovation #InfrastructureDesign #PatentPending #DeepTech #ClimateTech #NextGenInfrastructure #EngineeringInnovation #SystemsArchitecture #FutureOfCompute #ResilientInfrastructure #TrustedInfrastructure

5

3 Comments

NVIDIA-Groq $20B is a thought provoking deal – but where does the real value lives? The news around NVIDIA acquiring Groq is worth pausing on, not just as an impressive M&A headline, but as a signal of where real value in AI infrastructure is being created. First, kudos to Jonathan Ross and the entire Groq team. For me, beyond the hardware story that is debatable in my opinion, what Groq truly delivered was a robust, production-grade inference deployment and serving stack. That’s the hard part many underestimated and still do. Shipping impressive latency benchmarks is one thing; operationalizing newly announced models at scale, reliably and fast, is a different game. A lot of companies over the past years explored SRAM-based architectures, in-memory compute, and other novel XPU architectures. Far fewer succeeded in building a repeatable end-to-end inference deployment stack that deliver fast transition from model arrival to scaled production. Groq did. And it’s not surprising that NVIDIA recognized that value. When we started NeuReality, right from the first customers’ tour we concluded that software stack and AI serving is where we should start from since efficient compute is the goal but software is the mean to deliver it to customers with Ease-of-use and TTM. So we started with defining our Inference Serving Stack and our Scale-Out Networking Stack. Where our paths diverge is in philosophy. Groq chose full vertical integration, eliminating supply chain intermediaries, building the hardware and the software, and selling tokens directly to end users while we took a different approach of productizing the inference software and hardware infrastructure, making it work across any GPU or XPU, and enabling others - hyperscalers, silicon vendors, Neoclouds, and enterprises to unlock that same deployment velocity with their choice of XPU without locks. Different strategies. Same technology focus and realization. Designing silicon is impressive. Turning it into a real, deployable inference platform is rare. That’s the hard part -- and that’s where durable value lives. https://lnkd.in/dy6_89dA

89

4 Comments

Zero plus zero equals two. Not in some toy project. In Berkeley SoftFloat -- the IEEE 754 math library reference implementation. It's inside QEMU and most x86 emulators, and it's the oracle that hardware teams verify chip designs against. If you've ever emulated a CPU or validated a chip design in the last decade, Berkeley SoftFloat did the floating-point math. I found seven wrong results across five of six arithmetic operations in the 80-bit extended precision format. The one Intel invented for the x87 FPU in 1980. Some other fun highlights: - 0.5 + 0.5 = 3 - A huge finite number plus zero = infinity - Infinity x 0 = infinity (and no error raised) - Two tiny numbers added together = zero These aren't rounding errors. These are completely wrong answers for valid inputs. The root cause: the x87's 80-bit format has an explicit "integer bit" that every other IEEE format hides. This creates encodings -- unnormals, pseudo-denormals, pseudo-infinities -- where the bit says one thing and the exponent says another. The original 8087 handled all of them correctly. SoftFloat hasn't since its 2011 rewrite. Nobody noticed because the test suite has a structural blind spot. The test generator only produces the encodings that SoftFloat itself would output... the "nice" ones where the integer bit is consistent with the exponent. It never generates the inputs that trigger the bugs. I patched the generator to cover the full input space and failures lit up everywhere. I never would have found any of this if I hadn't been writing my own floating-point library from scratch. When my results disagreed with SoftFloat, I assumed I was wrong. Over and over. I'd go back to my code, recheck my math, trace through my logic... because the reference implementation couldn't possibly be wrong. That's what "reference" means. But the reference was wrong, and I wasn't, and suddenly... Suddenly I was very sad. SoftFloat is supposed to be "the thing that is correct." It's the ultimate tech industry oracle, the final reference on one plus one. TestFloat tests hardware against SoftFloat. FPGA developers validate against SoftFloat. When your personal deity lies, when addition and subtraction themselves dissemble, the epistemological foundation shifts under you. You can't trust the thing you trusted, and now you have to ask what else you can't trust. What makes my situation lonelier is that finding the bug doesn't feel like a win, because it shouldn't have been there in the first place. I wasn't looking for SoftFloat bugs. No one gives you an award for breaking addition and subtraction. I was trying to validate my own work and the ground moved, and now I just feel like I'm waiting for the next earthquake. https://lnkd.in/g67ZCkMu https://lnkd.in/gG7DC-26 https://lnkd.in/g9rJs6ej https://lnkd.in/gV28f9vp https://lnkd.in/g3EPE_wW

7

3 Comments

AI Inference Leaps with Cerebras-AWS; World Models Surge in Funding. Frontier Hardware and Model Innovations Reshape AI Landscape At a glance: AWS announced on March 16, 2026, the deployment of Cerebras CS-3 systems within AWS Bedrock, delivering the industry's fastest AI inference using open-source large language models alongside Amazon’s proprietary Nova models. This partnership introduces a disaggregated architecture that pairs AWS Trainium chips for the prefill phase of AI generation with Cerebras' Wafer-Scale Engine for the decode phase, achieving a fivefold increase in token throughput. The setup optimizes high-speed inference by assigning specialized hardware to distinct computational stages, enabling enterprises in logistics and finance to process complex queries at unprecedented speeds without compromising accuracy. Read full brief: https://lnkd.in/gSRyC9i6

1