About
Engineering leader with 20+ years building software and 12+ in management — across…
Articles by Ben (Xiaojun)
Activity
35K followers
-
Ben (Xiaojun) Li shared thisOpenAI recently reported daily inference usage in its research organization exceeding $600 at the median and $7,000 at the 90th percentile, valued at API prices. For most people working within tighter budgets, choosing the right model for the right task can be a struggle. For real work, what matters is not the cost per LLM token, but the overall cost of successfully completing a task: including retries, verification, and human correction time. It’s a bit counterintuitive, but more advanced models can be cheaper overall for complex agentic tasks. The Terminal-Bench 4.0 leaderboard (https://www.tbench.ai/) shows that Astra (GPT-6, running in #Codex at max effort) has the highest reported resolution rate, with substantially lower token usage and total cost than Fable 5.1 and Opus 5. This closely matches my recent experience working with Astra, Opus 5, and GPT-5.6 Sol. Terminal-Bench measures how effectively AI agents perform complex, multi-step tasks through a real command-line interface (CLI). The benchmark covers software engineering, machine learning, systems operations, security, media, and scientific research. If your tasks resemble those in Terminal-Bench, my recommendation is to start with Astra and/or Fable 5.1 (or Opus 5 if your organization doesn’t allow Fable). For basic automation tasks, lower-cost models can make sense, provided they reliably meet your quality requirements.
-
Ben (Xiaojun) Li shared this▎ Ten years ago, in Apple's chip department, we were designing A-series chips two generations ahead of what customers had in their newest iPhones. We could predict the future iPhone, but we could not simply make it arrive any faster. ▎ Fast forward to now. Researchers at OpenAI and Anthropic are building models ahead of anything the rest of us can access, and they are using those models to build the next ones. That lead does not stay fixed. It compounds. ▎ OpenAI published their competitive advantages yesterday. By mid-August, the median researcher in its research organization was spending more than $600 a day on inference at API prices, and the 90th-percentile user more than $7,000 a day. The research org now spends 3.1 agent-workdays of effort for every workday of human labor. Most companies cannot fund that, let alone match it, so the frontier gap is more likely to widen than to close in near future. ▎ OpenAI's chief scientist Jakub Pachocki published an essay yesterday on what he sees in the internal models and what it costs him sleep: #RSI (recursive self-improvement) and automated alignment research. He says "This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence." ▎ Over the next several years, OpenAI will prioritize work in service of three north stars: ▎ Navigating the next period of AI progress, by building an automated AI researcher, iterating with it on the alignment problem and finding ways for people to remain part of the self-improvement loop. ▎ Delivering the benefits of scientific progress and economic growth that very intelligent machines enable. ▎ Empowering everyone individually with a personal AGI. ▎ Read one way, these are about alignment, enablement and empowerment. Read another way, they are about competition, profit and distribution. ▎ Jakub hopes voluntary slowdowns become common until shared safety bars exist. I think achieving that globally will be difficult. With OpenAI targeting an automated AI researcher by March 2028, watching from the sidelines is not an option. Grokking the frontier models and the 3C harness tools (#Claude, #Codex, #Copilot) is how we build the judgment to direct work we increasingly won’t do ourselves. "finding ways for people to remain part of the self-improvement loop" is not something a lab does for us. It is a seat we choose take. https://lnkd.in/gUEzAsSf
-
Ben (Xiaojun) Li shared thisOne year ago, I claimed the single most important benchmark for AGI is a simple proof of the Fermat’s Last Theorem (FLT). For me, true AGI should be able to prove FLT in one page, using elementary number theory, or inventing a new mathematical idea to do it elegantly. At that time, Anthropic was struggling and Claude Code wasn't a big thing, and I excluded them in the AGI race in the AI Slop image in my post. Today, Anthropic made a major advancement in that AGI benchmark: Claude AI Formalizes Fermat’s Last Theorem in Just 11 Days. Anthropic’s #Claude completed a full formalization of FLT in Lean 4 in only 11 days. The project included more than 13 million lines of code and verified about 29,500 intermediate results across areas such as algebra and geometry. The formal proof follows the same broad path used by Andrew Wiles at Princeton University, relying on Frey curves and modularity. Each step was checked against the underlying mathematical axioms, with dozens of AI agents working together on different parts of the proof. The project could mark an important step for automated theorem proving. Large-scale AI systems may help mathematicians convert existing proofs into machine-checkable form, verify complex arguments, and expand formal mathematics libraries much faster. Although this is not reaching the AGI level I wished, it is still a significant step toward the final proof. I am happy to see scientists in Anthropic share the same passion on FLT and hope in another year, there will be the final breakthrough.
-
Ben (Xiaojun) Li shared thisFree half-day workshop in San Francisco by Microsoft and Anthropic experts on Claude and Agent in Microsoft Foundry. This workshop is designed to accelerate your ability to ideate, optimize, and deploy AI solutions in real-world scenarios. Agenda-at-a-glance: - Welcome and expectations for the day - Hands-on labs: Building an Agent & Making Your Agent Work over a Long Horizon - Preconfigured Azure environments - Azure-direct model access and configuration - Best practices for prompting, safety, optimization, and fine-tuning - Scenario-based exercises tailored to real use cases This event is a unique opportunity to gain direct experience with Claude and accelerate your path to production. Attendance is limited—claim your spot now. https://luma.com/p6m4n0g7
-
Ben (Xiaojun) Li shared thisThree years ago, one of the highest achieving engineer on my team completed about 1000 pull requests within a year; last month, Lauren Tan (@poteto on X) at Cursor/SpaceXAI did 1000 PRs within a month using coding agents. Coding agents are so powerful, but why most companies do not see 10x productivity increase? Boris Cherny at Anthropic mapped it out. He lays out five steps in AI adoption based on the number of agents you can orchestrate and manage: Step 0. Gated. Zero agents. Security review and procurement still own the decision, and whatever you build lives on your laptop. Step 1. Assisted. One agent. You pair with it, read every diff, and sit and watch while it works. Most engineers live at this stage now. Step 2. Parallel. Around ten agents, each in its own worktree. You stop writing code and start reviewing six streams of it. Step 3. Supervised autonomy. Around a hundred. Claude kicks off Claude. The maintenance backlog that used to wait for a free afternoon now runs in the background. Step 4. AI-native. A thousand or more. You steer by intent and check exceptions. A quarter-long migration becomes something you start Monday and check on Friday. Anthropic is on step 3. Boris is entering step 4. Getting from 1 to 2, and from 2 to 3, is on you. It will cost billions of tokens, and you will have to live inside Claude Code, Codex, or Copilot CLI every day. You need to build your own CLI environment with lots of MCPs, customized plugins and skills. You build your trust in your agents and do not babysit their work. Step 4 is a different stage. You need deep domain expertise, large job scope, unlimited token budget, and AI native infra at your company. One motivated engineer can't get there alone. I am in the mid of step 3, largely pulled back sometimes due to orchestrating agents through 10 repos for given projects. I am curious which step are you on, and what's the one thing blocking the next? AI Adoption roadmap - https://lnkd.in/gRB7qA_M.
-
Ben (Xiaojun) Li shared thisWith the explosion of open source LLM models, inference engineering becomes the fastest growing domain in AI industry. Many software engineers are learning AI engineering now, but the mistake to avoid is not stop at the LLM APIs. Understanding inference will help develop much better applications and user experiences. Philip Kiely wrote a book about it and made it free with both online interactive course and PDF/ePub versions. It is a good resource for SDEs who are new in AI domain and interested in learning inference engineering.
-
Ben (Xiaojun) Li shared thisAnthropic's ARR jumped 14x in the most recent quarter, and 70% of it came from API charges, and it's on a path to $500 billion ARR by 2027. This is unprecedented growth but not a total surprise. The software market has gone through three revenue models: Software-on-CDROM charged for development cost, Software-in-Cloud charged for hosting cost, and Software-by-Intelligence charges for compute cost. From now on, no token is free and every task costs a fortune. Welcome to the new era! Recently I noticed that my fresh sessions in Claude Code CLI and GitHub Copilot CLI, both set to a 1 million token context window, started compacting after just a couple of turns of prompt and response. That leads to huge API cost. Sometimes a single "hi" as the first prompt in a fresh session cost me about $2. So I looked into what my sessions were actually carrying, and found they constantly held close to 500K of MCP tool data in the context window (I have about 80 MCP servers). These tools should be lazy-loaded on demand, so I spent some time investigating. It turns out there's a CLI setting bug that stops MCP lazy-loading. Coincidentally, Anthropic published a blog post, "Maximizing the value of your Claude Code sessions," with a lot of great suggestions on how to manage and optimize the context window to reduce token waste. The most important ones are (1) minimize startup context, (2) set the right model, (3) keep the KV cache warm, (4) @-mention files, and (5) compact before you pause. The attached PDF has more detail on how to apply these in your own tasks. I still find it very odd that LLM vendors charge for input tokens, and everyone seems to just follow along. Input tokens are the user's effort to tell and steer the LLM on what to do and how to do it. It would be very funny if you spent time collecting evidence and affidavits for your suit, went to talk to your lawyer, and your lawyer charged you for the time you spent gathering it before you walked into his office. Regardless, this is the Software-by-Intelligence era, we're charged for compute, and it's a seller's market, so as customers we don't have much negotiating power. What we can do is get really good at understanding context and agent runtime, so we can minimize token waste.
-
Ben (Xiaojun) Li shared thisJeff Dean left Google to found DiscoveryLoop to pursue AI for Science. According to Pathfounders, Demis Hassabis originally planned to leave together with Jeff, but Google persuaded him to stay. AI for Science becomes possible due to the fast advancement of Recursive self-improvement (RSI) in Agentic LLM. Both Andrej Karpathy (joining Anthropic) and Thinking Machines Lab cofounder Lilian Weng (joining OpenAI) are working on RSI. It looks like RSI will become the next frontier of intelligence. Stanford University has a new class "CS329A Self-Improving AI Agents" taught by Prof. Azalia Mirhoseini and Aakanksha Chowdhery, Ph.D. The course covers the latest self-improvement techniques of LLM agents that can continuously improve themselves through interaction with themselves and the environment. It's a good starting journey for people who are interested in RSI.
-
Ben (Xiaojun) Li shared thisDatabricks published a big post on how they cut their AI token spend by close to 90% — model switching, context reduction, smart routing, etc.. Impressive engineering. But I think it's premature. Here's the hard truth: outside the frontier labs (OpenAI and Anthropic, etc.), maybe only 1% of employees at any company are generating real ROI from the tokens they burn. The rest are spending tokens on simple tasks, not real intelligence. If you're reading this post, you're probably in that 1%. There's a piece going around on X right now called "AI Adoption is a Myth" that puts numbers on it. In a typical enterprise rollout: 5-10% become power users, 20% use it badly, 70% never touch it. Roughly 10% of people burn 90% of the tokens. So we should stop booking employee token use as an expense. It's an investment — part of the training budget for employee re-skilling. Agentic tools are incredibly capable but also wide open, and most people need a long time before they really grasp them and see any return. Which brings me back to Databricks. If a 90% cost cut lands harder on their 1% of real adopters than on the 90% who barely use it, they're not saving money. They're throttling the only cohort of the company that's actually innovating. Honest question: how long do you think it takes before AI actually makes most employees faster?
Ben (Xiaojun) Li commented on a post
3w
Pretty cool! 💖
Ben (Xiaojun) Li replied to a comment
3w
Terrence T. I heard from Dwakesh podcast’s episode with guest Dylan Patel that OpenAI reserved 60% of their compute budget for research and development, not production reference for customers. Kind of crazy.
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisCareer update: after about a year and a half at OpenAI, I’m leaving to build something of my own. I joined the Codex team about a year and a half ago, 2 weeks before we launched the very first incarnation of Codex. Since then, I’ve had the chance to work across a ridiculous range of problems: reliability, product, agents, code review, infrastructure, capacity, safety, and a few things I had absolutely no idea how to do when I started. A year and a half at OpenAI somehow felt like five years, or maybe more, anywhere else. There was just so much to learn, absorb, build, get wrong, and figure out. To capture what the experience actually felt like, I wrote a piece about it, both as a reflection for myself and for founders and builders trying to make sense of this strange moment we’re in, as well as anyone who’s ever wondered what working on Codex at OpenAI actually feels like. Link - https://lnkd.in/gU44Pf2H As for what’s next, I’m joining Aishwarya Naresh Reganti at LevelUp Labs, where we’ll work closely with companies to figure out where AI can meaningfully change how they operate, build the systems that make that possible, and help teams learn to work with them.
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisLast week at OpenAI's annual DevDay, we launched AgentKit to help you build, deploy, and optimize agents. It's been a privilege to work with the dream team to bring AgentKit to life, and I can't wait to see all the agents people build! Watch the full talk here: https://lnkd.in/etxWPKHp
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisChatGPT is now live in Microsoft Teams 🎉 - https://lnkd.in/gaGnuNE6 Start with @ChatGPT in a channel, a group chat, or a DM, and it picks up relevant context right there. Answer the question in the thread. Unblock the decision. Turn a messy discussion into a brief, a plan, or a deck your team can use, without leaving Teams. This is what we built the Teams platform for. Conversation context, a user's identity and permissions, and admin-managed access to the tools a company already runs. That is what lets ChatGPT work natively, and seamlessly in Teams. Proud of where this landed and grateful to the people across Teams and OpenAI who did the hard work to make it real.. Excited for the partnership, with lots more to come, including the newly announced Dots integration! #MicrosoftTeams #OpenAI
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisToday, The Atlantic published the first excerpt from my upcoming book, THE AGI CHRONICLES. It's about the long-running feud between Sam Altman and Dario Amodei, who used to work together at OpenAI before Amodei and six of his colleagues left in 2020 to start a rival AI lab, Anthropic. The full story of the OpenAI/Anthropic split has never been told before, and it's pretty juicy. (Kanye West is involved, as is a stuffed panda named "Beary Bonds," and a secret plan to sell AGI to China and Russia.) Getting this story took months of digging and dozens of interviews, but I think it's important to understand why the leaders of the AI race are so motivated to beat each other. The book comes out next week! I've never worked harder on anything, and I'm very proud of how it turned out. https://lnkd.in/g6fR5nERInside the Biggest Feud in Artificial IntelligenceInside the Biggest Feud in Artificial Intelligence
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisSonnet 5.5 fixing a bug with Claude Code. 30% faster and 30% less usage.
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisSo proud of my daughter, Aria, for being named a 2026 Caroline D. Bradley Scholar! She’s one of 26 students nationwide selected to receive a merit-based, need-blind scholarship providing full high school tuition for four years through the Institute for Educational Advancement. More importantly for us, IEA works individually with Scholars to identify an optimally matched school or create an individualized educational plan. That’s particularly meaningful because Aria has never fit neatly into one lane - a quality I frequently see in the founders, entrepreneurs, students, and changemakers I have the privilege of meeting and advising. It’s been a gratifying, but at times challenging, journey to parent and support her. She loves mathematics deeply, studying Calculus BC as an eighth grader at Proof School and pursuing mathematical research in combinatorics with renowned mathematicians like Ken Ono. She also competes in freeride skiing, climbs and skis mountains, has performed at Carnegie Hall and danced with the San Francisco Ballet. Aria seems happiest when she’s discovering connections between worlds that adults usually put into separate boxes. For highly curious and passionate kids - and, I think, for unconventional founders and leaders - the challenge isn’t always finding more acceleration or more achievement. Sometimes it’s constructing a life in which intellectual depth, physical challenge, creativity, friendship, impact, and joy don’t have to compete with one another. One of Aria’s mentors, Paul Zeitz, once wrote that while her interests may seem orthogonal, they actually reinforce one another in unique ways. I see something similar in some of the most fulfilled founders and executives I meet: an ability to weave seemingly disparate obsessions and talents into a life tapestry that makes sense to them - even when it looks a little chaotic from the outside. Aria has been fortunate to encounter extraordinary people who have helped make that possible: teachers who took her questions seriously, mathematicians like Paul and Ken who treated her as a fellow explorer, coaches like Joe Stevens and Reine Barkered who pushed her while keeping skiing fun, artists like Sasha De Sola and Misa Kuranaga who showed her another language for expression, brands like The North Face and Atomic who have supported her, and communities that gave her places to belong. I’m enormously grateful to all of them, and to IEA, Deborah Monroe, Mallory Aldrich, and Daniel Martinez for now joining that village. Congratulations, Aria. Keep exploring all the things that make you, you. ❤️ IEA’s announcement and the full Caroline D. Bradley Class of 2031: https://lnkd.in/g5ftfHmr
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisReady to take AI from toy demos to enterprise infrastructure? We’re hosting two deep-dive technical workshops live in Sunnyvale in two weeks—built for anyone pushing the boundaries of compute, scale, and robotics. Choose your track (or conquer both): ⚡ Track 1: Reinforcement Learning & Custom Agents on GKE 🗓️ Tuesday, Oct 6 | 9:00 AM – 1:00 PM Cut API costs and scale production agent swarms. We're breaking down self-hosted Gemma on vLLM, hardware fungibility across GPUs/TPUs using GKE + Dynamic Workload Scheduler, and live, hands-on Ray RL loops for policy updates and DPO. 👉 RSVP: https://lnkd.in/gtwbqqV6 🤖 Track 2: Navigating the Cloud Physical AI Stack 🗓️ Thursday, Oct 8 | 9:00 AM – 1:00 PM Bridge the robotics data gap. Master NVIDIA Isaac Sim, Cosmos generative synthetic data, and Trillium TPU architectures powering sub-10ms multimodal reasoning. Features live embodied AI runs on Unitree robots and hands-on policy training. 👉 RSVP: https://lnkd.in/gr3EEU4d 📍 Where: Google Cloud Campus | Sunnyvale, CA 🛠️ Format: Architecture deep dives, live labs, and direct access to Google Cloud engineers (plus lunch on us). Grab your spot now!https://lnkd.in/gtwbqqV6 🤖 Track 2: Navigating the Cloud Physical AI Stack 🗓️ Thursday, Oct 8 | 9:00 AM – 1:00 PM Bridge the robotics data gap. Master NVIDIA Isaac Sim, Cosmos generative synthetic data, and Trillium TPU architectures powering sub-10ms multimodal reasoning. Features live embodied AI runs on Unitree robots and hands-on policy training. 👉 RSVP: https://lnkd.in/gr3EEU4d 📍 Where: Google Cloud Campus | Sunnyvale, CA 🛠️ Format: Architecture deep dives, live labs, and direct access to Google Cloud engineers (plus lunch on us). Grab your spot now!
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisLast week, we joined Amherst College in celebrating the grand opening of the new Ford Student Center—a 144,000-square-foot hub for campus life built through a highly coordinated blend of adaptive reuse and mass timber construction. Rather than starting from scratch, the team preserved and repurposed the existing concrete structure of the former Merrill Science Center, then built two new floors above in mass timber. The result brings dining, student life, cultural, recreation, and gathering spaces together under one roof—from Betty’s Kitchen and the Alumni Pub to cultural resource centers, performance spaces, lounges, terraces, and more. Congratulations to our partners at Herzog & de Meuron and Sasaki—and we can’t wait to share more from the project soon. 🔗 In the meantime, read more here: https://lnkd.in/e3b78EwJ #ShawmutBuilt
-
Ben (Xiaojun) Li liked thisBen (Xiaojun) Li liked thisWe’ve expanded OpenAI Academy with new courses for developers, leaders, educators, and college students 🎓 Learners practice applying AI to real tasks, whether they’re improving a business workflow, building with Codex and the OpenAI API, developing an AI strategy, or planning a class. Complete a course and pass the assessment to earn an OpenAI Academy course badge. Check out our blog post to explore the new learning paths and how to bring them to your organization: https://lnkd.in/ggKTN7jj
Experience & Education
Licenses & Certifications
Patents
-
System and Method for Managing Online Group Chat
Issued US 10,616,289
See patentPatent Approved
Big Group Collaboration US Patent:
System and Method for Managing Online Group Chat
Patent No. 10,616,289 Issue Date: April 7, 2020
Link: https://patentswarm.com/patents/US20170237785A1 -
AUTOMATED NOTIFICATION OF CONTENT UPDATE PROVIDING LIVE REPRESENTATION OF CONTENT INLINE THROUGH HOST SERVICE ENDPOINT(S)
US Pending
Languages
-
English
Native or bilingual proficiency
-
Chinese
Native or bilingual proficiency
Recommendations received
1 person has recommended Ben (Xiaojun)
Join now to viewOther similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content