Khanh Le (PhD)

I really enjoyed reading this article about why it’s generally not a good idea to “rely on your gut” when making big decisions. But I’m not sure the answer (just hire a product manager to tell you what to do) is quite right either. No project is made a success as a result of one job role or person. Of course all technical projects need good product managers, but that’s mainly because they’re responsible for ensuring they bring together the right team and combination of skillsets required. If you’re an AI product manager working out who to hire to make your product a success, we’ve recently published all the details of the approach we took to scaling our AI tool to 90% of our eligible employees in a year. It also includes the skills and experience needed for each stage of the roll out and could be used to help you decide what people to bring together. We hope it’s helpful and we’re open to feedback from others who’ve gone on a similar journey, working out how to get the people side of AI transformation right. https://lnkd.in/eyTBSHvU

15

Surprising that the report was "AI generated" without human-in-the-loop verification: https://lnkd.in/geFyMD2p Companies should wake up to the fact that AI will hallucinate - it is a Godelian inevitability. This is primarily due to the training process, where they are trained to answer questions, without verification of whether those answers are correct or not. This is similar to a student who guesses answers in a test - this is fine as long as there are no negative marks, which can cause the student to get significantly lower marks than otherwise. This is similar to AI hallucinations, which are fundamentally inevitable, as this study shows (link in the comment below).

16

1 Comment

Not all model outputs are created equal. Exploring a great tool for quantifying uncertainty in LLMs. As we integrate Large Language Models (LLMs) into more critical applications, understanding when to trust their output is just as important as the output itself. This is where Uncertainty Quantification (UQ) becomes essential. Open-source library called uQLM (Uncertainty Quantification for Language Models) that tackles this exact problem. It provides a comprehensive toolkit to measure an LLM's confidence in its own generations, going beyond a simple softmax probability. The repo includes implementations for several state-of-the-art methods: > Semantic entropy for detecting uncertainty in free-form generations > Pragmatic entropy and Pragmatic calibration > Ensemble-based methods This is crucial for building reliable AI systems in high-stakes fields like healthcare, finance, or legal, where a model's miscalibrated confidence can lead to significant errors. By quantifying doubt, we can build safeguards, trigger human review, and ultimately create more trustworthy and responsible AI. Major kudos to the AI team at CVS Health for open-sourcing this valuable contribution to the ML community. It's a concrete step towards more robust and transparent AI. Check out the repository here: https://lnkd.in/gHNwXEjb #AI #MachineLearning #LLM #UncertaintyQuantification #ResponsibleAI #ExplainableAI #XAI #MLOps #ArtificialIntelligence #OpenSource #HealthTech

10

A few preliminary thoughts on GPT-5. 1. API pricing is tweaked (lower input cost, higher output cost) in a way that is likely to increase end costs. 2. OpenAI claims SOTA performance on coding. Claude 4 Opus and Sonnet still top it on SWE-bench, it has an edge on T2. 3. This is incremental improvement, not game changing.

15

1 Comment

Yesterday I joined Brent Bultitude on Sydney's 2SM 1269 to discuss Datacom's newly released whitepaper, The Productivity Pivot: AI, People and the Future of Work. As Brent pointed out, it also happened to be World Computer Literacy Day. Perfect timing for our conversation. Just hours before we went live, Australia's National AI Plan was released. Nothing like a bit of last-minute preparation! Before we chatted, I speed-read the Plan to understand its three pillars: capturing opportunities, spreading benefits, and keeping Australians safe. So I was as ready as I could be to share my thoughts – it’s encouraging to see government taking leadership on AI infrastructure investment, skills development, and safety measures, but it will be crucial to work together as an ecosystem (government, industry, technology companies, and education providers) to translate actions into practical, tangible outcomes for all Australians. But the conversation took an unexpected turn. Not in a bad way – I just needed to remind myself that while I work for a tech company and spend most of my time discussing what's changed in AI today or in the last hour and how we need to respond, many people are still wrapping their heads around the basics of AI, let alone engaging with national strategies or productivity frameworks. We talked about how to build practical AI skills and why AI is a genuine opportunity for Australia to become more productive and stimulate economic growth. We talked about how when people are scared of AI's impact on their job, they tend to run from it rather than leaning in and building their own capability. We even talked about whether Brent's job as a radio presenter would be replaced by AI. My favourite question was about leadership. Brent asked if leaders can make or break AI transformation, and whether those who don't get it end up dismissing its significance for their business. When it comes to building AI literacy, I believe that leadership is the most critical layer. Leaders need to understand AI to have that penny-drop moment about the significant change it brings, so they can lead the transformation. And to understand it they must be actively using it day-to-day. It's that simple! The National AI Plan addresses some of this – how we can develop our skills, and how to innovate in a safe and responsible way. What I'd like to see next are clearer pathways for upskilling leaders specifically. We need to help them think more strategically about AI, to move from just adopting AI tools to adapting the way we deliver services, manage our operations and interact with customers. Thanks Brent for a great conversation! After your authentic Italian pronunciation of 'Compagnone', I'm convinced that your job is safe! 👀 Read the National AI Plan – https://lnkd.in/gH8bgdCP 📝 Download our The Productivity Pivot: AI, People and the Future of Work whitepaper – link in comments below. Laura Malcolm Aram Lauxtermann Paul Wilson Glen Grant Vikki O'Shannessey

48

9 Comments