All videos
0:00 / 0:00
ai

From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing?

The TWIML AI Podcast with Sam Charrington3 October 2026Watch on YouTube

What you'll learn

  • AI systems have evolved from grade-school math to contributing to research problems such as Navier-Stokes.
  • Traditional benchmarks are saturating, which is why the Epoch Capabilities Index is needed to keep measuring progress.
  • Capability gains appear surprisingly steady across model generations, pointing to linear progress.
  • Models still struggle with open-ended work, learning from experience, and identifying promising research directions.
  • When solving hard math problems, models rely on persistence and prior human work.

Frequently asked questions

Why are traditional benchmarks becoming less useful for measuring AI progress?
Traditional benchmarks are saturating, so they can no longer distinguish progress well. That is why the Epoch Capabilities Index is used to keep measuring capabilities.
How do AI systems solve hard math problems according to the episode?
They rely on persistence and on prior human work. Whether they produce genuinely new ideas is a central question in the discussion.
Where do models still struggle?
Models struggle with open-ended work, with learning from experience, and with identifying promising new research directions.
How do you measure progress when the answers are unknown, as with FrontierMath?
The episode discusses benchmarking when answers are unknown, in the context of FrontierMath and unsolved problems. It also covers how to detect AI research breakthroughs.

Topics

Read next

Description from the channel

AI systems have gone from struggling with grade-school math to helping solve research problems that have resisted mathematicians for decades, including Navier-Stokes. In this episode, Greg Burnham, who leads AI capabilities research at Epoch AI, joins us to examine what that progress says about where AI is going. We look at how these systems are solving hard math problems, how much they rely on persistence and prior human work, and whether they are starting to produce genuinely new ideas. We also discuss how to measure progress as traditional benchmarks become less useful, why capability gains appear surprisingly steady across model generations, and where models still struggle with open-ended work, learning from experience, and identifying promising new research directions. Full show notes including references: https://twimlai.com/go/778. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 📖 CHAPTERS =============================== 00:00:00 Intro 00:01:59 From Academic Exams to Real World Capabilities 00:03:42 Benchmark Saturation and the Epoch Capabilities Index 00:06:38 Why Capability Gains Look Linear 00:13:54 Trend Breaks and the Drivers of AI Progress 00:17:05 Detecting AI Research Breakthroughs 00:19:52 From Grade School Math to Navier Stokes 00:28:27 How AI Solves Hard Math Problems 00:36:16 What Counts as a Genuinely New Idea 00:44:00 FrontierMath and Unsolved Problems 00:46:23 Benchmarking When the Answers Are Unknown 00:50:23 Open Ended Work and Learning on the Fly 00:54:13 Models Versus Harnesses 00:58:56 Earthborne Rangers and Video Game Evaluations 01:01:40 Tracking Present Capabilities and Future Trajectories 01:06:41 Closing 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://twitter.com/twimlai Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/