All videos
0:00 / 0:00
research

Do AI Tokenomics Matter More Than Model Benchmarks?

The TWIML AI Podcast with Sam Charrington9 September 2026Watch on YouTube

What you'll learn

  • You learn what 'tokenflation' means, the idea that token usage grows faster than the measurable value those tokens produce.
  • You see why benchmarks alone provide an incomplete picture of model progress and why economic value matters.
  • You discover what inference-time scaling means for the cost and economics of increasingly capable AI models.
  • You learn why expert AI users achieve better results by challenging and iterating with models, and how AI fluency affects outcomes.
  • You hear how more efficient architectures could change the underlying economics of AI systems.

Frequently asked questions

What does Chris Potts exactly mean by 'tokenflation'?
Tokenflation is the idea that the token usage of AI systems grows faster than the measurable value those tokens produce. The concept analyzes the relationship between rising token usage costs and the actual economic return on them.
Why are benchmarks not a complete picture of model progress according to the episode?
Benchmarks only show part of the progress and give no insight into the economic value models create. Chris emphasizes the importance of also looking at what tokens actually deliver in terms of value and cost.
How does AI fluency affect the results users achieve?
Research shows that experienced AI users achieve better results by challenging models and keeping iterating. AI fluency, the degree to which users know how to effectively use models, therefore directly influences outcomes.
What is inference-time scaling and what does it mean for the economics of AI?
Inference-time scaling refers to using more and more tokens during inference, and reasoning models therefore consume more tokens. This makes systems more expensive to run, which makes it more important to understand what those tokens actually buy.

Topics

tokenomicscost-benefitinference-time scalingbenchmarksai fluency

Read next

Description from the channel

As reasoning models consume more tokens and AI systems become more expensive to run, understanding what those tokens actually buy is becoming increasingly important. In this episode, Stanford professor and Big Spin co-founder Chris Potts joins us to discuss AI tokenomics and his research into “tokenflation”—the possibility that token usage is growing faster than the measurable value those tokens produce. We explore how to measure the return on AI spending, why benchmarks alone provide an incomplete picture of model progress, and what inference-time scaling means for the economics of increasingly capable models. Chris also explains why expert AI users tend to get better results by challenging and iterating with models, how AI fluency affects outcomes, and why more efficient architectures could change the underlying economics. We also discuss DSPy, interpretability, the limits of today’s transformer architectures, and where Chris sees opportunities for more fundamental innovation in AI. 🗒️ Full show notes including references: https://twimlai.com/go/776. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 📖 CHAPTERS =============================== 00:00 - Introduction 05:38 - Linguistics in the Age of Language Models 09:26 - Scale Limitations in NLP Research 12:54 - Challenging the Bitter Lesson Mindset 15:12 - Relationship Between Data, Mechanistic Interpretability, and Efficiency 17:03 - DSPy 21:32 - Prompt Optimization and Model Variability 24:35 - Tokenomics and the Rising Cost of AI 28:13 - Measuring Token Purchasing Power with a CPI 32:18 - Inference-Time Scaling 35:36 - Defining Value Across Different AI Tasks 38:33 - AI Value Creation 40:38 - Predicting AI Costs 42:25 - Tokenflation 46:22 - AI Fluency 50:12 - Key Lessons of AI Fluency Work 54:44 - Future Directions 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://twitter.com/twimlai Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/