From Voice Agents to AI Avatars

What you'll learn

• Voice AI has improved a lot, but natural conversation remains a high bar because small delays, awkward interruptions or the wrong tone break the illusion.
• Alex Smola of Boson AI discusses the technical tradeoffs behind real-time voice, including audio tokenization, latency, model size and inference cost.
• Adding vision and visual presence raises the stakes for audiovisual agents and AI avatars.
• Emotional intelligence in AI plays a role in natural interaction, along with learning and personalization from human interactions.
• The episode covers what it takes to move from impressive demos toward interactions that actually feel natural.

Frequently asked questions

Topics

Read next

• Tavus introduceert Griffin: AI-videoavatar op echte videogesprekken

  Tavus lanceert Griffin, een 'Human Interaction Model' dat realtime videogesprekken voert en in een eigen onderzoek door 48 procent van de deelnemers na een gesprek van een minuut voor een echt persoon werd aangezien.

• Voice AI en autonome agents trekken honderden miljoenen aan investeringen

  Parloa haalde 350 miljoen dollar op, Deepgram 130 miljoen en ElevenLabs wordt naar verluidt gewaardeerd op 22 miljard dollar. De investeringsgolf illustreert hoe snel bedrijven AI-agents inzetten voor klantenservice die verder gaat dan eenvoudige chatbots.

Description from the channel

Voice AI has gotten remarkably good, but natural conversation remains a high bar. Small delays, awkward interruptions, or the wrong tone can quickly break the illusion, and adding vision and visual presence only raises the stakes. In this episode, Alex Smola, co-founder and CEO of Boson AI, explores the path from today’s voice agents to audiovisual agents and AI avatars. We discuss the technical tradeoffs behind real-time voice, including audio tokenization, latency, model size, and inference cost, as well as what changes when these systems can both see and be seen. We also explore the role of emotional intelligence in AI, how agents can learn from human interactions, and what it will take to move beyond impressive demos toward interactions that actually feel natural. Full show notes including references: https://twimlai.com/go/777. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 📖 CHAPTERS =============================== 00:00 Intro 01:11 The state of voice AI 06:10 Designing real-time voice systems 13:53 Boson AI’s path to audio 19:59 Building the audio data flywheel 28:41 Extending language models with audio 33:14 Architectures for responsive voice agents 39:26 Tools latency and natural interaction 46:20 Measuring interruption and proactivity 49:35 Emotional intelligence in AI 53:43 Learning and personalization from human interaction 01:03:56 Outro 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://twitter.com/twimlai Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/