Energy-Based Transformers explained | How EBTs and EBMs work

Description

I always wanted to know how energy-based models (EBMs) work. In this video, we break down EBMs — what they are, how they work, and how they’re different from standard neural networks. ► Then we zoom in on the Energy-Based Transformers (EBTs) paper by Gladstone et al. 2025, showing how the authors combined EBMs with transformers to create models that can refine their guesses, self-verify, and potentially adapt how much computation they use depending on the difficulty of the problem. AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/ Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 Vignesh Valliappan, Ivan Janov, Sunny Dhiana, Andy Ma 📃 Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam, Peixuan Han, Hyeonjeong Ha, Aman Chadha, Yilun Du, Heng Ji, Jundong Li, and Tariq Iqbal. "Energy-Based Transformers are Scalable Learners and Thinkers." (2025 https://arxiv.org/abs/2507.02092 ) Outline: 00:00 Energy-Based Transformers 00:47 EBT paper 01:07 Energy-based models explained 03:56 EBM training 06:37 EBM inference 07:55 Energy-based LLM 09:07 Stabilising Training via Noise 09:59 Replay Buffer 10:34 Randomising Step Size 10:26 Results 12:54 Energy-based Transformers for other Modalities ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak Join this channel as a Bean Member to get access to perks: https://www.youtube.com/channel/UCobqgqE4i5Kf7wrxRxhToQA/join ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter / X: https://twitter.com/AICoffeeBreak LinkedIn: https://www.linkedin.com/in/letitia-parcalabescu/ Threads: https://www.threads.net/@ai.coffee.break Bluesky: https://bsky.app/profile/aicoffeebreak.bsky.social Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak Substack: https://aicoffeebreakwl.substack.com/ Web: https://explanationmark.de/letitia https://aicoffeebreak.com #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​ Video editing: Nils Trost