0:00 / 0:00
research
Why Distillation Might Be Impossible to Stop
LangChain21 July 2026Watch on YouTube
Description
Eno Reyes, CTO of Factory, makes the case that different model architectures trained on the same data are converging toward a shared representation of reality — and that this makes distillation effectively unpreventable. His take: model labs know this, which is part of why regulatory capture is on the table. From the Max Agency podcast, hosted by Harrison Chase, Co-Founder and CEO of LangChain. #AI #LLM #MachineLearning #AgenticAI
What you'll learn
- Different model architectures trained on the same data converge toward a shared representation of reality
- Model distillation becomes practically unpreventable because internal model representations grow toward each other
- AI labs understand this convergence principle and factor it into regulatory discussions
- Model convergence makes it difficult to prevent knowledge transfer between models
Frequently asked questions
What is the core argument about model distillation in this video?
Eno Reyes argues that model distillation is practically inevitable because different model architectures, when trained on the same data, converge toward the same internal representation of reality. This makes it effectively impossible to prevent knowledge transfer between models.
How does model convergence relate to regulation according to Reyes?
Reyes argues that AI labs understand that models converge to the same representation, and this understanding influences their approach to regulation. This insight plays a role in regulatory debates and discussions around regulatory capture.
Why do different model architectures converge toward the same representation?
Because they are all trained on the same data, their internal representations of reality grow toward each other. According to Reyes, this is a natural consequence of learning from the same information.