Poolside’s Model Factory, Laguna S, Open Models, and the Race to AGI — Eiso Kant, Poolside AI

Description

From spending $12 million building language models for code before the world cared to creating a Model Factory that can take a model from pre-training to release in eight weeks, Eiso Kant has spent more than a decade betting that code is the path to AGI. In this episode, the Poolside co-founder joins swyx and Vibhu to explain why ChatGPT felt like vindication, why Poolside embraced open weights and open research, and why he would rather live in a world with 100 foundation model companies than five even if Poolside were one of the five. We go deep on Poolside’s Model Factory: the engineering systems behind 10,000-20,000 experiments per month, streaming data directly into training, reproducible experimentation, low-precision compute, and agents that increasingly write code, launch jobs, evaluate results, and modify the pipelines used to train future models. Eiso also unpacks Laguna S, why persistence, verification, and backtracking may matter more than raw intelligence, how much capability remains inside smaller models, why reinforcement learning will move earlier into pre-training, and why next-token prediction is still extracting too little from the web. We discuss: • How Andrej Karpathy’s RNN work inspired Eiso to start building language models for code in 2015 • Why Eiso spent four years and $12 million pursuing an idea before the market cared • The difference between releasing open weights and publishing genuinely open research • Why Poolside deliberately built a global research organization outside the Bay Area talent war • Why model building is ultimately 90% engineering • The Model Factory: Poolside’s end-to-end system for rapidly training and improving models • How Poolside moved from six-month model cycles to five- and eight-week launches • Why streaming data directly into training unlocked faster experimentation • How immutable data, versioned code, and reproducibility enable rigorous model research • Why Eiso wants capable researchers to leave their labs and become Poolside’s competitors • Why 95% of model building can be reduced to better data or better compute efficiency • Laguna S and why persistence, verification, and backtracking can outperform raw intelligence • Why distillation and environments have become the AI industry’s favorite “drugs” • Why mid-training is really an early form of curriculum design • Low-precision training, networking bottlenecks, and the next gains in compute efficiency • Laguna S: 118 billion total parameters, 8 billion active, and eight weeks from training to launch • Model versus harness: where agent capabilities actually come from • Why Poolside sees coding and long-horizon software tasks as a path to AGI • Why Poolside is prioritizing vision but does not expect to work on audio soon • Why language may be the most compute-efficient modality for encoding knowledge and reasoning • The story behind the Poolside name and why it represents refusing to lower the company’s ambitions • How Poolside raised $500M while investors still questioned whether AGI was real • When open models may become too capable to release without restrictions • How regulation could accidentally lock in an oligopoly of two or three AI companies • NVIDIA, TSMC, and the hardware systems underpinning progress in foundation models • Why reinforcement-learning wall-clock time is one of Poolside’s biggest bottlenecks • Why Poolside trains models from scratch instead of simply distilling larger models • How leaders align high-agency people through shared goals and clear constraints • Hiring across research, post-training, pre-training, architecture, evals, and engineering at Poolside Eiso Kant • LinkedIn: https://www.linkedin.com/in/eisokant • X: https://x.com/eisokant • Poolside: https://poolside.ai Timestamps 00:00:00 Introduction 00:00:54 Karpathy, RNNs, and Building Code Models Before Transformers 00:02:26 The $12M Failure and ChatGPT Vindication 00:03:39 Open Source and the Case for 100 Foundation Model Companies 00:09:22 Open Weights, Open Research, and Poolside’s Global Team 00:16:04 The Model Factory: Why Model Building Is 90% Engineering 00:20:19 Agents, Automated Experiments, and Early Signs of RSI 00:24:04 Streaming Data, Reproducibility, and Scientific Rigor 00:30:35 Creating More Foundation Model Companies 00:36:07 Laguna S: Persistence vs. Raw Intelligence 00:43:01 Reinventing Pre-Training, RL, and Curriculum Design 00:52:33 Low-Precision Training and Squeezing More From Smaller Models 00:58:37 Model Harnesses, Coding Agents, and the Path to AGI 01:09:26 Why MCP and Traditional Tool Calls Are “Stupid” 01:13:04 Vision, Multimodality, and Why Language Still Matters 01:18:15 Scaling Models and the Real Economics of Training 01:20:40 Why Poolside Is Called Poolside and Raising $500M 01:27:37 Open Models, AI Safety, and the Risk of an Oligopoly 01:33:53 NVIDIA, TSMC, and the Reinforcement-Learning Bottleneck 01:41:52 Smaller Models, Distillation, Engineering Productivity, and Hiring