Alle video's
0:00 / 0:00
research

Raymond Chua - Predictive Representations and Memory for Continual Reinforcement Learning

Cohere11 september 2026Bekijk op YouTube

Wat je leert

  • Je leert wat voorspellende toestandsrepresentaties zijn via Simple Successor Features, geïnspireerd op het Successor Representation-model en hippocampus cognitieve kaarten.
  • Je ziet hoe Simple Successor Features continual reinforcement learning verbeteren en tegelijk de theoretische eigenschappen van de onderliggende Successor Representations behouden.
  • Je leert hoe biologisch geïnspireerde synaptische consolidatie samenhangt met voorspellende representaties in dynamisch veranderende omgevingen.
  • Je hoort dat consolidatie effectiever is wanneer deze voorspellende representaties stabiliseert in plaats van alleen actie-waarde-koppelingen.
  • Je krijgt de hypothese mee dat gestructureerde interne modellen een robuuster computationeel substraat vormen voor levenslang leren.

Veelgestelde vragen

Wat zijn Simple Successor Features in dit onderzoek?
Simple Successor Features zijn een efficiënte vorm van voorspellende toestandsrepresentaties, geïnspireerd op het Successor Representation-model dat gekoppeld is aan hippocampus cognitieve kaarten. Ze verbeteren continual reinforcement learning en behouden de theoretische eigenschappen van de onderliggende Successor Representations.
Wat blijkt uit de resultaten over synaptische consolidatie?
Consolidatie blijkt bijzonder effectief wanneer deze voorspellende representaties stabiliseert in plaats van alleen actie-waarde-koppelingen. Dit ondersteunt het idee dat gestructureerde interne modellen een robuuster computationeel substraat vormen voor levenslang leren.
Waarom is continual learning een uitdaging voor deep reinforcement learning?
Biologische agentia zoals mensen en dieren passen zich continu aan in complexe, dynamische omgevingen, maar het mogelijk maken van continual learning blijft een grote uitdaging voor deep reinforcement learning-agentia ondanks recente vooruitgang in AI.
Wat onderzoekt Raymond Chua na zijn promotie?
Hij gaat naar het Center for Theoretical Neuroscience aan Columbia University als postdoctoraal onderzoeker, waar hij met Prof. Ken Miller en Prof. Kim Stachenfeld werkt aan computationele neurowetenschap en NeuroAI.

Onderwerpen

reinforcement learningcontinual learningpredictive representationscomputational neuroscienceneuroai

Beschrijving van het kanaal

​Biological agents, such as humans and animals, are capable of continuous adaptation and learning in complex, dynamic environments. Despite recent advancements in Artificial Intelligence (AI), enabling continual learning remains a major challenge for deep Reinforcement Learning (RL) agents. Understanding the computational principles that support lifelong learning therefore represents both a neuroscience question and an AI challenge. In this talk, I will present two pieces of work from my PhD that address this question. First, I will introduce predictive state representations through Simple Successor Features (SFs), which is an efficient approach inspired by Successor Representation, that is a computational model linked with hippocampal cognitive maps. I will show how these predictive representations improve continual RL while preserving the theoretical properties of the underlying SRs. I will then discuss biologically-inspired synaptic consolidation mechanisms that interact with these predictive representations to improve continual learning in dynamically changing environments. Our results suggest that consolidation is particularly effective when it stabilises predictive representations rather than action-value mappings alone, supporting the idea that structured internal models may provide a more robust computational substrate for lifelong learning. Together, these studies illustrate how computational principles inspired by neuroscience can improve continual learning in artificial agents while offering new hypotheses about how biological systems acquire, adapt, and preserve knowledge over time. Bio: Raymond Chua recently completed his Ph.D. in Computer Science at McGill University and Mila – Quebec AI Institute, where he was co-supervised by Prof. Doina Precup and Prof. Blake A. Richards. His research lies at the intersection of neuroscience and artificial intelligence, with a focus on continual learning, reinforcement learning, and biologically inspired algorithms. During his Ph.D., he developed methods based on predictive representations and synaptic consolidation to improve continual adaptation in reinforcement learning agents, drawing inspiration from computational neuroscience and hippocampal theories of learning and memory. He will soon be joining the Center for Theoretical Neuroscience at Columbia University as a postdoctoral researcher, where he will work with Prof. Ken Miller and Prof. Kim Stachenfeld on computational neuroscience and NeuroAI. His long-term goal is to understand the computational principles underlying biological intelligence and leverage them to build more adaptive AI systems while advancing our understanding of brain function. This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Rahul Narava, Lead of our Reinforcement Learning group for their dedication in organizing this event. If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker. Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommunityApp).