Mansi Maheshwari - Addressing the Plasticity Stability Dilemma in Reinforcement Learning

Description

Neural networks have shown remarkable success in supervised learning when trained on a single task using a fixed dataset. However, when neural networks are trained on a reinforcement learning task, their ability to continue learning from new experiences declines over time. This decline in learning ability is known as plasticity loss. To restore plasticity, prior work has explored periodically resetting the parameters of the learning network, a strategy that often improves overall performance. However, such resets come at the cost of a temporary drop in performance, which can be dangerous in real-world settings. To overcome this instability, we introduce AltNet, a reset-based approach that restores plasticity without performance degradation by leveraging twin networks. More broadly, plasticity underpins several desirable attributes of effective RL agents: rapid adaptation to distribution shift, efficient reuse of past data, and high performance with limited interactions. Without the capacity to change, these goals are compromised. Viewed through this lens, AltNet addresses more than plasticity loss: it enables rapid adaptation and efficient data reuse while maintaining stable learning dynamics through its twin-network anchoring mechanism. Together, these capabilities are foundational for reinforcement learning agents that must continuously adapt over time while remaining stable and data-efficient. Mansi Maheshwari is a Master's student in Computer Science at the University of Massachusetts Amherst, where she is advised by Professor Bruno Castro da Silva at the Autonomous Learning Lab. Her research focuses on lifelong reinforcement learning, studying how RL agents can continually adapt under non-stationarity. This work has been published at CoLLAs 2025 (poster) and accepted at AAMAS 2026 (oral). Alongside her research, Mansi is deeply committed to broadening participation in AI. She is teaching Fundamentals of AI to high school students as an AI Instructor at the University of Washington and is consulting with iCEV to help design an upcoming AI textbook for secondary education. Previously, she earned her B.S. in Electrical Engineering from the University of Washington. This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Rahul Narava and Gusti Winata, Leads of our Reinforcement Learning group for their dedication in organizing this event. If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker. Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommunityApp).