Journal Article
Computer Vision

Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

Kai Olav Ellefsen(Norwegian University of Science and Technology), Jean-Baptiste Mouret(Centre National de la Recherche Scientifique), Jeff Clune(University of Wyoming)
April 2, 2015PLoS Computational Biology177 citations
PDFDOI

177

Citations

8

Influential Citations

PLoS Computational Biology

Venue

2015

Year

Abstract

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.

Analysis

Why This Paper Matters

Catastrophic forgetting is a fundamental challenge in neural networks, where learning new tasks overwrites previously acquired knowledge. This paper addresses this problem by leveraging modularity, a principle observed in biological brains. The key insight is that by evolving networks with a connection cost, the authors induce modular structures that naturally separate functionality, allowing selective learning. This is significant because it offers a biologically plausible and computationally feasible method to enable lifelong learning in AI agents, moving beyond simple regularization or replay-based approaches.

The paper is particularly relevant for AI practitioners working on continual learning, reinforcement learning, and evolutionary robotics. It demonstrates that architectural constraints (modularity) can be as important as algorithmic innovations for mitigating interference. The use of neuromodulation to gate learning rates per connection is a clever mechanism that aligns with neuroscience findings, making the approach both principled and practical.

Technical Contributions

  • Connection Cost for Modularity: The authors evolve networks with a penalty on the number of connections, which naturally leads to modular architectures. This confirms prior work and provides a simple, effective way to induce modularity without hand-designing modules.
  • Neuromodulation for Selective Learning: Learning is controlled by neuromodulatory signals that adjust plasticity per connection based on task context. This allows the network to lock in old skills while adapting new ones.
  • Separate Reinforcement Learning Module: The evolved modular networks often contain a dedicated module for reinforcement learning, which only updates in response to reward signals, further reducing interference with sensory or motor processing.
  • Empirical Validation: The authors compare modular vs. non-modular networks on a set of tasks, measuring both learning speed and retention of old skills.

Results

  • Modular networks with connection costs achieve higher overall performance than non-modular networks across multiple tasks.
  • They learn new skills faster and retain old skills significantly better, directly addressing catastrophic forgetting.
  • The presence of a separate reinforcement learning module is a key factor: networks with such a module outperform those without.
  • The connection cost technique reliably produces modularity, replicating previous findings and extending them to a learning context.

Significance

This paper provides strong evidence that architectural modularity can alleviate catastrophic forgetting, a problem that has plagued neural networks for decades. It offers a practical technique (connection cost + neuromodulation) that can be integrated into evolutionary or gradient-based training pipelines. The work also bridges neuroscience and AI, suggesting that the modular organization of animal brains may serve to enable continual learning. For AI practitioners, this implies that designing networks with built-in modularity and selective plasticity could be a key strategy for building agents that accumulate skills over a lifetime.

catastrophic-forgettingmodular-neural-networksneuromodulationevolutionary-algorithmsreinforcement-learningconnection-costlifelong-learning
Reinforcement LearningNeural Networks

Related Papers