Evolution sets the stage for more powerful spiking neural networks
Indirect encoding of neural network components creates a pathway to complex applications for neural networks.
Left, an overview of the new evolutionary approach. Right, comparison of the new approach with another evolutionary approach to SNNs (EONS) showing the new approach reaches comparable performance to EONS, but does so more gradually.
Image courtesy of Catherine Schuman
The Science:
Spiking neural networks (SNNs) are a type of artificial neural network that most closely replicates the structure of the human brain. This makes SNNs an important step on the road to developing artificial intelligence. Researchers recently advanced a key technique for training SNNs using an evolutionary approach. This approach involves recognizing and making use of the different strengths of individual elements of the SNN, called "encodings." The researchers' approach involves indirect encoding, which replicates encodings instead of creating an entirely new encoding for each individual. This is a much more efficient way to build large--and therefore smarter--SNNs.
The Impact:
Indirect encoding helps scientists build significantly larger neural networks with more complex connections between neurons. These changes would help bring SNNs to increasingly complex applications. For example, improved SNNs would bring new power to the complex task of classifying large scientific images. By improving the performance of artificial intelligence systems, this research could enable scientists to find solutions to the challenges caused by climate change. It will also help scientists discover or improve clean energy technologies.
Summary:
Spiking neural networks offer tremendous potential for the future of artificial intelligence. For one, they can be implemented efficiently on neuromorphic systems, which closely mimic biological brains. One of the challenges in building functioning SNNs is the training process. Standard training processes are difficult to apply to SNNs. In this work, scientists created an evolutionary approach for training SNNs implementing an indirect encoding of individuals. Specifically, the researchers evolved SNNs using Compositional Pattern Producing Networks (CPPNs), which can learn the connectivity patterns between neurons defined in a coordinate space. Using CPPNs rather than direct encodings to evolve parameters allows representations of small numbers of individual encodings (on the order of tens or hundreds of individual encodings) to potentially represent thousands to millions of SNN parameter values.
Funding:
This research was funded by the Department of Energy Office of Science, Advanced Scientific Computing Research program.