Dynamic network plasticity and sample efficiency in biological neural cultures: A comparative study with deep reinforcement learning
image: (A) Schematic illustration of the DishBrain feedback loop, the simulated game environment, and electrode configurations. (B) A schematic illustration of the overall network construction framework. The spiking time series data are first transformed into a 3D space using t-SNE embedding. These lower-dimensional representations are then combined into a tensor, which is decomposed using Tucker decomposition. The K-medoids algorithm is then applied to identify consistent representative channels across all cultures. These channels become network nodes, and pairwise Pearson correlation values serve as edge weights. The network layout reflects the physical placement of channels on the MEA, with node colors distinguishing sensory (green) from motor (blue) regions. (C) Schematic comparing the information input routes in the DishBrain system (left) and the 3 implementations of the deep RL algorithms (right). In each design, the input information to the computing module (deep RL algorithms or DishBrain) is denoted by a vector I. Note that in the DishBrain system, while this figure depicts stylized waveforms for illustrative purposes, the actual stimulation consisted of discrete electrical pulses.
Credit: Moein Khajehnejad, Cortical Labs.
Biological intelligence is typically regarded as the natural standard of intelligence, while artificial intelligence, particularly in terms of mimicking biological learning processes, has made significant progress. Biological neural systems exhibit extremely high learning efficiency, enabling them to quickly adapt to new tasks with minimal trials and typically achieve learning with low energy consumption. Existing deep reinforcement learning (RL) algorithms, despite surpassing human performance in certain tasks, such as large-scale pattern recognition, still face significant challenges in terms of sample efficiency, computational resource consumption, and adaptability to the environment. Training reinforcement learning algorithms often requires large amounts of samples and computational resources, and they are prone to issues such as catastrophic forgetting and low sample efficiency when dealing with complex tasks, which contrasts sharply with the high learning efficiency of biological systems. Biological neural networks are able to achieve high learning outcomes with minimal input stimuli and data, while their energy efficiency far exceeds that of artificial intelligence systems. “Therefore, understanding the adaptability and efficiency exhibited by biological neural networks during the learning process is crucial for improving artificial intelligence systems, especially in enhancing the learning efficiency of reinforcement learning models.” said the author Moein Khajehnejad, a researcher at Cortical Labs, “To this end, we have constructed the DishBrain biological neural system and compared it with state-of-the-art reinforcement learning methods in the Pong game to explore the differences between the two in the learning process, particularly in terms of sample efficiency.”
The authors first constructed an innovative platform, DishBrain, which combines in vitro neural cultures with high-density multi-electrode arrays (HD-MEA) to simulate and study the learning process of biological neural networks. In the DishBrain system, neural cultures are grown on the multi-electrode arrays, allowing real-time recording of neural activity combined with structured electrical stimulation to create a closed-loop game environment. In this system, the electrophysiological activity of the neural network is closely monitored and regulated in interaction with the virtual game environment, enabling the neural network to learn based on the input stimuli and feedback. Specifically, the DishBrain system uses biphasic electrical stimulation to simulate the movement of the paddle and ball in the Pong game, where the frequency and position of the stimulation encode key information about the game.Through this system, the biological neural network can interact with the game environment via sensory inputs and feedback, and respond to the game task through self-organized electrophysiological dynamics.
The authors then describe how they constructed and analyzed the functional connectivity of biological neural networks to reveal the dynamic changes in neural activity during the learning process. First, high-density multi-electrode arrays (HD-MEA) were used to record neural firing data obtained from 24 different cultures, covering 437 experimental sessions, including 262 "game" sessions and 175 "rest" sessions. Due to the high dimensionality of the neural data recorded during the game, the authors applied dimensionality reduction techniques (such as t-SNE) to effectively reduce the complexity of the data while preserving the dynamic properties of the network. Next, the authors used Tucker decomposition and the K-medoids clustering algorithm to extract 30 representative channels from all the recorded game sessions, which were considered to be closely related to the neural network's behavior during the game. In this way, a network matrix was constructed, with the nodes representing these representative channels, and the edges between them representing their functional connectivity, calculated using zero-lag Pearson correlation coefficients. Finally, the authors analyzed the changes in the network's connectivity between game and rest states through these network structures, revealing significant changes in the network's structure and functional connectivity during the game.
The experimental results demonstrate that in vitro biological neural networks exhibit significant network plasticity and learning ability in a simplified Pong game environment, especially outperforming deep reinforcement learning algorithms in terms of sample efficiency. First, through the analysis of neural activity, the study found that the functional connectivity of the biological neural network changed significantly in the "game" state. Compared to the "rest" state, several metrics of the neural network, such as the number of nodes, number of edges, density, mean participation coefficient, and modularity index, showed significant differences. This indicates that the neural network reorganizes its internal structure and functional connectivity in a self-organized manner during the learning process, further demonstrating its high adaptability and plasticity. Next, in terms of sample efficiency, the study compared biological neural networks with deep reinforcement learning algorithms (such as Deep Q Network, Advantage Actor-Critic, and Proximal Policy Optimization). The results showed that under limited sample conditions, biological neural networks achieved higher game performance in a shorter period of time, surpassing these reinforcement learning algorithms. The biological neural networks demonstrated higher learning efficiency, especially when sample size and computational time were limited, allowing for rapid learning and better performance. Furthermore, the biological neural networks also significantly outperformed the reinforcement learning algorithms in the game, particularly in terms of "long rally" frequency, rally length, and hit accuracy. The superior performance of biological neural networks in these metrics demonstrates their faster learning speed and better game performance. This was especially true when using different input designs (such as image input, paddle and ball position input, etc.), where the biological system's learning effectiveness clearly exceeded that of deep reinforcement learning algorithms.
However, despite the impressive performance of the DishBrain biological neural network system in the simplified Pong game, there are still several limitations. First, the networks used are relatively small in scale, and the tasks are simple. Therefore, future research could expand the network scale and adopt more complex tasks and environments to explore the performance of biological neural networks in higher-dimensional and more complex problems. This would help provide a more comprehensive assessment of their potential in real-world applications. Secondly, the experiments in this study primarily focused on short-term game tasks, and the performance of the neural networks may vary under different experimental conditions. Future research should address the long-term stability and reproducibility of biological neural networks, ensuring their consistency during prolonged learning and adaptation. This is crucial for applying them to more complex tasks. Finally, while this paper provides a comparison between biological neural networks and deep reinforcement learning algorithms, the integration of the two has not been fully explored. “In the future, we will further investigate how to combine biological neural networks with existing artificial intelligence algorithms, utilizing the efficient learning characteristics of biological systems to improve current AI technologies, particularly in terms of sample efficiency and energy efficiency.” said Moein Khajehnejad.
Authors of the paper include Moein Khajehnejad, Forough Habibollahi, Alon Loeffler, Aswin Paul, Adeel Razi, and Brett J. Kagan.
During the course of this research, M.K. and A.R. were funded by an Office of National Intelligence grant (Ref. NI230100153).
The paper, “Dynamic Network Plasticity and Sample Efficiency in Biological Neural Cultures: A Comparative Study with Deep Reinforcement Learning” was published in the journal Cyborg and Bionic Systems on Aug. 4, 2025, at DOI: 10.34133/cbsystems.0336.