A group of researchers on the University of California – San Diego is exploring how artificial neural networks may mimic sleep patterns of the human mind to mitigate the issue of catastrophic forgetting.
The analysis was printed in PLOS Computational Biology.
On common, people require 7 to 13 hours of sleep per 24 hours. While sleep relaxes the physique in some ways, the mind nonetheless stays very energetic.
Active Brain During Sleep
Maxim Bazhenov, PhD, is a professor of medication and sleep researcher at University of California San Diego School of Medicine.
“The brain is very busy when we sleep, repeating what we learned during the day,” Bazhenov says. “Sleep helps reorganize memories and presents them in the most efficient way.”
Bazhenov and his group have printed earlier work on how sleep builds rational reminiscence, which is the power to recollect arbitrary or oblique associations between objects, folks or occasions. It additionally protects in opposition to forgetting previous reminiscences.
The Problem of Catasrophic Forgetting
Artificial neural networks draw inspiration from the structure of the human mind to enhance AI applied sciences and methods. While these applied sciences have managed to realize superhuman efficiency within the type of computational velocity, they’ve one main limitation. When neural networks study sequentially, new data overwrites earlier data in a phenomenon known as catastrophic forgetting.
“In contrast, the human brain learns continuously and incorporates new data into existing knowledge, and it typically learns best when new training is interweaved with periods of sleep for memory consolidation,” Bazhenov says.
The group used spiking neural networks that artificially mimic pure neural methods. Rather than being communicated repeatedly, data is transmitted as discrete occasions, or spikes, at sure time factors.
Mimicking Sleep in Neural Networks
The researchers found that when spiking networks have been skilled on new duties with occasional off-line durations mimicking sleep, the issue of catastrophic forgetting was mitigated. Similar to the human mind, the researchers say “sleep” permits the networks to replay previous reminiscences with out explicitly utilizing previous coaching knowledge.
“When we learn new information, neurons fire in specific order and this increases synapses between them,” Bazhenov says. “During sleep, the spiking patterns learned during our awake state are repeated spontaneously. It’s called reactivation or replay.
“Synaptic plasticity, the capacity to be altered or molded, is still in place during sleep and it can further enhance synaptic weight patterns that represent the memory, helping to prevent forgetting or to enable transfer of knowledge from old to new tasks.”
The group discovered that by making use of this strategy to synthetic neural networks, it helped the networks keep away from catastrophic forgetting.
“It meant that these networks could learn continuously, like humans or animals,” Bazhenov continues. “Understanding how the human brain processes information during sleep can help to augment memory in human subjects. Augmenting sleep rhythms can lead to better memory.
“In other projects, we use computer models to develop optimal strategies to apply stimulation during sleep, such as auditory tones, that enhance sleep rhythms and improve learning. This may be particularly important when memory is non-optimal, such as when memory declines in aging or in some conditions like Alzheimer’s disease.”