Scientists show how the surface learning mechanism used by the brain can compete with deep learning

Neural network learning techniques come from brain dynamics. However, these two scenarios, brain learning and deep learning, are inherently different. One of the biggest differences is the number of layers each has.

Deep learning architectures typically consist of many layers that can be scaled up to hundreds, enabling efficient learning of complex classification tasks. In contrast, the brain is made up of very few layers, but despite its shallow architecture and noisy, slow dynamics, it can efficiently perform complex classification tasks.

The crucial question motivating the new research is the possible mechanism underlying the brain’s effective shallow learning, allowing it to perform classification tasks with the same accuracy as deep learning. In an article published in Physica A: Statistical mechanics and its applicationsresearchers from Bar-Ilan University in Israel show how such shallow learning mechanisms can compete with deep learning.

“Instead of deep architecture, like that of a skyscraper, the brain is made of wide, shallow architecture, more like a very wide building with only very few floors,” said the Professor Ido Kanter of the physics department of Bar-Ilan and Gonda (Goldschmied). Multidisciplinary Brain Research Center, which led the research.

“The ability to correctly classify objects increases where the architecture becomes deeper, with more layers. In contrast, the brain’s surface mechanism indicates that a larger network classifies objects better,” said student Ronit Gross. undergraduate and one of the main contributors to this study. work. “Wider and taller architectures represent two complementary mechanisms,” she added.

Nevertheless, achieving very wide and shallow architectures, mimicking brain dynamics, requires a change in the properties of advanced GPU technology, which is capable of accelerating deep architecture, but fails in implementing d wide and shallow architectures.