Skip to Main content Skip to Navigation

A mathematical approach to unsupervised learning in recurrent neural networks

Mathieu Galtier 1 
CRISAM - Inria Sophia Antipolis - Méditerranée , INRIA Rocquencourt, ENS-PSL - École normale supérieure - Paris, UNS - Université Nice Sophia Antipolis (1965 - 2019), CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : In this thesis, we propose to give a mathematical sense to the claim: the neocortex builds itself a model of its environment. We study the neocortex as a network of spiking neurons undergoing slow STDP learning. By considering that the number of neurons is close to infinity, we propose a new mean-field method to find the ''smoother'' equation describing the firing-rate of populations of these neurons. Then, we study the dynamics of this averaged system with learning. By assuming the modification of the synapses' strength is very slow compared the activity of the network, it is possible to use tools from temporal averaging theory. They lead to showing that the connectivity of the network always converges towards a single equilibrium point which can be computed explicitely. This connectivity gathers the knowledge of the network about the world. Finally, we analyze the equilibrium connectivity and compare it to the inputs. By seeing the inputs as the solution of a dynamical system, we are able to show that the connectivity embedded the entire information about this dynamical system. Indeed, we show that the symmetric part of the connectivity leads to finding the manifold over which the inputs dynamical system is defined, and that the anti-symmetric part of the connectivity corresponds to the vector field of the inputs dynamical system. In this context, the network acts as a predictor of the future events in its environment.
Complete list of metadata

Cited literature [152 references]  Display  Hide  Download
Contributor : Bibliothèque MINES ParisTech Connect in order to contact the contributor
Submitted on : Tuesday, February 7, 2012 - 2:54:53 PM
Last modification on : Thursday, August 4, 2022 - 4:54:04 PM
Long-term archiving on: : Tuesday, May 8, 2012 - 2:31:41 AM


  • HAL Id : pastel-00667368, version 1


Mathieu Galtier. A mathematical approach to unsupervised learning in recurrent neural networks. Neural and Evolutionary Computing [cs.NE]. École Nationale Supérieure des Mines de Paris, 2011. English. ⟨NNT : 2011ENMP0057⟩. ⟨pastel-00667368⟩



Record views


Files downloads