A mathematical approach to unsupervised learning in recurrent neural networks

Abstract : In this thesis, we propose to give a mathematical sense to the claim: the neocortex builds itself a model of its environment. We study the neocortex as a network of spiking neurons undergoing slow STDP learning. By considering that the number of neurons is close to infinity, we propose a new mean-field method to find the ''smoother'' equation describing the firing-rate of populations of these neurons. Then, we study the dynamics of this averaged system with learning. By assuming the modification of the synapses' strength is very slow compared the activity of the network, it is possible to use tools from temporal averaging theory. They lead to showing that the connectivity of the network always converges towards a single equilibrium point which can be computed explicitely. This connectivity gathers the knowledge of the network about the world. Finally, we analyze the equilibrium connectivity and compare it to the inputs. By seeing the inputs as the solution of a dynamical system, we are able to show that the connectivity embedded the entire information about this dynamical system. Indeed, we show that the symmetric part of the connectivity leads to finding the manifold over which the inputs dynamical system is defined, and that the anti-symmetric part of the connectivity corresponds to the vector field of the inputs dynamical system. In this context, the network acts as a predictor of the future events in its environment.
Document type :
Theses
Neural and Evolutionary Computing. École Nationale Supérieure des Mines de Paris, 2011. English. <NNT : 2011ENMP0057>


https://pastel.archives-ouvertes.fr/pastel-00667368
Contributor : Doriane Ibarra <>
Submitted on : Tuesday, February 7, 2012 - 2:54:53 PM
Last modification on : Monday, March 5, 2012 - 3:33:50 PM

Identifiers

  • HAL Id : pastel-00667368, version 1

Citation

Mathieu Galtier. A mathematical approach to unsupervised learning in recurrent neural networks. Neural and Evolutionary Computing. École Nationale Supérieure des Mines de Paris, 2011. English. <NNT : 2011ENMP0057>. <pastel-00667368>

Export

Share

Metrics

Consultation de
la notice

606

Téléchargement du document

248