Natural Elastic Nets for Faithful Representations

Abstract

In this work, we derive self-organization by constructing a generative mode, which is composed of piecewise multivariate Gaussian distributions for characterizing the parameter space. The fitness of this generative model to all parameters provides a smoothness criterion to the essential exploration of the clustering topology within the parameter space. Combined with the minimal wiring criterion proposed by Durbin and Willshaw, the new criterion is able to produce faithful representations of self-organization. We apply a hybrid of the mean field annealing and the gradient descent method to the optimization of mathematical framework in treatment of discrete combinatorial variables and continuous geometrical variables, and obtain three sets of interactive dynamics to which the corresponding unsupervised learning process is termed as natural elastic net algorithm. If the covariance matrix of each piecewise multivariate Gaussian distribution is fixed as an identity matrix, the interactive dynamics describe a learning process for the elastic net of Durbin and Willshaw.