A HYBRID PARADIGM FOR WEIGHT INITIALIZATION IN SUPERVISED FEEDFORWARD NEURAL NETWORK LEARNING

Abstract

This initial set of weights to be used in supervised learning for multilayer neural networks has a strong influence in the learning speed and in the quality of the solution obtained after convergence. An inadequate initial choice for the weight values may cause the training process to get stuck in a poor local minimum or to face abnormal numerical problems. Nowadays, there are several proposed techniques that try to avoid both local minima and numerical instability, only by means of a proper definition of the initial set of weights. However, the problem of the majority of these approaches is that they persist on ignoring useful properties of the training set when presented to the neural network. An alternative hybrid paradigm for weight initialization in feedfroward neural networks is proposed here, and applied to several benchmark problems. The results are then compared with that produced by other approaches found in the literature.