Separationi of Internal Representations of the Hidden Layer

Abstract

We devise a method to separate the internal representations of the hidden layer where the Hamming distance between every two representations is required to be as large as possible. Each representation is isolated as far as possible from all others in the layer space. When the representations of certain patterns can be isolated within a Hamming radius, we can discriminate these patterns from all other patterns using a single neuron is the next upper layer. This space is a hypercube which is different from the grid plane used in a self-organizing map. Such representations will exhaust this hypercube uniformly and have tolerance for noisy patterns. This method directly resolves the ambiguous internal representation problem, Which causes back-propagation learning to be inefficient. The layered network is developed as an adjustable kernel to separate multiple classes as much as possible. By employing this method along with the back-propagation learning algorithm, multilayer networks can be trained for various tasks