Hybrid neural networks: An evolutionary approach with local search

Eduardo Masato Iyoda, Fernando J. Von Zuben
2002 Integrated Computer-Aided Engineering  
Considering computational algorithms available in the literature, associated with supervised learning in feedforward neural networks, a wide range of distinct approaches can be identified. While the adjustment of the connection weights represents an omnipresent stage, the algorithms differ on three basic aspects: the technique chosen to determine the dimension of the multilayer neural network, the procedure adopted to specify the activation functions, and the kind of composition used to produce
more » ... the output. Advanced learning algorithms should be developed to simultaneously treat all these aspects during learning, and an evolutionary learning algorithm with local search is proposed here. The essence of this approach is a synergy between genetic algorithms and conjugate gradient optimization, operating on a hybrid neural network architecture. As a consequence, the final neural network is automatically generated, and is characterized to be dedicated and computationally parsimonious. * Corresponding author. ing processes still present a strong restriction: additive composition is the only way to combine the possibly distinct activation functions in order to produce the network output. So, we present in this paper a new neural network architecture, called hybrid neural network [16] , whose output is produced by a hybrid composition of activation functions that also accepts the multiplicative operator, besides the additive one. The hybrid neural network can be described as a neural network composed of generalized neurons, which are arranged into a mixture of a layered and a cascade configuration. A generalized neuron differs from the conventional one in two aspects: the level of internal activity can be produced by means of an additive or multiplicative composition of the inputs; and the shape of the activation function may be previously defined or determined during learning. This paper is organized as follows: in Section 2, the traditional single hidden layer neural network with identical activation functions is presented. Projection pursuit learning is introduced in Section 3. In Section 4, the hybrid neural network architecture is proposed to overcome the limitations presented by the previous models. Learning aspects of the hybrid neural
doi:10.3233/ica-2002-9104 fatcat:242tnk65jnfdrpupygebascniy