Nonmonotone BFGS-trained recurrent neural networks for temporal sequence processing

Chun-Cheng Peng, George D. Magoulas
2011 Applied Mathematics and Computation  
In this paper we propose a nonmonotone approach to recurrent neural networks training for temporal sequence processing applications. This approach allows learning performance to deteriorate in some iterations, nevertheless the network's performance is improved over time. A self-scaling BFGS is equipped with an adaptive nonmonotone technique that employs approximations of the Lipschitz constant and is tested on a set of sequence processing problems. Simulation results show that the proposed
more » ... ithm outperforms the BFGS as well as other methods previously applied to these sequences, providing an effective modification that is capable of training recurrent networks of various architectures.
doi:10.1016/j.amc.2010.12.012 fatcat:neps6mha25bbhetgt5tliy6jmq