Recurrent Neural Networks
Intelligent Systems Reference Library
PREFACE Recurrent neural networks have been an interesting and important part of neural network research during the 1990's. They have already been applied to a wide variety of problems involving time sequences of events and ordered data such as characters in words. Novel current uses range from motion detection and music synthesis to financial forecasting. This book is a summary of work on recurrent neural networks and is exemplary of current research ideas and challenges in this subfield of
... ificial neural network research and development. By sharing these perspectives, we hope to illuminate opportunities and encourage further work in this promising area. Two broad areas of importance in recurrent neural network research, the architectures and learning techniques, are addressed in every chapter. Architectures range from fully interconnected to partially connected networks, including recurrent multilayer feedforward. Learning is a critical issue and one of the primary advantages of neural networks. The added complexity of learning in recurrent networks has given rise to a variety of techniques and associated research projects. A goal is to design better algorithms that are both computationally efficient and simple to implement. Another broad division of work in recurrent neural networks, on which this book is structured, is the design perspective and application issues. The first section concentrates on ideas for alternate designs and advances in theoretical aspects of recurrent neural networks. Some authors discuss aspects of improving recurrent neural network performance and connections with Bayesian analysis and knowledge representation, including extended neuro-fuzzy systems. Others address real-time solutions of optimization problems and a unified method for designing optimization neural network models with global convergence. The second section of this book looks at recent applications of recurrent neural networks. Problems dealing with trajectories, control systems, robotics, and language learning are included, along with an interesting use of recurrent neural networks in chaotic systems. The latter work presents evidence for a computational paradigm that has higher potential for pattern capacity and boundary flexibility than a multilayer static feedforward network. Other chapters examine natural language as a dynamic system appropriate for grammar induction and language learning using recurrent neural networks. Another chapter applies a recurrent neural network technique to problems in controls and signal processing, and other work addresses trajectory problems and robot behavior. The next decade should produce significant improvements in theory and design of recurrent neural networks, as well as many more applications for the creative solution of important practical problems. The widespread application of recurrent neural networks should foster more interest in research and development and raise further theoretical and design questions.