Filters








334 Hits in 7.6 sec

Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems [article]

Jimmy T.H. Smith, Scott W. Linderman, David Sussillo
2021 arXiv   pre-print
We present a new model that overcomes these limitations by co-training an RNN with a novel switching linear dynamical system (SLDS) formulation.  ...  Recurrent neural networks (RNNs) are powerful models for processing time-series data, but it remains challenging to understand how they function.  ...  J.T.H.S. received funding support from a Stanford Graduate Fellowhip in Science and Engineering (Mayfield fellowship).  ... 
arXiv:2111.01256v1 fatcat:muiicle7orhc7ifcyyih6m2hdu

Stiff Neural Ordinary Differential Equations [article]

Suyong Kim, Weiqi Ji, Sili Deng, Yingbo Ma, Christopher Rackauckas
2021 arXiv   pre-print
The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering  ...  Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications.  ...  GELU was chosen to mitigate potential issues with vanishing gradients typically seen in recurrant neural networks with saturating activation functions 38 .  ... 
arXiv:2103.15341v3 fatcat:ocaccsxhrre6bgbzi6mdksl76e

Emergent behavior and neural dynamics in artificial agents tracking turbulent plumes [article]

Satpreet Harcharan Singh, Floris van Breugel, Rajesh P. N. Rao, Bingni Wen Brunton
2021 arXiv   pre-print
Specifically, we use deep reinforcement learning (DRL) to train recurrent neural network (RNN) agents to locate the source of simulated turbulent plumes.  ...  At the level of neural dynamics, the RNNs' population activity is low-dimensional and organized into distinct dynamical structures, with some correspondence to behavioral modules.  ...  Tree- ing naturalistic human behaviors in long-term video and neural structured recurrent switching linear dynamical systems for multi- recordings.  ... 
arXiv:2109.12434v2 fatcat:tin2h4v2frhuhpeezh7gzeqvxq

Training Recurrent Neural Networks via Dynamical Trajectory-Based Optimization [article]

Hamid Khodabandehlou, M. Sami Fadali
2018 arXiv   pre-print
This paper introduces a new method to train recurrent neural networks using dynamical trajectory-based optimization.  ...  The optimization method utilizes a projected gradient system (PGS) and a quotient gradient system (QGS) to determine the feasible regions of an optimization problem and search the feasible regions for  ...  Sami Fadali is with the University of Nevada, Reno, NV, 89557 USA (e-mail: fadali@unr.edu) Training Recurrent Neural Networks via Dynamical Trajectory-Based Optimization to train recurrent neural networks  ... 
arXiv:1805.04152v1 fatcat:jljulygxnfdntk5v2phimlv6pu

Attack and Defense of Dynamic Analysis-Based, Adversarial Neural Malware Detection Models

Jack W. Stokes, De Wang, Mady Marinescu, Marc Marino, Brian Bussone
2018 MILCOM 2018 - 2018 IEEE Military Communications Conference (MILCOM)  
To overcome this limitation, anti-malware companies typically perform dynamic analysis by emulating each file in the anti-malware engine or performing indepth scanning in a virtual machine.  ...  Experiments show that with these three defenses, the number of successfully crafted adversarial samples is reduced compared to an unprotected baseline system.  ...  Typically, a malware file is packed, or encrypted, to make it difficult to reverse engineer by malware analysts.  ... 
doi:10.1109/milcom.2018.8599855 dblp:conf/milcom/StokesWMMB18 fatcat:t6geeeco2fa5vd7r4zi267jby4

Attack and Defense of Dynamic Analysis-Based, Adversarial Neural Malware Classification Models [article]

Jack W. Stokes, De Wang, Mady Marinescu, Marc Marino, Brian Bussone
2017 arXiv   pre-print
To overcome this limitation, anti-malware companies typically perform dynamic analysis by emulating each file in the anti-malware engine or performing in-depth scanning in a virtual machine.  ...  Experiments show that with these three defensive strategies, the number of successfully crafted adversarial samples is reduced compared to a standard baseline system without any defenses.  ...  Hu and Tan [18] propose a generative adversarial network (GAN) to create adversarial malware samples.  ... 
arXiv:1712.05919v1 fatcat:rbwgkq22mvetbczgvm7tyey4bq

Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks

David Sussillo, Omri Barak
2013 Neural Computation  
Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships between time-varying inputs and outputs with complex temporal dependencies.  ...  Linearization around these slow regions can be used to explore, or reverse-engineer, the behavior of the RNN.  ...  in reverse engineering the RNN.  ... 
doi:10.1162/neco_a_00409 pmid:23272922 fatcat:bvsawhntaffudmapfzf3srr7qm

Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time [article]

Zahra Monfared, Daniel Durstewitz
2020 arXiv   pre-print
Recurrent neural networks (RNN) as used in machine learning are commonly formulated in discrete time, i.e. as recursive maps.  ...  On the other hand, mathematical analysis of dynamical systems inferred from data is often more convenient and enables additional insights if these are formulated in continuous time, i.e. as systems of  ...  Synthesis of recurrent neural networks for dynamical system simulation. Neural Networks, 80:67-78, 2016. Vlachas, P., Byeon, W., Wan, Z., Sapsis, T., and Koumoutsakos, P.  ... 
arXiv:2007.00321v1 fatcat:64inh5p4vncrnav5aobadcu2d4

Universal Differential Equations for Scientific Machine Learning [article]

Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, Alan Edelman
2021 arXiv   pre-print
In this manuscript we introduce the SciML software ecosystem as a tool for mixing the information of physical laws and scientific models with data-driven machine learning approaches.  ...  This funnels the wide variety of SciML applications into a core set of training mechanisms which are highly optimized, stabilized for stiff equations, and compatible with distributed parallelism and GPU  ...  This allows for common deep architectures, such as convolutional neural networks and recurrent neural networks, to be efficiently used as the basis for a UDE without any Jacobians being calculated in the  ... 
arXiv:2001.04385v4 fatcat:megacwjw2fbwxkwcnxw7wploia

Review of the applications of neural networks in chemical process control — simulation and online implementation

Mohamed Azlan Hussain
1999 Artificial Intelligence in Engineering  
As a result of good modeling capabilities, neural networks have been used extensively for a number of chemical engineering applications such as sensor data analysis, fault detection and nonlinear process  ...  The review reveals the tremendous prospect of using neural networks in process control.  ...  Macchieto from the Center of Process Systems Engineering, Imperial College, London, for their comments to the paper and for allowing me to use the facilities in the center while doing part of this review  ... 
doi:10.1016/s0954-1810(98)00011-9 fatcat:3l2exdspx5d2ngy6qgxse4gnry

Networks of piecewise linear neural mass models [article]

S Coombes, Y-M Lai, M Sayli, R Thul
2018 arXiv   pre-print
With the simple restriction of the classic sigmoidal nonlinearity to a piecewise linear caricature we show that the famous Wilson-Cowan neural mass model can be analysed at both the node and network level  ...  Here we report on the use of ideas originally developed for the study of Glass networks to treat the stability of periodic network states in neural mass models with discontinuous interactions.  ...  A piecewise linear Wilson-Cowan network The study of coupled oscillator networks in biology, physics, and engineering is now commonplace.  ... 
arXiv:1801.08366v1 fatcat:3avsga5xfbh3nld35ddx5o4sa4

A review of adaptive neural control applied to proton exchange membrane fuel cell systems

Christophe Lin-Kwong-Chon, Brigitte Grondin-Pérez, Jean-Jacques Amangoua Kadjo, Cédric Damour, Michel Benne
2019 Annual Reviews in Control  
In this context, neural networks-based controllers with adaptive learning appear to be especially appropriate methods for system state of health consideration.  ...  For this reason, this paper aims to bring a literature review for adaptive neural-based control applied on proton exchange membrane fuel cell systems.  ...  The employed linear model is obtained from linearization of the ninth-order dynamic model [156] and neural weights are adjusted by BP method.  ... 
doi:10.1016/j.arcontrol.2019.03.009 fatcat:b3zxqhsvmfcyll2wrdfmkxx5wi

Networks of piecewise linear neural mass models

S. COOMBES, Y. M. LAI, M. ŞAYLI, R. THUL
2018 European journal of applied mathematics  
With the simple restriction of the classic sigmoidal non-linearity to a piecewise linear caricature, we show that the famous Wilson–Cowan neural mass model can be explicitly analysed at both the node and  ...  Here, we report on the use of ideas originally developed for the study of Glass networks to treat the stability of periodic network states in neural mass models with discontinuous interactions.  ...  They considered a neural mass network model with recurrent excitation, cross-inhibition, adaptation, and synaptic depression and showed that the use of a Heaviside non-linearity allowed the explicit calculation  ... 
doi:10.1017/s0956792518000050 fatcat:alvrd6oyxban3iaahy5a3wphui

Interpreting Recurrent Neural Networks Behaviour via Excitable Network Attractors

Andrea Ceni, Peter Ashwin, Lorenzo Livi
2019 Cognitive Computation  
Results and Discussion: As the behaviour of recurrent neural networks depends both on training and on inputs to the system, we introduce an algorithm to extract network attractors directly from the trajectory  ...  Methods: Our research aims to open the black-box of recurrent neural networks, an important family of neural networks used for processing sequential data.  ...  Acknowledgements We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.  ... 
doi:10.1007/s12559-019-09634-2 fatcat:k4l5iqd6znfknl2hiogbtfzcuy

2021 Index IEEE Transactions on Neural Networks and Learning Systems Vol. 32

2021 IEEE Transactions on Neural Networks and Learning Systems  
., TNNLS March 2021 1096-1109 Discrete systems Outlier-Resistant Remote State Estimation for Recurrent Neural Networks With Mixed Time-Delays.  ...  Neural-Network-Based Event-Triggered Adaptive Control of Nonaffine Nonlinear Multiagent Systems With Dynamic Uncertainties.  ...  Image coding Deep Multiscale Detail Networks for Multiband Spectral Image Sharpening.  ... 
doi:10.1109/tnnls.2021.3134132 fatcat:2e7comcq2fhrziselptjubwjme
« Previous Showing results 1 — 15 out of 334 results