14,257 Hits in 4.8 sec

Fast Reverse-Mode Automatic Differentiation using Expression Templates in C++

Robin J. Hogan
2014 ACM Transactions on Mathematical Software  
An implementation is freely available in the Adept C++ software library. ACM Reference Format: Robin J. Hogan. 2014. Fast reverse-mode automatic differentiation using expression templates in C++. ACM  ...  In this paper a fast new operator-overloading method is presented that uses the expression template programming technique in C++ to provide a compiletime representation of each mathematical expression  ...  They have also been used to accelerate the differentiation of individual expressions via "expression-level reverse mode" within an overall forward-mode automatic differentiation framework [Phipps and  ... 
doi:10.1145/2560359 fatcat:x7miqimrq5deta45vmfwdvtkqm

Manifold Geometry with Fast Automatic Derivatives and Coordinate Frame Semantics Checking in C++ [article]

Leonid Koppel, Steven L. Waslander
2018 arXiv   pre-print
To support rapid and error-free development, we present wave_geometry, a C++ manifold geometry library with two key contributions: expression template-based automatic differentiation and compile-time enforcement  ...  We contrast the library with existing open source packages and show that it can evaluate Jacobians in forward and reverse mode with little to no runtime overhead compared to hand-coded derivatives.  ...  AUTOMATIC DIFFERENTIATION In this section, we explore three AD implementations built on the expression template scaffolding.  ... 
arXiv:1805.01810v1 fatcat:oqflbwokgrcpfmi6zaxjciilqi

A review of automatic differentiation and its efficient implementation

Charles C. Margossian
2019 Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery  
Among these techniques are operation overloading, region based memory, and expression templates.  ...  Derivatives play a critical role in computational statistics, examples being Bayesian inference using Hamiltonian Monte Carlo sampling and the training of neural networks.  ...  the expression graph in memory in forward than in reverse mode.  ... 
doi:10.1002/widm.1305 fatcat:sr2zebiuwbfaro62ogofhwo46y

On the implementation of automatic differentiation tools

Christian H. Bischof, Paul D. Hovland, Boyana Norris
2008 Higher-Order and Symbolic Computation  
We describe the forward and reverse modes of automatic differentiation and provide a survey of implementation strategies.  ...  We describe some of the challenges in the implementation of automatic differentiation tools, with a focus on tools based on source transformation.  ...  The complete code for foo and func using the forward and reverse modes is included in the Appendix.  ... 
doi:10.1007/s10990-008-9034-4 fatcat:zwfoedfaozgtxn7fanpc5uyvxe

Implementation of automatic differentiation tools

Christian H. Bischof, Paul D. Hovland, Boyana Norris
2002 Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulation - PEPM '02  
We describe the forward and reverse modes of automatic differentiation and provide a survey of implementation strategies.  ...  We describe some of the challenges in the implementation of automatic differentiation tools, with a focus on tools based on source transformation.  ...  The complete code for foo and func using the forward and reverse modes is included in the Appendix.  ... 
doi:10.1145/503046.503047 fatcat:lpdzvsgmjvb4zlyjttirqmh3se

TMB: Automatic Differentiation and Laplace Approximation

Kasper Kristensen, Anders Nielsen, Casper W. Berg, Hans Skaug, Bradley M. Bell
2016 Journal of Statistical Software  
This approximation, and its derivatives, are obtained using automatic differentiation (up to order three) of the joint likelihood.  ...  The user defines the joint likelihood for the data and the random effects as a C++ template function, while all the other operations are done in R; e.g., reading in the data.  ...  Eigen's use of templates enables matrix calculations where corresponding scalar types can do automatic differentiation. These features are important in the implementation and use of TMB.  ... 
doi:10.18637/jss.v070.i05 fatcat:2d3ti2biuzanzcxn5zoav6yy3e

A backward automatic differentiation framework for reservoir simulation

Xiang Li, Dongxiao Zhang
2014 Computational Geosciences  
In numerical reservoir simulations, Newton's method is a concise, robust and, perhaps the most commonly used method to solve nonlinear partial differential equations (PDEs).  ...  This paper presents an automatic differentiation (AD) framework that is specially designed for simplifying coding and simultaneously maintaining computational efficiency.  ...  + library [21, 22] , in which the forward mode is realized by expression templates; and ADEPT-a backward mode C++ library based on expression templates [15] .  ... 
doi:10.1007/s10596-014-9441-z fatcat:x2y4ueixijgkzphsh2pzs26nmy

Automatic differentiation of rigid body dynamics for optimal control and estimation

Markus Giftthaler, Michael Neunert, Markus Stäuble, Marco Frigerio, Claudio Semini, Jonas Buchli
2017 Advanced Robotics  
Second, we present a hardware experiment in which a 6 DoF robotic arm avoids a randomly moving obstacle in a go-to task by fast, dynamic replanning.  ...  To overcome this issue, we extend the modelling tool 'RobCoGen' to be compatible with Automatic Differentiation.  ...  In Auto-Differentiation, there are two internal modes of building the expression graph: Forward and reverse.  ... 
doi:10.1080/01691864.2017.1395361 fatcat:33hdymegc5cqdaw3tzydyvstge

Large-Scale Transient Sensitivity Analysis of a Radiation-Damaged Bipolar Junction Transistor via Automatic Differentiation [chapter]

Eric T. Phipps, Roscoe A. Bartlett, David M. Gay, Robert J. Hoekstra
2008 Lecture Notes in Computational Science and Engineering  
Automatic differentiation (AD) is useful in transient sensitivity analysis of a computational simulation of a bipolar junction transistor subject to radiation damage.  ...  We used forward-mode AD, implemented in a new Trilinos package called Sacado, to compute analytic derivatives for implicit time integration and forward sensitivity analysis.  ...  This package provides operator overloading for forward, reverse, and Taylor mode automatic differentiation in C++ codes.  ... 
doi:10.1007/978-3-540-68942-3_31 fatcat:bvqyx3ygnfhszfh4ygejoc3anu

Development and First Applications of TAC++ [chapter]

Michael Voßbeck, Ralf Giering, Thomas Kaminski
2008 Lecture Notes in Computational Science and Engineering  
The paper describes the development of the software tool Transformation of Algorithms in C++ (TAC++) for automatic differentiation (AD) of C(++) codes by sourceto-source translation.  ...  TAC++ features forward and reverse as well as scalar and vector modes of AD. Efficient higher order derivative code is generated by multiple application of TAC++.  ...  We enjoyed the joint work with Bernard Pinty and Thomas Lavergne on the two-stream inversion package and with Mike Giles on differentiation of the LIBOR code.  ... 
doi:10.1007/978-3-540-68942-3_17 fatcat:mxh44o25qbblza7hry5jikasfu

High-Performance Derivative Computations using CoDiPack [article]

Max Sagebaum, Tim Albring, Nicolas R. Gauger
2017 arXiv   pre-print
Especially for Jacobi taping, recent advances by using expression templates make this approach very attractive for large scale software.  ...  There are several AD tools available, which all implement different strategies for the reverse mode of AD.  ...  The implementation will be the AD reverse mode of the value method in the MULT expression example.  ... 
arXiv:1709.07229v1 fatcat:ko7odqzrfreire6ippz4rmtcsu

Automatic differentiation in machine learning: a survey [article]

Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind
2018 arXiv   pre-print
By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation  ...  derivatives of numeric functions expressed as computer programs.  ...  expressed or implied, of the sponsors.  ... 
arXiv:1502.05767v4 fatcat:wlm4yhmccjcapn6d3hfbo5ixri

Universal Numbers Library: design and implementation of a high-performance reproducible number systems library [article]

E. Theodore L. Omtzigt, Peter Gottschling, Mark Seligman, William Zorn
2020 arXiv   pre-print
We will highlight the integration of the library in larger application environments in computational science and engineering to enable multi-precision and adaptive precision algorithms to improve performance  ...  The library implementation is described, and examples are provided to show educational examples to elucidate the number system properties, and how specialization is used to yield very high-performance  ...  In a reverse mode automatic differentiation algorithm, the output variable of a function is evaluated first.  ... 
arXiv:2012.11011v1 fatcat:jtkrzjbkqbbivkdtymthnmhk6a

AutoMat – Automatic Differentiation for Generalized Standard Materials on GPUs [article]

Johannes Blühdorn, Nicolas R. Gauger, Matthias Kabel
2020 arXiv   pre-print
By moving AutoMat to the GPU, we close the performance gap to conventional evaluation routines and demonstrate in detail that the expression level reverse mode of automatic differentiation as well as its  ...  By means of automatic differentiation and a numerical integration scheme, AutoMat reduces the implementation effort to two potential functions.  ...  Kabel thanks Quentin Horn for setting up the GPU benchmarking framework of AutoMat during his internship at Fraunhofer ITWM.  ... 
arXiv:2006.04391v2 fatcat:tznuwrumajayzhs7kl3sv6jbtm

GPU-accelerated adjoint algorithmic differentiation

Felix Gremse, Andreas Höfter, Lukas Razik, Fabian Kiessling, Uwe Naumann
2016 Computer Physics Communications  
Many scientific problems such as classifier training or medical image reconstruction can be expressed as minimization of differentiable real-valued cost functions and solved with iterative gradient-based  ...  The vectorization allowed usage of optimized parallel libraries during forward and reverse passes which resulted in high speedups for the vectorized CPU version compared to the naive reference implementation  ...  There are other approaches that perform on line pruning during creation of the tape [15] or use C++ expression templates to combine a whole statement into one taped operation [11, 12] .  ... 
doi:10.1016/j.cpc.2015.10.027 pmid:26941443 pmcid:PMC4772124 fatcat:pnyxd3butjdohcxbbib7nvctju
« Previous Showing results 1 — 15 out of 14,257 results