3,174 Hits in 6.0 sec

A sparse matrix approach to reverse mode automatic differentiation in Matlab

Shaun A. Forth, Naveen Kr. Sharma
2010 Procedia Computer Science  
We detail a Matlab operator overloaded approach to construct the extended Jacobian that enables the function Jacobian to be computed using Matlab's sparse matrix operations.  ...  We review the extended Jacobian approach to automatic differentiation of a user-supplied function and highlight the Schur complement form's forward and reverse variants.  ...  In this article we investigate whether we might use Matlab's sparse matrix features to effect reverse mode AD without recourse to the usual tape-based mechanisms [1, Chap 6.1].  ... 
doi:10.1016/j.procs.2010.04.208 fatcat:wdxud5emyjbp5oxxjgrw3q4ela

ADMIT-1: automatic differentiation and MATLAB interface toolbox

Thomas F. Coleman, Arun Verma
2000 ACM Transactions on Mathematical Software  
Given a function to be differentiated, ADMIT-1 will exploit sparsity if present to yield sparse derivative matrices (in sparse MATLAB form).  ...  ADMIT-1 enables the computation of sparse Jacobian and Hessian matrices, using automatic differentiation technology, from a MATLAB environment.  ...  Automatic Differentiation has two basic modes of operations, the forward mode and the reverse mode.  ... 
doi:10.1145/347837.347879 fatcat:hzrqc665zbd33mhk53jpc5hcaa

An efficient overloaded implementation of forward mode automatic differentiation in MATLAB

Shaun A. Forth
2006 ACM Transactions on Mathematical Software  
Additionally by internally using a matrix (two-dimensional) representation of arbitrary dimension directional derivatives we may utilise MATLAB's sparse matrix class to propagate sparse directional derivatives  ...  The underlying algorithm is the well-known forward mode of automatic differentiation implemented via operator overloading on variables of the class fmad.  ...  We made just two changes to the supplied MATLAB code to enable automatic differentiation.  ... 
doi:10.1145/1141885.1141888 fatcat:7h6ckafgnfaodkxjqgqfy6c2r4

Source Transformation for MATLAB Automatic Differentiation [chapter]

Rahul V. Kharche, Shaun A. Forth
2006 Lecture Notes in Computer Science  
We present MSAD, a source transformation implementation of forward mode automatic differentiation for MATLAB.  ...  Compared to MAD, results from several test cases demonstrate significant improvement in efficiency across all problem sizes.  ...  Automatic Differentiation in MATLAB MATLAB is popular for rapid prototyping and numerical computing owing to its high-level abstraction of matrices and its rich set of function and GUI libraries.  ... 
doi:10.1007/11758549_77 fatcat:jyltz6bgknffhartinx2cnk5ai

Differentiation of the Cholesky decomposition [article]

Iain Murray
2016 arXiv   pre-print
The symbolic and algorithmic approaches can be combined to get the best of both worlds.  ...  We recommend new 'blocked' algorithms, based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library, which uses 'Level 3' matrix-matrix operations from BLAS, and so is cache-friendly and  ...  Reverse-mode differentiation: Reverse mode automatic differentiation traverses an algorithm backwards, reversing the direction of loops and the updates within them.  ... 
arXiv:1602.07527v1 fatcat:2mwd4a4n7zbrdpzaod63wyheei

An efficient overloaded method for computing derivatives of mathematical functions in MATLAB

Michael A. Patterson, Matthew Weinstein, Anil V. Rao
2013 ACM Transactions on Mathematical Software  
The method implements forward mode automatic differentiation via operator overloading in a manner that produces a new MATLAB code which computes the derivatives of the outputs of the original function  ...  A detailed description of the method is presented and the approach is illustrated and is shown to be efficient on four examples. ACM Reference Format: Patterson, M. A., Weinstein, M., and Rao, A.  ...  In this paper we present a new approach for automatic differentiation of MATLAB code.  ... 
doi:10.1145/2450153.2450155 fatcat:2yiakqzudnf7bom2mz34ewbazu

A benchmark of selected algorithmic differentiation tools on some problems in computer vision and machine learning

Filip Srajer, Zuzana Kukelova, Andrew Fitzgibbon
2018 Optimization Methods and Software  
In this paper, we compare fifteen ways of computing derivatives including eleven automatic differentiation tools implementing various methods and written in various languages (C++, F#, MATLAB, Julia and  ...  However, we contend that this paper presents an important datapoint: a skilled programmer devoting roughly a week to each tool produced the timings we present.  ...  We thank Jonathan Taylor for an example implementation of a hand tracking function in Python. Funding Zuzana Kukelova was supported by The Czech Science Foundation Project GACR P103/12/G084.  ... 
doi:10.1080/10556788.2018.1435651 fatcat:yd57h6bfhfds5krnrazqsfyiq4

Implementation of sparse forward mode automatic differentiation with application to electromagnetic shape optimization

Jukka I. Toivanen, Raino A. E. Mäkinen
2011 Optimization Methods and Software  
In this paper we present the details of a simple lightweight implementation of so called sparse forward mode automatic differentiation (AD) in the C++ programming language.  ...  It is shown that the use of the sparse forward mode can save computation time even though the total number of independent variables in this example is quite small.  ...  The authors would also like to thank Dr. Jussi Rahola for introducing us to interesting electromagnetic design problems.  ... 
doi:10.1080/10556781003642305 fatcat:eirwa7amgze2hosyufsxyfn3yq

Efficient Differentiable Programming in a Functional Array-Processing Language [article]

Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, Simon Peyton Jones, Christoph Koch
2018 arXiv   pre-print
We present a system for the automatic differentiation of a higher-order functional array-processing language.  ...  The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations.  ...  Our contributions In this paper, we present a novel automatic differentiation technique based on forward mode, which combines the benefits of both forward and reverse mode in many cases, and which, even  ... 
arXiv:1806.02136v1 fatcat:plpqeyuhzfhvpejbniwbcjyaxq

Algorithm 984

Matthew J. Weinstein, Anil V. Rao
2017 ACM Transactions on Mathematical Software  
A toolbox called ADiGator is described for algorithmically differentiating mathematical functions in MAT-LAB.  ...  ADiGator performs source transformation via operator overloading using forward mode algorithmic differentiation and produces a file that can be evaluated to obtain the derivative of the original function  ...  In this article, a new open-source MATLAB algorithmic differentiation toolbox called ADiGator (Automatic Differentiation by Gators) is described.  ... 
doi:10.1145/3104990 fatcat:dox2rwi7wngc3e573crwix4kw4

Fast (Structured) Newton Computations

Thomas F. Coleman, Wei Xu
2009 SIAM Journal on Scientific Computing  
This auxiliary matrix can be sparse even when the true Jacobian matrix is dense; consequently, sparse matrix technology can be used, to great speed advantage, both in forming the auxiliary matrix and in  ...  It is often not necessary, nor economic, to form the true Jacobian in the process of computing the Newton step; instead, a more cost-effective auxiliary Jacobian matrix is used.  ...  Specifically, the Jacobian matrix J is formed by differentiating F using the forward-mode automatic differentiation (AD) (equivalent in cost to obtaining J column-by-column using forward finite-differences  ... 
doi:10.1137/070701005 fatcat:prqtkmyy5ncntmjwgmf5rbg6nq

Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming

Richard D. Neidinger
2010 SIAM Review  
A survey of more advanced topics in automatic differentiation includes an introduction to the reverse mode (our implementation is forward mode) and considerations in arbitrary-order multivariable series  ...  An introduction to both automatic differentiation and object-oriented programming can enrich a numerical analysis course that typically incorporates numerical differentiation and basic MATLAB computation  ...  Exactly where and how these are written can vary, from a block of program code to storage on a tape or to entries in a sparse extended Jacobian matrix.  ... 
doi:10.1137/080743627 fatcat:ug4rpqbgtzhtpepbmoh4vxgssu

ADjoint: An Approach for the Rapid Development of Discrete Adjoint Solvers

C. A. Mader, J. R. R. A. Martins, J. J. Alonso, E. Van Der Weide
2008 AIAA Journal  
An automatic differentiation tool is used to develop the adjoint code for a three-dimensional computational fluid dynamics solver.  ...  This selective application of automatic differentiation is the central idea behind the automatic differentiation adjoint (ADjoint) approach.  ...  The stencil-based approach, in conjunction with the reverse mode of automatic differentiation resulted in a very efficient flux Jacobian computation.  ... 
doi:10.2514/1.29123 fatcat:k4vrrjc4ubhotlnauotyogga6i

Blind separation of sparse sources with relative Newton method

Michael Zibulevsky, Michael A. Unser, Akram Aldroubi, Andrew F. Laine
2003 Wavelets: Applications in Signal and Image Processing X  
We demonstrate the efficiency of the presented approach on example of sparse sources. The nonlinearity in this case is based on smooth approximation of the absolute value function.  ...  Sequential optimization with the gradual reduction of the smoothing parameter leads to the super-efficient separation.  ...  In order to guarantee descent direction in the case of nonconvex objective function, we use modified Cholessky factorization 1 [21] , which automatically finds such a diagonal matrix Ê, that the matrix  ... 
doi:10.1117/12.505053 fatcat:mk7hmijjdrai5c2kh57aebiyzi

Efficient differentiable programming in a functional array-processing language

Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, Simon Peyton Jones
2019 Proceedings of the ACM on Programming Languages (PACMPL)  
We present a system for the automatic differentiation (AD) of a higher-order functional array-processing language.  ...  In combination, gradient computation with forward-mode AD can be as efficient as reverse mode, and that the Jacobian matrices required for numerical algorithms such as Gauss-Newton and Levenberg-Marquardt  ...  forward-mode automatic differentiation.  ... 
doi:10.1145/3341701 fatcat:fdz6kt3rera4ppiibekg6kbcr4
« Previous Showing results 1 — 15 out of 3,174 results