A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
On the Maximum Hessian Eigenvalue and Generalization
[article]
2022
arXiv
pre-print
Preserved Fulltext
Other works question the link between λ_max and generalization. In this paper, we present findings that call λ_max's influence on generalization further into question. ...
of the Hessian of the loss); and algorithms, such as Sharpness-Aware Minimization (SAM) [1], that directly optimize for flatness. ...
In this paper, we focus exclusively on the leading Hessian eigenvalue λ max and its relationship to generalization. λ max as a metric for flatness The maximum Hessian eigenvalue λ max is commonly viewed ...
arXiv:2206.10654v1
fatcat:bd4l3cpfgrc63c3nbqxzlxadtu
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
by small eigenvalues of the Hessian of the training loss. ...
in wider regions visited), the overall training speed, and the generalization ability of the final model. ...
10 eigenvalues of the Hessian for SimpleCNN and Resnet-32 trained on the CIFAR-10 dataset with η = 0.1 and S = 128. ...
arXiv:1807.05031v6
fatcat:vyhiz37wh5de7mlbekuhtnjvba
Adaptable Center Detection of a Laser Line with a Normalization Approach using Hessian-matrix Eigenvalues
2014
Journal of the Optical Society of Korea
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The Gaussian recognition function estimates the characteristic that one eigenvalue approaches zero, and enhances the sensitivity of the decision function to that characteristic, which corresponds to the ...
Second, the feature points in an integral pixel level are selected as the initiating light line centers by the eigenvalues of the Hessian matrix. ...
Then the eigenvalues and eigenvectors of the Hessian matrix are calculated after obtaining the Hessian matrix of each pixel on the laser line. ...
doi:10.3807/josk.2014.18.4.317
fatcat:e5onyin5frgkbls4s7he2msw7e
On the saddle point problem for non-convex optimization
[article]
2014
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Here we argue, based on results from statistical physics, random matrix theory, and neural network theory, that a deeper and more profound difficulty originates from the proliferation of saddle points, ...
Such saddle points are surrounded by high error plateaus that can dramatically slow down learning, and give the illusory impression of the existence of a local minimum. ...
and CIFAR. ...
arXiv:1405.4604v2
fatcat:xnjcqly4mvbj5cn544fldwk5n4
Maximizing the hyperpolarizability poorly determines the potential
[article]
2011
arXiv
pre-print
Preserved Fulltext
The Internet Archive has a preservation copy of this work in our general collections.
The file type is application/pdf.
Other Versions
The Hessian of \beta at the maximum has in each case only two large eigenvalues; the other eigenvalues diminish seemingly exponentially quickly, demonstrating a very wide range of nearby nearly optimal ...
potentials, and that there are only two important parameters for optimizing \beta. ...
In each case, we optimized β int and calculated the eigenvalues and eigenvectors of the Hessian at the maximum. ...
arXiv:1010.4919v2
fatcat:k6ewoilwcfdjbndtxhkfutmizu
Experimental observation of saddle points over the quantum control landscape of a two-spin system
2015
Physical Review A. Atomic, Molecular, and Optical Physics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
We address the saddles with a combined theoretical and experimental approach, measure the Hessian at each identified saddle point, and study how their presence can influence the search effort utilizing ...
Theoretical analyses have predicted the existence of critical points over the landscapes, including saddle points with indefinite Hessians. ...
Another important conclusion is the low rank of the Hessian at critical points, i.e., there exists a specific maximum number of positive and negative eigenvalues dependent on the nature of the quantum ...
doi:10.1103/physreva.91.043412
fatcat:uvxcgdikyrf7jhefgqyf6kthvi
Matrix conditioning and adaptive simultaneous perturbation stochastic approximation method
2001
Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2004; you can also visit the original URL.
The file type is application/pdf.
This paper proposes a modification to the simultaneous perturbation stochastic approximation (SPSA) methods based on the comparisons made between the first order and the second order SPSA (1SPSA and 2SPSA ...
At finite iterations, the convergence rate depends on the matrix conditioning of the loss function Hessian. ...
For a symmetric Hessian matrix H with all positive eigenvalues, its condition number with respect to the spectral norm (~2) is the ratio of the maximum eigenvalue to the minimum one (Horn and Johnson, ...
doi:10.1109/acc.2001.945918
fatcat:fl3gzjj6qjeb3l7b2fw4svubzu
Strategies for walking on potential energy surfaces using local quadratic approximations
1990
International Journal of Quantum Chemistry
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
This method utilizes local gradient and Hessian (Le., fust and second energy derivative) information to generate a series of "steps" that are folIowed to the desired stationary point. ...
By stepping uphill along one local eigenmode of the Hessian wIDIe minimizing the energy along all other modes, one )ocates transition states. ...
Acknowledgments This wark was supported in part by the Office of Naval Research and by NSF Grant No. 8814765; ...
doi:10.1002/qua.560382427
fatcat:yo5ctkl46restj5vqwxrpx3rru
Search for stationary points on surfaces
1985
The Journal of Physical Chemistry
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2007; you can also visit the original URL.
The file type is application/pdf.
The data indicate that the S2ion of Na2S or H2S is strongly adsorbed on the CdS-Nafion surface and is oxidized to sulfate (Figures 4 and 6) without irradiation. Conclusions conclusions: is small. ...
with cation exchange sites. (4) Adsorbed sulfide ions on Nafion and CdS-Nafion are oxidized to sulfate ions at 300 K in the presence of oxygen. (5) The gray-blue deposit formed on cubic CdS-Nafion surfaces ...
We also acknowledge the financial support of the National Science Foundation (Grant 8206845) and the donors of the Petroleum Research Fund, administered by the American Chemical Society. ...
doi:10.1021/j100247a015
fatcat:y6zo3o6r6zc7tbwfqifo44t6oa
Characterizing the loss landscape of variational quantum circuits
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
The eigenvalues of the Hessian give information on the local curvature and we discuss how this information can be interpreted and compared to classical neural networks. ...
We benchmark our result on several examples, starting with a simple analytic toy model to provide some intuition about the behavior of the Hessian, then going to bigger circuits, and also train VQCs on ...
II C, we study the Hessian and its eigenvalues on an analytical example of a VQC. We show how to calculate the Hessian on an actual quantum hardware in Sec. IV, apply it to a general example in Sec. ...
arXiv:2008.02785v1
fatcat:z3gow6fenjblpp27tkfn2eozfy
Molecular Embedding via a Second Order Dissimilarity Parameterized Approach
2009
SIAM Journal on Scientific Computing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2009; you can also visit the original URL.
The file type is application/pdf.
The nonconvexity arises due to matrix rank constraints in the problem, and we focus on their efficient computational treatment. ...
We present numerical results for a number of synthetic and real protein data sets and comment on features of real experimental data that can cause computational difficulties. ...
The authors thank the reviewers of this paper; their comments have greatly improved the paper. ...
doi:10.1137/070688547
fatcat:7wq4civvlzez7a4crtjkvi55fy
Selecting the Metric in Hamiltonian Monte Carlo
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2019 pre-print arXiv:1905.11916v2 fatcat:fdbrugmgwrg57bu7ucoe53ehcyOther Versions
The effectiveness of the selection criterion and adaptation are demonstrated on a number of applied problems. An implementation for the Stan probabilistic programming language is provided. ...
and the availability of warmup draws. ...
For a normal posterior, Eq. 9 and Eq. 13 are equivalent and the smallest Hessian eigenvalue, λ min , corresponds to the largest covariance eigenvalue. ...
arXiv:1905.11916v3
fatcat:ovmikfyb35bedf3eexegk6w3ii
The Information Content of the Elastic Reflection Matrix
1996
Geophysical Journal International
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
These are the parameters associated with the eigenvectors of the Hessian matrix (or the normalized Fisher information matrix) corresponding to the smallest eigenvalues. ...
S U M M A R Y The P-SV reflection matrix for a plane interface between two elastic media depends on the density, P-wave velocity, and S-wave velocity of the two media. ...
The Hessian matrix depends on maximum slowness, the type of reflection coefficients, and the earth model used. ...
doi:10.1111/j.1365-246x.1996.tb06547.x
fatcat:h7oqoe6wmjaija2ajeauhytbli
Multiscale vessel enhancement filtering
[chapter]
1998
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2003; you can also visit the original URL.
The file type is application/pdf.
A vesselness measure is obtained on the basis of all eigenvalues of the Hessian. This measure is tested on two dimensional DSA and three dimensional aortoiliac and cerebral MRA data. ...
Its clinical utility is shown by the simultaneous noise and background suppression and vessel enhancement in maximum intensity projections and volumetric displays. ...
Theo van Walsum, Onno Wink and Joes Staal for fruitful discussions and various contributions to the paper. ...
doi:10.1007/bfb0056195
fatcat:tqtoxxal7jebzi4b3rxyxsfhyq
A Strain Energy Filter for 3D Vessel Enhancement
[chapter]
2010
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Moreover, a mathematical description of Hessian eigenvalues for general vessel shapes is obtained, based on an intensity continuity assumption, and a relative Hessian strength term is presented to ensure ...
The proposed method is validated in experiments with a digital phantom and non-contrast-enhanced pulmonary CT data. ...
Acknowledgments This research was funded by STW (Grant LPG.07998) of the Netherlands and by the National Natural Science Foundation of China (Nos. 60835004 and 60871096). ...
doi:10.1007/978-3-642-15711-0_46
fatcat:nk2nl7t7vzg6zkavwfpnrghiba
« Previous
Showing results 1 — 15 out of 27,993 results