27 Hits in 3.3 sec

Hessian-Aware Pruning and Optimal Neural Implant [article]

Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Sehoon Kim, Michael W Mahoney, Kurt Keutzer
2021 arXiv   pre-print
To address this problem, we introduce a new Hessian Aware Pruning (HAP) method coupled with a Neural Implant approach that uses second-order sensitivity as a metric for structured pruning.  ...  The basic idea is to prune insensitive components and to use a Neural Implant for moderately sensitive components, instead of completely pruning them.  ...  As for Neural Implant, we select a fixed neural implant ratio of 0.2, meaning that 20% of the pruned 3x3 convolution kernels are replaced by 1x1 convolution kernels.  ... 
arXiv:2101.08940v3 fatcat:iaett7rizfgmxjsyukpdhhcrqe

The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models [article]

Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
2022 arXiv   pre-print
We introduce Optimal BERT Surgeon (oBERT), an efficient and accurate weight pruning method based on approximate second-order information, which we show to yield state-of-the-art results in both stages  ...  Specifically, oBERT extends existing work on unstructured second-order pruning by allowing for pruning blocks of weights, and by being applicable at the BERT scale.  ...  Hessian-aware pruning and optimal neural implant. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3880-3891.  ... 
arXiv:2203.07259v2 fatcat:5qjip6bwfjhw7pbxuooox2c3ym

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness [article]

Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin
2020 arXiv   pre-print
A high correlation is observed between the adversarial robustness loss and the largest eigenvalue of the input Hessian matrix, for which theoretical justifications are provided.  ...  Mode connectivity provides novel geometric insights on analyzing loss landscapes and enables building high-accuracy pathways between well-trained neural networks.  ...  James Martens and Ilya Sutskever. Training deep and recurrent networks with hessian-free optimization. In Neural networks: Tricks of the trade, pp. 479-535. Springer, 2012.  ... 
arXiv:2005.00060v2 fatcat:dk63su5vsjgbtbymuofhjiwy2a

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks [article]

Tilman Räuker, Anson Ho, Stephen Casper, Dylan Hadfield-Menell
2022 arXiv   pre-print
The last decade of machine learning has seen drastic increases in scale and capabilities, and deep neural networks (DNNs) are increasingly being deployed across a wide range of domains.  ...  Finally, we discuss key challenges and argue for future work in interpretability for AI safety that focuses on diagnostics, benchmarking, and robustness.  ...  ACKNOWLEDGEMENTS We thank Davis Brown and Peter Hase for feedback. Tilman Räuker is supported in part by the Long-Term Future Fund.  ... 
arXiv:2207.13243v2 fatcat:dnwbajsdyjazpdznmsofbzxifq

Artificial neural network algorithm for analysis of Rutherford backscattering data

N. P. Barradas, A. Vieira
2000 Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics  
We have developed an artificial neural network ͑ANN͒ for the same purpose, applied to the specific case of Ge-implanted Si.  ...  This opens the door to automated on-line optimization of the experimental conditions.  ...  CONCLUSIONS As far as we are aware, the artificial neural network algorithm demonstrated here is the first method that can analyze RBS data instantaneously.  ... 
doi:10.1103/physreve.62.5818 pmid:11089142 fatcat:ne2zoh63ubhglnus4r3ysprhcy

Applications and Techniques for Fast Machine Learning in Science

Allison McCarn Deiana, Nhan Tran, Joshua Agar, Michaela Blott, Giuseppe Di Guglielmo, Javier Duarte, Philip Harris, Scott Hauck, Mia Liu, Mark S. Neubauer, Jennifer Ngadiuba, Seda Ogrenci-Memik (+35 others)
2022 Frontiers in Big Data  
training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms.  ...  This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions.  ...  W., and Keutzer, K. (2021). Hessian-aware pruning and optimal neural implant. arXiv preprint arXiv:2101.08940.  ... 
doi:10.3389/fdata.2022.787421 pmid:35496379 pmcid:PMC9041419 fatcat:5w2exf7vvrfvnhln7nj5uppjga

Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes [article]

Sanghyun Hong, Michael-Andrei Panaitescu-Liess, Yiğitcan Kaya, Tudor Dumitraş
2021 arXiv   pre-print
To study this hypothesis, we weaponize quantization-aware training and propose a new training framework to implement adversarial quantization outcomes.  ...  Quantization is a popular technique that transforms the parameter representation of a neural network from floating-point numbers into lower-precision ones (e.g., 8-bit integers).  ...  Acknowledgments and Disclosure of Funding We thank Tom Goldstein and the anonymous reviewers for their constructive feedback.  ... 
arXiv:2110.13541v2 fatcat:hzacgl6dujgnzjhfamwax77anm

Segmentation of Vasculature from Fluorescently Labeled Endothelial Cells in Multi-Photon Microscopy Images

Russell Bates, Benjamin Irving, Bostjan Markelc, Jakob Kaeppler, Graham Brown, Ruth J. Muschel, Sir Michael Brady, Vicente Grau, Julia A. Schnabel
2017 IEEE Transactions on Medical Imaging  
As such, considerable effort has been focused on the automated segmentation of vasculature in medical and pre-clinical images.  ...  We show that our method can provide complete and semantically meaningful segmentations of complex vasculature using a supervoxel-Markov random field approach.  ...  All the parameters and thresholds are optimized on a calibration image and held constant over the test set.  ... 
doi:10.1109/tmi.2017.2725639 pmid:28796613 fatcat:aoigqq3ufvenrnf3fyy2ktsjba

Extracting 3D Vascular Structures from Microscopy Images using Convolutional Recurrent Networks [article]

Russell Bates, Benjamin Irving, Bostjan Markelc, Jakob Kaeppler, Ruth Muschel, Vicente Grau, Julia A. Schnabel
2017 arXiv   pre-print
In order to address this we propose a method to directly extract a medial representation of the vessels using Convolutional Neural Networks.  ...  As such, considerable effort has been focused on the automated measurement and analysis of vasculature in medical and pre-clinical images.  ...  This work was also supported by Cancer Research UK (CR-UK) grant numbers C5255/A18085 and C5255/A15935, through the CRUK Oxford Centre and by CRUK/EPSRC Oxford Cancer Imaging Centre (grant number C5255  ... 
arXiv:1705.09597v1 fatcat:63e2zjuknfdy5kblst4f4revyy

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning [article]

Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A. Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, Fabio Roli
2022 arXiv   pre-print
We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly.  ...  Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical  ...  Bajcsy and Majurski [5] follow the same approach, using clean data and a pruned model.  ... 
arXiv:2205.01992v1 fatcat:634zayldxfgfrlucascahjesxm

Threats to Training: A Survey of Poisoning Attacks and Defenses on Machine Learning Systems

Zhibo Wang, Jingjing Ma, Xue Wang, Jiahui Hu, Zhan Qin, Kui Ren
2022 ACM Computing Surveys  
Our ultimate motivation is to provide a comprehensive and self-contained survey of this growing field of research and lay the foundation for a more standardized approach to reproducible studies.  ...  In this survey, we summarize and categorize existing attack methods and corresponding defenses, as well as demonstrate compelling application scenarios, thus providing a unified framework to analyze poisoning  ...  For deep neural networks, the bilevel objective has to be approximated and improved.  ... 
doi:10.1145/3538707 fatcat:pcxpqbsrgzgidb7ngrcb5ggeoa

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions

Laith Alzubaidi, Jinglan Zhang, Amjad J. Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al-Shamma, J. Santamaría, Mohammed A. Fadhel, Muthana Al-Amidie, Laith Farhan
2021 Journal of Big Data  
It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with  ...  The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.  ...  Acknowledgements We would like to thank the professors from the Queensland University of Technology and the University of Information Technology and Communications who gave their feedback on the paper.  ... 
doi:10.1186/s40537-021-00444-8 pmid:33816053 pmcid:PMC8010506 fatcat:x2h5qs5c2jbntipu7oi5hfnb6u

A Roadmap for Big Model [article]

Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han (+88 others)
2022 arXiv   pre-print
and Application.  ...  Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields.  ...  This indicates that big models can be overparameterized, and there is much room for optimization. 2. Model Pruning.  ... 
arXiv:2203.14101v4 fatcat:rdikzudoezak5b36cf6hhne5u4

CARS 2016—Computer Assisted Radiology and Surgery Proceedings of the 30th International Congress and Exhibition Heidelberg, Germany, June 21–25, 2016

2016 International Journal of Computer Assisted Radiology and Surgery  
'', and Amazon Inc., for providing valuable computing resources through an ''AWS in Education Research'' grant.  ...  References Acknowledgments This work was supported by projects IPT-2012-0401-300000, TEC2013-48251-C2-1-R, DTS14/00192, PI15/02121, EU FP7 IRSES TAHITI (#269300) and FEDER funds.  ...  understand the effect of the inaccuracies arising in the workflow and to optimize future implant design.  ... 
doi:10.1007/s11548-016-1412-5 pmid:27206418 fatcat:uk5r46n2xvhedkfjzmeiweyneq

Report on Optimal Substructure Techniques for Stellar, Gas and Combined Samples [article]

I. Joncour, A. Buckner, P. Khalaj, E. Moraux, F. Motte
2020 arXiv   pre-print
It is the deliverable: Report on Optimal Substructure Techniques for Stellar, Gas and Combined Samples, for the EU H2020 (COMPET-5-2015 - Space) project (A Gaia and Herschel Study of the Density Distribution  ...  Summary and Conclusions  ...  Moreover, CUPID (Section 3.2.1) includes a re-implantation of this algorithm.  ... 
arXiv:2006.07830v1 fatcat:x7nsxwqorjdhzouele7vdspwv4
« Previous Showing results 1 — 15 out of 27 results