5,172 Hits in 2.6 sec

Reverse-Engineering Deep ReLU Networks [article]

David Rolnick, Konrad P. Kording
2020 arXiv   pre-print
Here, we prove that in fact it is often possible to identify the architecture, weights, and biases of an unknown deep ReLU network by observing only its output.  ...  Every ReLU network defines a piecewise linear function, where the boundaries between linear regions correspond to inputs for which some neuron in the network switches between inactive and active ReLU states  ...  queries. • We demonstrate the success of our algorithm in reverse-engineering both trained and untrained ReLU networks.  ... 
arXiv:1910.00744v2 fatcat:x2rms7hvebcrnncwyucuynldpi

Reverse Engineering the Neural Tangent Kernel [article]

James B. Simon, Sajant Anand, Michael R. DeWeese
2022 arXiv   pre-print
The development of methods to guide the design of neural networks is an important open challenge for deep learning theory.  ...  We verify our construction numerically and demonstrate its utility as a design tool for finite fully-connected networks in several experiments.  ...  SA gratefully acknowledges support of the Department of Defense (DoD) through the National Defense Science and Engineering Graduate (NDSEG) Fellowship Program.  ... 
arXiv:2106.03186v4 fatcat:gqrnfhurszenphmib7oggbv2ue

Rectifier Neural Network with a Dual-Pathway Architecture for Image Denoising [article]

Keting Zhang, Liqing Zhang
2020 arXiv   pre-print
Recently deep neural networks based on tanh activation function have shown their impressive power in image denoising.  ...  In this letter, we try to use rectifier function instead of tanh and propose a dual-pathway rectifier neural network by combining two rectifier neurons with reversed input and output weights in the same  ...  In this letter, we study the use of rectifier function in deep neural network for image denoising.  ... 
arXiv:1609.03024v3 fatcat:tiu5r5h3and2bb6sssvtnwkntq


Anushka Chauhan, Aman Rastogi, Agrima Gaur, Anugrah Singh, Shaili Gupta
2020 International Journal of Engineering Applied Sciences and Technology  
Convolutional Neural Networks mostly use deep learning algorithms to detect and identify traffic signs till now but they are lacking in so many ways.  ...  The filters which are there in primitive methods are engineered manually with training. These filters are learned by the ConvNets.  ...  Shaili Gupta, Assistant Professor, IMS Engineering College, Ghaziabad, India for her patient guidance and constructive suggestions for the research in the area of traffic sign detection using deep learning  ... 
doi:10.33564/ijeast.2020.v05i01.057 fatcat:ey7hbelbzrapphhrzl7ralsozq

Distributed Layer-Partitioned Training for Privacy-Preserved Deep Learning [article]

Chun-Hsien Yu, Chun-Nan Chou, Emily Chang
2019 arXiv   pre-print
Deep Learning techniques have achieved remarkable results in many domains.  ...  Often, training deep learning models requires large datasets, which may require sensitive information to be uploaded to the cloud to accelerate training.  ...  Next, we explain how to reverse-engineer the three conventional activation functions. Finally, we propose a step-wise activation function.  ... 
arXiv:1904.06049v1 fatcat:ria6usa5incybfymz72hvtxgoi

The Limits of SEMA on Distinguishing Similar Activation Functions of Embedded Deep Neural Networks

Go Takatoi, Takeshi Sugawara, Kazuo Sakiyama, Yuko Hara-Azumi, Yang Li
2022 Applied Sciences  
As selecting the appropriate activation functions to enable fast training of accurate deep neural networks is an active area of research, it is important to conceal the information of the activation functions  ...  used in a neural network architecture as well.  ...  The proposed SEMA attack can be combined with the full reverse-engineering of the neural network in [15] .  ... 
doi:10.3390/app12094135 fatcat:rlta7qbks5ggtnhppgsz3u4xaq

Interpretations of Deep Learning by Forests and Haar Wavelets [article]

Changcun Huang
2019 arXiv   pre-print
ReLU deep learning is given.  ...  Finally, generalize some of the conclusions of ReLU deep learning to the case of sigmoid-unit deep learning.  ...  Since ReLU has nearly become the dominant choice of neural units used by deep learning in recent years [9, 18] , the main topics of this paper are general and useful both in theory and engineering.  ... 
arXiv:1906.06706v7 fatcat:gkezdt7z4ffzld6ovlmtrvejbm

Analysis and Mitigations of Reverse Engineering Attacks on Local Feature Descriptors [article]

Deeksha Dangwal, Vincent T. Lee, Hyo Jin Kim, Tianwei Shen, Meghan Cowan, Rajvi Shah, Caroline Trippel, Brandon Reagen, Timothy Sherwood, Vasileios Balntas, Armin Alaghi, Eddy Ilg
2021 arXiv   pre-print
However, recent work has demonstrated that under certain conditions reverse engineering attacks are possible and allow an adversary to reconstruct RGB images.  ...  Subsequently, we show under controlled conditions a reverse engineering attack on sparse feature maps and analyze the vulnerability of popular descriptors including FREAK, SIFT and SOSNet.  ...  Architecture Implementation Details Our reverse engineering attack uses a deep convolutional generator-discriminator network (see main paper).  ... 
arXiv:2105.03812v1 fatcat:owrcwtt3cfel5cy4iwedfs4kiu

Residual Semantic Segmentation of the Prostate from Magnetic Resonance Images [chapter]

Md Sazzad Hossain, Andrew P. Paplinski, John M. Betts
2018 Lecture Notes in Computer Science  
To overcome these limitations, we demonstrate an automatic segmentation of the prostate region in MRI images using a VGG19-based fully convolutional neural network.  ...  This new network, VGG19RSeg, identifies a region of interest in the image using semantic segmentation, that is, a pixel-wise classification of the content of the input image.  ...  Fig. 1 . 1 VGG19 deep neural network model. Fig. 2 . 2 An example of a structure of a semantic segmentation network.  ... 
doi:10.1007/978-3-030-04239-4_46 fatcat:7njh7sp5hnhizbqvlzquypblvu

COVID-19 Diagnosis Using X-ray Images Based on Convolutional Neural Networks

Wafaa A. Shalaby, Waleed Saad, Mona Shokair, Moawad I. Dessouky, Fathi E. Abd El-Samie
2021 2021 International Conference on Electronic Engineering (ICEEM)  
Recently, Convolutional Neural Network has been considered as a type of deep learning tools, and it can be used for detecting diseases such as COVID-19.  ...  Deep feature extraction is performed using the proposed CNN model and different pre-trained models.  ...  & ReLU Conv6 256 33 11 [1 1 1 1] 33128256 1414256 Batch Normalization & ReLU Conv7 512 33 22 [1 1 1 1] 33256512 77512 Batch Normalization & ReLU Conv8 512 33 11 [1 1 1 1] 33512512 7  ... 
doi:10.1109/iceem52022.2021.9480659 fatcat:r72gmjjzqrhs7eozhdwzqyqboe

Aero-Engine Surge Fault Diagnosis Using Deep Neural Network

Kexin Zhang, Bin Lin, Jixin Chen, Xinlong Wu, Chao Lu, Desheng Zheng, Lulu Tian
2022 Computer systems science and engineering  
In the field of aero-engine fault diagnosis, the introduction of deep learning technology is of great significance.  ...  On the basis of analyzing the mechanism of aero-engine surge, an aero-engine surge fault diagnosis method based on deep learning technology is proposed.  ...  Zheng, et al. proposed an intelligent fault diagnosis method for aero-engine sensors based on deep learning and time-frequency analysis.  ... 
doi:10.32604/csse.2022.021132 fatcat:ihvtsxzh7rcptj7h4rpjgm2m4q

Neural Computing for Determining the Accuracy of Ultimate Tensile Strength of Friction Stir Welded Joints by using Various Activation Functions

Akshansh Mishra, Project Scientific Officer, Center of Artificial Intelligence based Friction Stir Welding, Stir Research Technologies, Uttar Pradesh, India
activation function, Rectified Linear unit (ReLu) activation function and Hyperbolic tangent activation function.  ...  Activation functions in a particular Artificial Neural Network (ANN) architecture plays a vital role. It imparts non-linear properties to our Neural Networks.  ...  Conclusion Previous research show that in deep learning the ReLU has become the activation function of choice because the math is much simpler from sigmoid activation functions such as tanh or logit, especially  ... 
doi:10.24321/2454.8650.201905 fatcat:c6z3eqtiq5drvnkbt3k2oaf5wq

SGBA: A Stealthy Scapegoat Backdoor Attack against Deep Neural Networks [article]

Ying He, Zhili Shen, Chang Xia, Jingyu Hua, Wei Tong, Sheng Zhong
2022 arXiv   pre-print
Due to the severity of such attacks, many backdoor detection and containment systems have recently, been proposed for deep neural networks.  ...  Outsourced deep neural networks have been demonstrated to suffer from patch-based trojan attacks, in which an adversary poisons the training sets to inject a backdoor in the obtained model so that regular  ...  INTRODUCTION D EEP Neural Networks (DNN) show a strong learning ability due to their deep network architectures.  ... 
arXiv:2104.01026v3 fatcat:7lalphh2dje6hlbfcv775bqsk4

Two-phase flow regime prediction using LSTM based deep recurrent neural network [article]

Zhuoran Dang, Mamoru Ishii
2019 arXiv   pre-print
In this paper, a methodology of using LSTM-based deep-RNN for two-phase flow regime prediction is proposed, motivated by previous research on constructing deep RNN.  ...  The result shows that the prediction accuracy depends on the depth of network and the number of layer cells. However, deeper and larger network consumes more time in predicting.  ...  This paper follows a similar approach of constructing deep LSTM network with [10] . Their ideas of constructing deep RNN network is as follows: 1) input-hidden; 2) hidden-hidden; 3) hidden-output.  ... 
arXiv:1904.00291v1 fatcat:f2gddddo25hgbm4tsu2nuwodwm

Smish: A Novel Activation Function for Deep Learning Methods

Xueliang Wang, Honge Ren, Achuan Wang
2022 Electronics  
Activation functions are crucial in deep learning networks, given that the nonlinear ability of activation functions endows deep neural networks with real artificial intelligence.  ...  The experimental results show that with Smish, the EfficientNetB3 network exhibits a Top-1 accuracy of 84.1% on the CIFAR-10 dataset; the EfficientNetB5 network has a Top-1 accuracy of 99.89% on the MNIST  ...  In deep learning networks, nonlinear activation functions-Sigmoid, ReLU, Swish, Mish, and Logish-are frequently used [8, 9] .  ... 
doi:10.3390/electronics11040540 fatcat:3gh62ruxvngivbrwzziujkz65m
« Previous Showing results 1 — 15 out of 5,172 results