66,424 Hits in 2.9 sec

Adaptive Activation-based Structured Pruning [article]

Kaiqi Zhao, Animesh Jain, Ming Zhao
2022 arXiv   pre-print
To address these limitations, this paper presents an adaptive, activation-based, structured pruning approach to automatically and efficiently generate small, accurate, and hardware-efficient models that  ...  First, it proposes iterative structured pruning using activation-based attention feature maps to effectively identify and prune unimportant filters.  ...  METHODOLOGY Algorithm 1 illustrates the overall flow of the proposed adaptive activation-based structured pruning.  ... 
arXiv:2201.10520v1 fatcat:qh2pr5nqjrgufa7zs7qwwv5oty

E2-AEN: End-to-End Incremental Learning with Adaptively Expandable Network [article]

Guimei Cao, Zhanzhan Cheng, Yunlu Xu, Duo Li, Shiliang Pu, Yi Niu, Fei Wu
2022 arXiv   pre-print
These adapters are controlled via an adaptive gate-based pruning strategy which decides whether the expanded structures can be pruned, making the network structure dynamically changeable according to the  ...  Considering that different tasks may need different structures, recent methods design dynamic structures adapted to different tasks via sophisticated skills.  ...  adapters and an adaptive gate-based pruning strategy.  ... 
arXiv:2207.06754v1 fatcat:zbj7hruhsnbilo677saxlrthvy

Target Aware Network Adaptation for Efficient Representation Learning [article]

Yang Zhong, Vladimir Li, Ryuzo Okada, Atsuto Maki
2018 arXiv   pre-print
We achieve this with a few novel ideas: (i) cumulative sum of activation statistics for each layer, and (ii) a priority evaluation of pruning across multiple layers.  ...  This paper presents an automatic network adaptation method that finds a ConvNet structure well-suited to a given target task, e.g., image classification, for efficiency as well as accuracy in transfer  ...  based as used in our method, i.e. activation based methods offer a more reasonable pruning.  ... 
arXiv:1810.01104v1 fatcat:77ma2obe7vcppem2laabx626re

Adaptive structure radial basis function network model for processes with operating region migration

D.K. Siong Tok, Ding-Li Yu, Christian Mathews, Dong-Ya Zhao, Quan-Min Zhu
2015 Neurocomputing  
A center grouping algorithm is also developed to divide the centers into active and non-active groups, so that a structure with a smaller size is maintained in the final network model.  ...  A center grouping algorithm is also developed to divide the centers into active and non-active groups, so that a structure with a smaller size is maintained in the final network model.  ...  structure and adaptive structure.  ... 
doi:10.1016/j.neucom.2014.12.030 fatcat:xwq3zd77uvdwzbcmatzfrxlwcu

On-Demand Real-Time Optimizable Dynamic Model Sizing for Digital Predistortion of Broadband RF Power Amplifiers

Yue Li, Anding Zhu
2020 IEEE transactions on microwave theory and techniques  
In section III, a novel model structure adaptation algorithm suitable for real time deployment is detailed, featuring a statistical hypothesis-based model pruning method and a neighborhood-based model  ...  PROPOSED DYNAMIC MODEL SIZING AND ADAPTATION ALGORITHM In this section, a novel model structure adaptation algorithm is proposed that can well address the challenges of adaptive model pruning problem while  ... 
doi:10.1109/tmtt.2020.2982165 fatcat:bnw7gftbwrdkzerzj3hpdiv4jq

Local Feature Descriptor Learning with Adaptive Siamese Network [article]

Chong Huang, Qiong Liu, Yan-Ying Chen, Kwang-Ting Cheng
2017 arXiv   pre-print
In order to address the above problem, we introduce an adaptive pruning Siamese Architecture based on neuron activation to learn local feature descriptors, making the network more computationally efficient  ...  Specifically, the local feature is represented in a low dimensional space, so the neural network should have more compact structure.  ...  We follow the evaluation protocol of MatchNet [7] and evaluate the patch-based matching before and after adaptive pruning matching.  ... 
arXiv:1706.05358v1 fatcat:kgipaoenarcypncstbthkyouxy

Towards Efficient Neuromorphic Hardware: Unsupervised Adaptive Neuron Pruning

Wenzhe Guo, Hasan Erdem Yantır, Mohammed E. Fouda, Ahmed M. Eltawil, Khaled Nabil Salama
2020 Electronics  
Based on these criteria, we demonstrate that pruning with an adaptive spike count threshold provides a simple and effective approach that can reduce network size significantly and maintain high classification  ...  To solve real-time challenges, neuromorphic systems generally require deep and complex network structures.  ...  Therefore, we propose an adaptive neuron pruning strategy that enables the threshold to be adapted dynamically to the network firing activity. The pruning algorithm is described in Algorithm 1.  ... 
doi:10.3390/electronics9071059 fatcat:3vsynudhu5hcleyu7ruqog5um4

Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning [article]

Chaohui Yu, Jindong Wang, Yiqiang Chen, Zijing Wu
2019 arXiv   pre-print
Deep unsupervised domain adaptation (UDA) has recently received increasing attention from researchers.  ...  To be more specific, in VGG16, we get even higher accuracy after pruning 26% floating point operations (FLOPs); in ResNet50, we also get higher accuracy on half of the tasks after pruning 12% FLOPs.  ...  Pruned structure analysis. To explore if there is any pattern in the structure of the pruned models, we show the structure of pruned models on task A→W and I→P in Fig. 6 with TCP.  ... 
arXiv:1904.02654v1 fatcat:rewlryuwynhttdik5i7qjydmwa

Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning [article]

Minyoung Song, Jaehong Yoon, Eunho Yang, Sung Ju Hwang
2020 arXiv   pre-print
To overcome their limitations, we propose Set-based Task-Adaptive Meta Pruning (STAMP), which task-adaptively prunes a network pretrained on a large reference dataset by generating a pruning mask on it  ...  important weights or activations of a given network.  ...  Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning We introduce a novel structural pruning method for deep neural networks, Set-based Task-Adaptive Meta-Pruning (STAMP  ... 
arXiv:2006.12139v1 fatcat:n3bvr32ycrhnhkiazp57ddkdru

High-Efficiency Video Coder in Pruned Environment Using Adaptive Quantization Parameter Selection

Krishan Kumar, Mohamed Abouhawwash, Amit Kant Pandit, Shubham Mahajan, Mofreh A. Hogo
2022 Computers Materials & Continua  
The quantization method is adapted based on video sequence using statistical analysis, improving bit budget, quality and complexity reduction.  ...  This paper proposes an adaptive information-based variable quantization matrix (AI-VQM) developed for different video formats having variable energy levels.  ...  In HEVC structure, better video quality is achieved using efficient utilization of soft computing based quadtree structure combined with adaptive information based variable quantization method (AI-VQM)  ... 
doi:10.32604/cmc.2022.027850 fatcat:vcezyu45zrcu5ahze4ii3h5u6m

Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic Nodes

Miao Wang, Xu Yang, Yunchong Qian, Yunlin Lei, Jian Cai, Ziyi Huan, Xialv Lin, Hao Dong
2022 Current Issues in Molecular Biology  
Then, the network uses a pruning algorithm based on Hebb's rule or Pearson's correlation for adaptation in the pruning step. In addition" we combine genetic algorithm to optimize DNS(GA-DNS).  ...  We propose a Dynamic Node-based neural network Structure optimization algorithm (DNS) to handle these issues. DNS consists of two steps: the generation step and the pruning step.  ...  This construction process is heuristically applied to the construction of neural networks, and an adaptive neural network algorithm based on correlation analysis-neural network structure optimization algorithm  ... 
doi:10.3390/cimb44020056 pmid:35723341 pmcid:PMC8929060 fatcat:rff6k47l4rdr5b3v6jz7kpi3ba

Neuromorphic hardware as a self-organizing computing system [article]

Lyes Khacef, Bernard Girau, Nicolas Rougier, Andres Upegui, Benoit Miramond
2018 arXiv   pre-print
The objective is to study neural-based self-organization in computing systems and to prove the feasibility of a self-organizing hardware structure.  ...  From a biological point of view, this corresponds to a combination of the so-called synaptic and structural plasticities.  ...  Synaptic pruning and sprouting are two biological mechanisms permitting structural plasticity.  ... 
arXiv:1810.12640v1 fatcat:xbswtihjjbgdhl5aefnxd3v32e

Adaptive Pixel-wise Structured Sparse Network for Efficient CNNs [article]

Chen Tang, Wenyu Sun, Zhuqing Yuan, Yongpan Liu
2021 arXiv   pre-print
The sparse scheme is pixel-wise refined, regional adaptive under a unified importance map, which makes it friendly to hardware implementation.  ...  To accelerate deep CNN models, this paper proposes a novel spatially adaptive framework that can dynamically generate pixel-wise sparsity according to the input image.  ...  Conclusion In this paper, we propose a novel spatially adaptive method to generate structured sparsity in CNN-based models.  ... 
arXiv:2010.11083v3 fatcat:2suisxshxfapjoyy2exctpfjji

Efficient Deep Learning Inference Based on Model Compression

Qing Zhang, Mengru Zhang, Mengdi Wang, Wanchen Sui, Chen Meng, Jun Yang, Weidan Kong, Xiaoyuan Cui, Wei Lin
2018 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)  
In this paper, we will introduce a DL inference optimization pipeline, which consists of a series of model compression methods, including Tensor Decomposition (TD), Graph Adaptive Pruning (GAP), Intrinsic  ...  Sparse Structures (ISS) in Long Short-Term Memory (LSTM), Knowledge Distillation (KD) and low-bit model quantization.  ...  Based on the conception, we propose to adaptively prune network at vertex-level by a more structural way.  ... 
doi:10.1109/cvprw.2018.00221 dblp:conf/cvpr/ZhangZWSMYKCL18 fatcat:l3tchfxwkratvogotbhyt7jdoi

Research and Application of Improved AGP Algorithm for Structural Optimization Based on Feedforward Neural Networks

Ruliang Wang, Huanlong Sun, Benbo Zha, Lei Wang
2015 Mathematical Problems in Engineering  
The adaptive growing and pruning algorithm (AGP) has been improved, and the network pruning is based on the sigmoidal activation value of the node and all the weights of its outgoing connections.  ...  The nodes are pruned directly, but those nodes that have internal relation are not removed. The network growing is based on the idea of variance. We directly copy those nodes with high correlation.  ...  In the structural design, the algorithm is based on the sigmoidal activation value of the node to adjust the neural network by pruning the little value neurons, merging similar neurons, and increasing  ... 
doi:10.1155/2015/481919 fatcat:y7assx767ff7jgdbmcwbkdsxfq
« Previous Showing results 1 — 15 out of 66,424 results