2,766 Hits in 3.1 sec

SOFT: Softmax-free Transformer with Linear Complexity [article]

Jiachen Lu, Jinghan Yao, Junge Zhang, Xiatian Zhu, Hang Xu, Weiguo Gao, Chunjing Xu, Tao Xiang, Li Zhang
2022 arXiv   pre-print
Keeping this softmax operation challenges any subsequent linearization efforts. Based on this insight, for the first time, a softmax-free transformer or SOFT is proposed.  ...  Crucially, with a linear complexity, much longer token sequences are permitted in SOFT, resulting in superior trade-off between accuracy and complexity.  ...  (I) We introduce a novel softmax-free Transformer with linear space and time complexity.  ... 
arXiv:2110.11945v3 fatcat:od7crqsonrcbxbrxgznwhldqw4

Discriminative Fast Soft Competitive Learning [chapter]

Frank-Michael Schleif
2014 Lecture Notes in Computer Science  
The algorithm has linear computational and memory requirements and performs favorable to traditional techniques.  ...  Here we extend fast soft competitive learning to a discriminative and vector labeled learning algorithm for proximity data.  ...  While standard R-SCL has squared complexity a linear cost algorithm can be obtained by using the Nyström approximation [5] .  ... 
doi:10.1007/978-3-319-11179-7_11 fatcat:ejlp7suhwfa7hkxqgikgsmwtua

CMDNet: Learning a Probabilistic Relaxation of Discrete Variables for Soft Detection with Low Complexity [article]

Edgar Beck, Carsten Bockelmann, Armin Dekorsy
2021 arXiv   pre-print
The main motivation behind is that the complexity of Maximum A-Posteriori (MAP) detection grows exponentially with system dimensions.  ...  This is crucial for soft decoding in today's communication systems.  ...  As Tensorflow does not natively support computation with complex numbers, we transform the complex-valued system model (1a) into its real-valued equivalent to allow for training and comparison to DNN-based  ... 
arXiv:2102.12756v3 fatcat:4nsl6kt3wnfzbfond2acent2te

From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference [article]

Randall Balestriero, Richard G. Baraniuk
2018 arXiv   pre-print
We further extend the framework by hybridizing the hard and soft VQ optimizations to create a β-VQ inference that interpolates between hard, soft, and linear VQ inference.  ...  Finally, we validate with experiments an important assertion of our theory, namely that DN performance can be significantly improved by enforcing orthogonality in its linear filters.  ...  We illustrate by showing that the soft versions of ReLU and max-pooling are the sigmoid gated linear unit and softmax pooling, respectively.  ... 
arXiv:1810.09274v1 fatcat:tbycafpyjjbkpkuhzcopo2rja4

LUTNet: Rethinking Inference in FPGA Soft Logic [article]

Erwei Wang, James J. Davis, Peter Y. K. Cheung, George A. Constantinides
2019 arXiv   pre-print
Network binarisation on FPGAs greatly increases area efficiency by replacing resource-hungry multipliers with lightweight XNOR gates.  ...  BNN to LUTNet architectural transformation for a single channel, mirroring the replacement of (1) with (2) .  ...  Our aim in proposing this inference node function is to play to the strengths of FPGA soft logic.  ... 
arXiv:1904.00938v1 fatcat:2rclxhsodvbqpitblbmy3ipkfq

DeepSIC: Deep Soft Interference Cancellation for Multiuser MIMO Detection [article]

Nir Shlezinger, Rong Fu, Yonina C. Eldar
2020 arXiv   pre-print
Our numerical evaluations demonstrate that for linear channels with full CSI, DeepSIC approaches the performance of iterative SIC, which is comparable to the optimal performance, and outperforms previously  ...  In particular, we propose a data-driven implementation of the iterative soft interference cancellation (SIC) algorithm which we refer to as DeepSIC.  ...  A common strategy to implement joint detection with affordable computational complexity, suitable for channels in which Y [i] is given by a linear transformation of S[i] corrupted by additive noise, is  ... 
arXiv:2002.03214v2 fatcat:2aw2k3htwva25nkn3evup3uikq

Autonomous soft hand grasping – Literature review [article]

Tai Hoang
2022 arXiv   pre-print
It is the trending research topic for soft hand grasping in recent years as it has shown a high performance when dealing with a large number of various objects.  ...  Recently, soft robotic hands, a new trend has emerged, aiming to make the design adequately complex and affordable while requiring much less effort to control.  ...  In the early 2000s, a lot of designs followed the human hand structure with high mechanical complexity, (i.e fully actuated with many degrees of freedom) [2] .  ... 
arXiv:2203.04762v1 fatcat:7z22gtloyzaa5mtrz47a4w2b3u

Deep Learning-Based Decoding and AP Selection for Radio Stripe Network

Aman Kumar Mishra, Vijayakumar Ponnusamy
2022 Intelligent Automation and Soft Computing  
Radio Stripe architecture of cell-free mMI-MO is one such architecture of cell-free mMIMO which is suitable for practical deployment.  ...  The proposed DNNBPDRS framework not only improves Symbol Error Rate (SER) performance when compared to counterparts but is also proved to be comparatively far lesser computational complex.  ...  DL replaces the complex computations with the trained model, which reduces the complexity considerably.  ... 
doi:10.32604/iasc.2022.021017 fatcat:j3wed74lr5amvgcb3sfeaq2ndq

SoDA: Multi-Object Tracking with Soft Data Association [article]

Wei-Chih Hung, Henrik Kretzschmar, Tsung-Yi Lin, Yuning Chai, Ruichi Yu, Ming-Hsuan Yang, Dragomir Anguelov
2020 arXiv   pre-print
Tracking objects, however, remains a highly challenging problem, especially in cluttered autonomous driving scenes in which objects tend to interact with each other in complex ways and frequently get occluded  ...  Instead, our model aggregates information from all object detections via soft data associations.  ...  Note that, in contrast to our work, the Transformer-XL obtains R k by applying the linear transform to the sinusoidal positional encoding proposed by the original formulation of the Transformer [43] .  ... 
arXiv:2008.07725v2 fatcat:ljvlc74juzfmrbyugn2gzm36fq

Detection of Microbial Activity in Silver Nanoparticles Using Modified Convolution Network

D. Devina Merin, P. A. Jagatheeswari
2022 Intelligent Automation and Soft Computing  
The unique architecture of DL networks can be trained according to classify any complex tasks in a limited duration.  ...  AgNP images from scanning electron microscope are pre-processed using Adaptive Histogram Equalization in the networking system and the DL classification model Deep convolution neural network (DCNN) with  ...  Acknowledgement: The authors with a deep sense of gratitude would thank the supervisor for his guidance and constant support rendered during this research.  ... 
doi:10.32604/iasc.2022.024495 fatcat:jehgoijnljbifplecq5gi5ucyi

SENSORIMOTOR GRAPH: Action-Conditioned Graph Neural Network for Learning Robotic Soft Hand Dynamics [article]

João Damião Almeida, Paul Schydlo, Atabak Dehban, José Santos-Victor
2021 arXiv   pre-print
Soft robotics is a thriving branch of robotics which takes inspiration from nature and uses affordable flexible materials to design adaptable non-rigid robots.  ...  In this work, we take inspiration from sensorimotor learning, and apply a Graph Neural Network to the problem of modelling a non-rigid kinematic chain (i.e. a robotic soft hand) taking advantage of two  ...  This approach is useful when the relations between the different parts are simple, but it becomes unfeasible for systems with complex non-trivial dynamics such as soft robotics and rubbery materials commonly  ... 
arXiv:2107.08492v1 fatcat:ilqvssa3qvefdn3wk2kozm6lru

Towards Full-to-Empty Room Generation with Structure-Aware Feature Encoding and Soft Semantic Region-Adaptive Normalization [article]

Vasileios Gkitsas, Nikolaos Zioulis, Vladimiros Sterzentsenko, Alexandros Doumanoglou, Dimitrios Zarpalas
2021 arXiv   pre-print
To tackle these drawbacks, we treat scene layout generation as a feature linear transformation problem and propose a simple yet effective adjusted fully differentiable soft semantic region-adaptive normalization  ...  We showcase the applicability in diminished reality and depth estimation tasks, where our approach besides the advantages of mitigating training complexity and non-differentiability issues, surpasses the  ...  Similarly, we employ a linear transformation, Lt to obtain the soft semantic mask of the input scene.  ... 
arXiv:2112.05396v1 fatcat:xyjwmgc4xzbpdcpsztse6n4ewm

An extended hybrid image compression based on soft-to-hard quantification

Haisheng Fu, Feng Liang, Bo Lei
2020 IEEE Access  
In this paper, we propose an extended hybrid image compression scheme based on soft-to-hard quantification, which has only two layers.  ...  INDEX TERMS Extend hybrid image compression scheme, soft-to-hard quantification, bit rates allocation, enhancement layer codecs. 95832 This work is licensed under a Creative Commons Attribution 4.0 License  ...  [1] introduced a softto-hard quantification method with an approximate softmax function.  ... 
doi:10.1109/access.2020.2994393 fatcat:3njlexiuzfbjxgsfjlrgmj7upi

Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction

Zhenyu Zhang, Xiaobo Shu, Bowen Yu, Tingwen Liu, Jiapeng Zhao, Quangang Li, Li Guo
In this paper, we aim to explore the supervision with soft labels in relation extraction, which makes it possible to integrate prior knowledge.  ...  Extracting relations from plain text is an important task with wide application.  ...  a linear layer followed by softmax function: h base = [h sent , h s , h o ] (7) The Teacher Network The teacher network focuses on improving the performance of RE with soft rules yield from the bipartite  ... 
doi:10.1609/aaai.v34i05.6509 fatcat:cqcm3hstd5gztbeyuwbo2rgqvy

End-to-end Handwritten Chinese Paragraph Text Recognition Using Residual Attention Networks

Yintong Wang, Yingjie Yang, Haiyan Chen, Hao Zheng, Heyou Chang
2022 Intelligent Automation and Soft Computing  
The existing methods are facing huge challenges including the complex structure of character/line-touching, the discriminate ability of similar characters and the labeling of training datasets.  ...  In experimental, the proposed method is verified with two widely adopted handwritten Chinese text datasets, and achieves competitive results to the current state-of-the-art methods.  ...  Therefore, segmentation-free recurrent-free model architectures can better confront the complex text structure and utilize the parallel computing with limited computing resources to achieve efficient HCTR  ... 
doi:10.32604/iasc.2022.027146 fatcat:57s5jf4apvhmzchhnc6jg24g4e
« Previous Showing results 1 — 15 out of 2,766 results