Filters








707,369 Hits in 6.8 sec

Efficient Transformers: A Survey [article]

Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler
2022 arXiv   pre-print
Recently, a dizzying number of "X-former" models have been proposed - Reformer, Linformer, Performer, Longformer, to name a few - which improve upon the original Transformer architecture, many of which  ...  Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision and reinforcement learning.  ...  Throughout this survey, we refer to the efficiency of Transformers both in terms of memory and computation.  ... 
arXiv:2009.06732v3 fatcat:rxchuq3adrg3vlgn672pwd6evu

Efficient Transformers: A Survey

Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler
2022 ACM Computing Surveys  
Recently, a dizzying number of "X-former" models have been proposed - Reformer, Linformer, Performer, Longformer, to name a few - which improve upon the original Transformer architecture, many of which  ...  Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision and reinforcement learning.  ...  A SURVEY OF EFFICIENT TRANSFORMER MODELS In this section, we provide a high-level overview of eicient Transformer models. We begin by presenting a characterization of the diferent models.  ... 
doi:10.1145/3530811 fatcat:sil36wgiz5c23hnr5iwzu26pwi

A Survey of Efficient Hough Transform Methods

J. Illingworth, J. Kittler
1987 Procedings of the Alvey Vision Conference 1987  
The Hough Transform, HT, has long been recognised as a technique of almost unique promise for shape and motion analysis in images containing noisy, missing and extraneous data.  ...  The aims of Alvey project MMI/IP078 are • the investigation of the properties of HTs • the study of efficient implementations of HTs • the production of a HT hardware device for use in real time industrial  ...  There has recently been much activity concerned with space and time efficient implementations of HT ideas and this paper presents a survey of this area.  ... 
doi:10.5244/c.1.43 dblp:conf/bmvc/IllingworthK87 fatcat:6glrbt76unflfnmzjnkdmwbbx4

Voice Transformation: A survey

Yannis Stylianou
2009 2009 IEEE International Conference on Acoustics, Speech and Signal Processing  
Voice transformation refers to the various modifications one may apply to the sound produced by a person, speaking or singing.  ...  Voice Transformation is usually seen as an add-on or an external system in speech synthesis systems since it may create virtual voices in a simple and flexible way.  ...  To efficiently control this information, Voice Transformation systems need to understand the process and mechanism at a production and a perception level.  ... 
doi:10.1109/icassp.2009.4960401 dblp:conf/icassp/Stylianou09 fatcat:mp2kvialhnecjfvomut2z5qmny

A Survey of Transformers [article]

Tianyang Lin, Yuxin Wang, Xiangyang Liu, Xipeng Qiu
2021 arXiv   pre-print
In this survey, we provide a comprehensive review of various X-formers. We first briefly introduce the vanilla Transformer and then propose a new taxonomy of X-formers.  ...  Up to the present, a great variety of Transformer variants (a.k.a.  ...  This line of work aims to adapt the Transformer to specific downstream tasks and applications. In this survey, we aim to provide a comprehensive review of the Transformer and its variants.  ... 
arXiv:2106.04554v2 fatcat:pjctgoqeffhq7ntyw52jqwfzsy

Transformers in Vision: A Survey [article]

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah
2021 arXiv   pre-print
This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.  ...  These strengths have led to exciting progress on a number of vision tasks using Transformer networks.  ...  We would also like to thank Mohamed Afham for his help with a figure.  ... 
arXiv:2101.01169v4 fatcat:ynsnfuuaize37jlvhsdki54cy4

A Survey on Vision Transformer [article]

Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, Zhaohui Yang, Yiman Zhang (+1 others)
2021 arXiv   pre-print
We also include efficient transformer methods for pushing transformer into real device-based applications.  ...  Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism.  ...  As such, a survey of the existing works is urgent and would be beneficial for the community.  ... 
arXiv:2012.12556v4 fatcat:ldtbdgy6tbdttfqzhzml7n577m

A Survey of Visual Transformers [article]

Yang Liu, Yao Zhang, Yixin Wang, Feng Hou, Jin Yuan, Jiang Tian, Yang Zhang, Zhongchao Shi, Jianping Fan, Zhiqiang He
2022 arXiv   pre-print
In this survey, we have reviewed over one hundred of different visual Transformers comprehensively according to three fundamental CV tasks and different data stream types, where a taxonomy is proposed  ...  Furthermore, we have revealed a series of essential but unexploited aspects that may empower such visual Transformers to stand out from numerous architectures, e.g., slack high-level semantic embeddings  ...  Thus, a systematic literature survey is strongly desired to identify, categorize, and evaluate the performance of these existing visual Transformers.  ... 
arXiv:2111.06091v3 fatcat:a3fq6lvvzzgglb3qtus5qwrwpe

A survey of Hough Transform

Priyanka Mukhopadhyay, Bidyut B. Chaudhuri
2015 Pattern Recognition  
In 1962 Hough earned the patent for a method [1], popularly called Hough Transform (HT) that efficiently identifies lines in images.  ...  Our survey, along with more than 200 references, will help the researchers and students to get a comprehensive view on HT and guide them in applying it properly to their problems of interest.  ...  Chaudhuri, A survey of Hough Transform, Pattern Recognition (2014), http://dx.doi.  ... 
doi:10.1016/j.patcog.2014.08.027 fatcat:oye56pu2mrastlppxds6cggvue

Transformers in Time Series: A Survey [article]

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun
2022 arXiv   pre-print
We hope this survey will ignite further research interests in time series Transformers.  ...  In this paper, we systematically review transformer schemes for time series modeling by highlighting their strengths as well as limitations through a new taxonomy to summarize existing time series transformers  ...  Conclusion In this paper, we provide a comprehensive survey on time series Transformers in various tasks.  ... 
arXiv:2202.07125v3 fatcat:q6vmkszqavbgrdtrbpsgaco7u4

The Hermite Transform: A Survey

Jean-Bernard Martens
2006 EURASIP Journal on Advances in Signal Processing  
With this survey on the Hermite transformation we want to pursue the following two goals.  ...  As a consequence, the existing publications on the Hermite transformation do not fully reflect our most recent insights, and the current paper intends to remedy this.  ...  CONCLUSION With this survey we have attempted to give an overview of the main characteristics of the Hermite transformation and their implications for applications.  ... 
doi:10.1155/asp/2006/26145 fatcat:4m2wkm4a35btbdlggt7juhllqi

Multimodal Learning with Transformers: A Survey [article]

Peng Xu, Xiatian Zhu, David A. Clifton
2022 arXiv   pre-print
This paper presents a comprehensive survey of Transformer techniques oriented at multimodal data.  ...  The main contents of this survey include: (1) a background of multimodal learning, Transformer ecosystem, and the multimodal big data era, (2) a theoretical review of Vanilla Transformer, Vision Transformer  ...  We hope that this survey gives a helpful and detailed overview for new researchers and practitioners, provides a convenient reference for relevant experts (e.g., multimodal machine learning researchers  ... 
arXiv:2206.06488v1 fatcat:6aoaczzbtvc43my2kmobo7glvy

Transformers in Remote Sensing: A Survey [article]

Abdulaziz Amer Aleissaee, Amandeep Kumar, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal, Gui-Song Xia, Fahad Shahbaz khan
2022 arXiv   pre-print
Although a number of surveys have focused on transformers in computer vision in general, to the best of our knowledge we are the first to present a systematic review of recent advances based on transformers  ...  We conclude the survey by discussing different challenges and open issues of transformers in remote sensing.  ...  The proposed Efficient-T employs hierarchical Swin transformers along with MLP head.  ... 
arXiv:2209.01206v1 fatcat:luchmgyyord5nnjthywqszyj6q

3D Vision with Transformers: A Survey [article]

Jean Lahoud, Jiale Cao, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Ming-Hsuan Yang
2022 arXiv   pre-print
Although a number of surveys have focused on transformers in vision in general, 3D vision requires special attention due to the difference in data representation and processing when compared to 2D vision  ...  We conclude the survey by discussing different open directions and challenges for transformers in 3D vision.  ...  Since points belong to a continuous domain, an efficient sampling technique is required prior to processing.  ... 
arXiv:2208.04309v1 fatcat:h7xk3hydevhwhatx5lil3ftnri

Transformers in computational visual media: A survey

Yifan Xu, Huapeng Wei, Minxuan Lin, Yingying Deng, Kekai Sheng, Mengdan Zhang, Fan Tang, Weiming Dong, Feiyue Huang, Changsheng Xu
2021 Computational Visual Media  
This study comprehensively surveys recent visual transformer works.  ...  Transformers are sequence-to-sequence models, which use a self-attention mechanism rather than the RNN sequential structure.  ...  Thus, efficiency of transformers for high-level vision remains a pressing research direction.  ... 
doi:10.1007/s41095-021-0247-3 fatcat:fszqbepk3jg2jbeb33mn7cjgim
« Previous Showing results 1 — 15 out of 707,369 results