457,916 Hits in 3.5 sec

Complex Transformer: A Framework for Modeling Complex-Valued Sequence [article]

Muqiao Yang, Martin Q. Ma, Dongyu Li, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov
2021 arXiv   pre-print
In this paper, we propose a Complex Transformer, which incorporates the transformer model as a backbone for sequence modeling; we also develop attention and encoder-decoder network operating for complex  ...  However, speech, signal and audio data are naturally complex-valued after Fourier Transform, and studies have shown a potentially richer representation of complex nets.  ...  We introduce a Complex Transformer to solve complex-valued sequence modeling tasks, including prediction and generation.  ... 
arXiv:1910.10202v2 fatcat:pqaez64mjzfd5cfre4kg4o2olq

Transformations, Dynamics and Complexity [article]

Nikolaj Glazunov
2011 arXiv   pre-print
maps, and some aspects of complexity of the systems.  ...  We review and investigate some new problems and results in the field of dynamical systems generated by iteration of maps, β-transformations, partitions, group actions, bundle dynamical systems, Hasse-Kloosterman  ...  Kolmogorov defines the complexity of a measurepreserving transformation by generators.  ... 
arXiv:1104.1898v1 fatcat:eo7h7zqbxzct7k2aun7ltnuv5y

Towards analyzing space complexity of streaming XML transformations

Jana Dvorakova
2008 2008 Second International Conference on Research Challenges in Information Science  
We present a formal framework that enables us to analyze space complexity of automatic streaming processing of XML transformations.  ...  Within the framework, the classes of XML transformations as well as the streaming algorithms are represented as formal models.  ...  The work contains results from a PhD thesis of Comenius University in Bratislava, Slovakia.  ... 
doi:10.1109/rcis.2008.4632137 dblp:conf/rcis/Dvorakova08 fatcat:ij5pd7qkebahnomdm5f7gw5bm4

Multidimensional, mapping-based complex wavelet transforms

F.C.A. Fernandes, R.L.C. van Spaendonck, C.S. Burrus
2005 IEEE Transactions on Image Processing  
This allows us to create a directional, nonredundant, complex wavelet transform with potential benefits for image coding systems.  ...  To overcome these disadvantages, we introduce multidimensional, mapping-based, complex wavelet transforms that consist of a mapping onto a complex function space followed by a DWT of the complex mapping  ...  [23] subsequently defined mapping-based CWTs, a new framework for the implementation of complex wavelet transforms.  ... 
doi:10.1109/tip.2004.838701 pmid:15646876 fatcat:tws7mdlogbfhfmgvoic6bbvl5m

Handling Complexity Of A Complex System Design: Paradigm, Formalism And Transformations

Hycham Aboutaleb, Bruno Monsuez
2015 Zenodo  
Finally, a set of transformations are defined to handle the model complexity.  ...  The exponential growing effort, cost and time investment of complex systems in modeling phase emphasize the need for a paradigm, a framework and an environment to handle the system model complexity.  ...  To optimize the design time, it is important to have a useful framework for analyzing complex systems and study their evolution.  ... 
doi:10.5281/zenodo.1107612 fatcat:sumocpijozh3bctx2j53f7lu3u

Transforming Complex Loop Nests for Locality

Qing Yi, Ken Kennedy, Vikram Adve
2004 Journal of Supercomputing  
To optimize complex loop structures both effectively and inexpensively, we present a novel loop transformation, dependence hoisting, for optimizing arbitrarily nested loops, and an efficient framework  ...  The extended model has a complexity comparable to that of the traditional dependence models and is described in more detail in Section 4.  ...  The complexity of applying dependence hoisting is thus equivalent to that of the corresponding sequence of sub-transformations.  ... 
doi:10.1023/b:supe.0000011386.69245.f5 fatcat:2pbceg3h5nh27enc6p66ypncve

Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform [article]

Kehelwala D. G. Maduranga, Kyle E. Helfrich, Qiang Ye
2019 arXiv   pre-print
In this paper, we develop a unitary RNN architecture based on a complex scaled Cayley transform.  ...  Unlike the real orthogonal case, the transformation uses a diagonal scaling matrix consisting of entries on the complex unit circle which can be optimized using gradient descent and no longer requires  ...  We would also like to thank Devin Willmott for his help on this project.  ... 
arXiv:1811.04142v2 fatcat:3c5td5wwvvchbk6topme2usfta

Relative Positional Encoding for Transformers with Linear Complexity [article]

Antoine Liutkus, Ondřej Cífka, Shih-Lun Wu, Umut Şimşekli, Yi-Hsuan Yang, Gaël Richard
2021 arXiv   pre-print
Recent advances in Transformer models allow for unprecedented sequence lengths, due to linear space and time complexity.  ...  In the meantime, relative positional encoding (RPE) was proposed as beneficial for classical Transformers and consists in exploiting lags instead of absolute positions for inference.  ...  Introduction Linear Complexity Transformers The Transformer model (Vaswani et al., 2017) is a new kind of neural network that quickly became state-of-theart in many application domains, including the  ... 
arXiv:2105.08399v2 fatcat:4x2biihoz5dzrbcupisnlh6o5q

Foundations for Streaming Model Transformations by Complex Event Processing

István Dávid, István Ráth, Dániel Varró
2016 Journal of Software and Systems Modeling  
the Viatra reactive transformation framework.  ...  Furthermore, a reactive rule engine carries out transformations on identified complex event patterns.  ...  Our streaming transformation approach requires at least two languages: one for complex event processing and one for model transformations.  ... 
doi:10.1007/s10270-016-0533-1 pmid:29449795 pmcid:PMC5807515 fatcat:57era3rizfbhxmlvopue6dyuge

On the Power of Saturated Transformers: A View from Circuit Complexity [article]

William Merrill and Yoav Goldberg and Noah A. Smith
2021 arXiv   pre-print
Transformers have become a standard architecture for many NLP problems.  ...  However, hard attention is a restrictive assumption, which may complicate the relevance of these results for practical transformers.  ...  Acknowledgments Thanks to Dana Angluin and Yiding Hao for sharing a draft of their work. Thanks also to Matt Gardner for his detailed feedback.  ... 
arXiv:2106.16213v2 fatcat:o4gbvkvzvrd7be2gl3p2y2tk7q

Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition [article]

Yihong Dong, Ying Peng, Muqiao Yang, Songtao Lu, Qingjiang Shi
2021 arXiv   pre-print
., real and imaginary parts consisted in practical time-series signals, we propose a Complex-valued Attentional MEta Learner (CAMEL) for the problem of few-shot signal recognition by leveraging attention  ...  Deep neural networks have been shown as a class of useful tools for addressing signal recognition issues in recent years, especially for identifying the nonlinear feature structures of signals.  ...  the Complex Transformer model using 8 attention functions to represent the complex-valued attention.  ... 
arXiv:2106.04392v2 fatcat:pk3vkxlck5bpncbivok3x3gnoe

Euler Transformation of Polyhedral Complexes [article]

Prashant Gupta, Bala Krishnamoorthy
2021 arXiv   pre-print
We propose an Euler transformation that transforms a given d-dimensional cell complex K for d=2,3 into a new d-complex K̂ in which every vertex is part of a uniform even number of edges.  ...  For 2-complexes in ℝ^2 (d=2) under mild assumptions (that no two adjacent edges of a 2-cell in K are boundary edges), we show that the Euler transformed 2-complex K̂ has a geometric realization in ℝ^2,  ...  Department of Energy and administered by the Oak Ridge Institute for Science and Education.  ... 
arXiv:1812.02412v3 fatcat:vmlqwvqr65hlhgdcth6ydg7mcm

Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds [article]

Fangcen Liu, Chenqiang Gao, Fang Chen, Deyu Meng, Wangmeng Zuo, Xinbo Gao
2021 arXiv   pre-print
To this end, we propose a robust and general infrared small-dim target detection method with the transformer.  ...  We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range.  ...  A model with a low false alarm rate will have a higher AUC value. 3) Implementation details: The framework of the proposed method is implemented using Pytorch 1.7.1, and accelerated by CUDA 11.2.  ... 
arXiv:2109.14379v2 fatcat:tjc4ud65mjcsbmfhgthw4jzfui

Motion estimation using a complex-valued wavelet transform

J. Magarey, N. Kingsbury
1998 IEEE Transactions on Signal Processing  
The DWT is based on a complex-valued pair of 4-tap FIR lters with Gabor-like characteristics.  ...  Introduction There are many applications in which a compact representation of the changes between the successive frames in a digital image sequence is required.  ...  Acknowledgement The authors would like to thank Dr Anil Kokaram for supplying the comparison algorithms, and for many helpful discussions.  ... 
doi:10.1109/78.668557 fatcat:6aiq7xbd4bfxjkbptnxior336e

Implicit complexity via structure transformation [article]

Daniel Leivant, Jean-Yves Marion
2018 arXiv   pre-print
We consider here "uninterpreted" programs for the transformation of finite structures, which define functions over a free algebra A once the elements of A are themselves considered as finite structures  ...  Implicit computational complexity, which aims at characterizing complexity classes by machine-independent means, has traditionally been based, on the one hand, on programs and deductive formalisms for  ...  One is descriptive complexity, which focuses on finite structures, and as such forms a branch of Finite Model Theory [18] .  ... 
arXiv:1802.03115v1 fatcat:hkozhl4t2jgvve22iq7oqnnva4
« Previous Showing results 1 — 15 out of 457,916 results