A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is
Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequence into one context vector. However, a naive use of the multi-head attention does not guarantee such richness as the attention heads may have positional and representationalarXiv:1910.04500v1 fatcat:qd3ph25cobggzgql7qwzu4w2au
more »... ancy. In this paper, we propose a regularization technique for multi-head attention mechanism in an end-to-end neural keyword spotting system. Augmenting regularization terms which penalize positional and contextual non-orthogonality between the attention heads encourages to output different representations from separate subsequences, which in turn enables leveraging structured information without explicit sequence models such as hidden Markov models. In addition, intra-head contextual non-orthogonality regularization encourages each attention head to have similar representations across keyword examples, which helps classification by reducing feature variability. The experimental results demonstrate that the proposed regularization technique significantly improves the keyword spotting performance for the keyword "Hey Snapdragon".
A stagewise decomposition algorithm called value function gradient learning (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely-used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approacharXiv:2205.08934v1 fatcat:5odcfb7wojdanfmklgwu6aspw4
more »... has been successful for linear problems, nonlinear problems may suffer from the increasing size of each subproblem as the iteration proceeds. On the other hand, VFGL has a fixed number of parameters; thus, the size of the subproblems remains constant throughout the iteration. Furthermore, VFGL can learn the parameters by means of stochastic gradient descent, which means that it can be easily parallelized and does not require a scenario tree approximation of the underlying uncertainties. VFGL was compared with a deterministic equivalent formulation of the multistage stochastic programming problem and SDDP approaches for three illustrative examples: production planning, hydrothermal generation, and the lifetime financial planning problem. Numerical examples show that VFGL generates high-quality solutions and is computationally efficient.
The paper begins with a novel variational formulation of Duffing equation using the extended framework of Hamilton's principle (EHP). This formulation properly accounts for initial conditions, and it recovers all the governing differential equations as its Euler-Lagrange equation. Thus, it provides elegant structure for the development of versatile temporal finite element methods. Herein, the simplest temporal finite element method is presented by adopting linear temporal shape functions.arXiv:1903.06524v1 fatcat:leleyv2x2bbldlyld6ffbhq6qu
more »... cal examples are included to verify and investigate performance of non-iterative algorithm in the developed method.
ACM SIGBED Review
It has been widely studied how to schedule real-time tasks on multiprocessor platforms. Several studies find optimal scheduling policies for implicit deadline task systems, but it is hard to understand how each policy utilizes the two important aspects of scheduling real-time tasks on multiprocessors:inter-job concurrency and job urgency. In this paper, we introduce a new scheduling policy that considers these two properties. We prove that the policy is optimal for the special case when thedoi:10.1145/1851166.1851173 fatcat:xgkmff567nfhznhtqigdkwbyhy
more »... ution time of all tasks are equally one and deadlines are implicit, and observe that the policy is a new concept in that it is not an instance of Pfair or ERfair. It remains open to find a schedulability condition for general task systems under our scheduling policy. Abstract-It has been widely studied how to schedule realtime tasks on multiprocessor platforms. Several studies find optimal scheduling policies for implicit deadline task systems, but it is hard to understand how each policy utilizes the two important aspects of scheduling real-time tasks on multiprocessors: inter-job concurrency and job urgency. In this paper, we introduce a new scheduling policy that considers these two properties. We prove that the policy is optimal for the special case when the execution time of all tasks are equally one and deadlines are implicit, and observe that the policy is a new concept in that it is not an instance of Pfair or ERfair. It remains open to find a scheduliability condition for general task systems under our scheduling policy. During [t, t + D * 1 (t)], D * 1 (t) · m jobs are serviced, and thus, using U * sys (t), we calculate the system dynamic density at t + D 1 (t) of this set as follows.
In general, an experimental environment for deep learning assumes that the training and the test dataset are sampled from the same distribution. However, in real-world situations, a difference in the distribution between two datasets, domain shift, may occur, which becomes a major factor impeding the generalization performance of the model. The research field to solve this problem is called domain generalization, and it alleviates the domain shift problem by extracting domain-invariant featuresarXiv:2104.09841v1 fatcat:5hxntvaz6retrhbi6zk5jejnxa
more »... explicitly or implicitly. In recent studies, contrastive learning-based domain generalization approaches have been proposed and achieved high performance. These approaches require sampling of the negative data pair. However, the performance of contrastive learning fundamentally depends on quality and quantity of negative data pairs. To address this issue, we propose a new regularization method for domain generalization based on contrastive learning, self-supervised contrastive regularization (SelfReg). The proposed approach use only positive data pairs, thus it resolves various problems caused by negative pair sampling. Moreover, we propose a class-specific domain perturbation layer (CDPL), which makes it possible to effectively apply mixup augmentation even when only positive data pairs are used. The experimental results show that the techniques incorporated by SelfReg contributed to the performance in a compatible manner. In the recent benchmark, DomainBed, the proposed method shows comparable performance to the conventional state-of-the-art alternatives. Codes are available at https://github.com/dnap512/SelfReg.
Lee et al. demonstrate that web pages can be reconstructed by acquisition of deallocated pages  . Kong et al. present a method to recover messages from a web mail client via memory dumps  . ...doi:10.1587/elex.14.20170309 fatcat:rxqvitzfbjebplzvuytbgifbq4
Publication in the conference proceedings of EUSIPCO, Marrakech, Morocco, 2013doi:10.5281/zenodo.43690 fatcat:awkxxorlkrbfzmba6j6c4xr52u
Design of AT-DMB Baseband Receiver SoC Joohyun Lee, Hyuk Kim, Jinkyu Kim, Bontae Koo, Nakwoong Eum, and Hyuckjae Lee Fig. 1 . 1 Hierarchical modulation procedure (LP:QPSK). ...doi:10.4218/etrij.09.1209.0009 fatcat:ufoupekkrvbabh5rqmifj3reoq
Also, Lee et al. proposed a new schedulability analysis [11, 12] using the well-known schedulability analysis technique for preemptive scheduling algorithms, called RTA (Response Time Analysis). ...doi:10.3390/sym12010172 fatcat:4uescstrtjahhgrlxscxuqtkzm
For timing guarantees of a set of real-time tasks under a target scheduling algorithm, a number of schedulability tests have been studied. However, there still exist many task sets that are potentially schedulable by a target scheduling algorithm, but proven schedulable by none of existing schedulability tests, especially on a multiprocessor platform. In this paper, we propose a new notion of time-reversibility of schedulability tests, which yields tighter schedulability guarantees by viewingdoi:10.1109/rtss.2014.18 dblp:conf/rtss/Lee14 fatcat:y7mgwq4vondjxpq3q5nt6bxf7u
more »... al-time scheduling under a change in the sign of time. To this end, we first define the notion of a time-reversed scheduling algorithm against a target scheduling algorithm; for example, the time-reversed scheduling algorithm against EDF (Earliest Deadline First) is LCFS (Last-Come, First-Served), and the converse also holds. Then, a schedulability test for a scheduling algorithm is said to be time-reversible with respect to schedulability, if all task sets deemed schedulable by the test are also schedulable by its time-reversed scheduling algorithm. To exploit the notion of time-reversibility for tighter schedulability guarantees, we not only prove timereversibility of an existing schedulability test, but also develop a new time-reversible schedulability test, both of which cover additional schedulable task sets. Next, we generalize the time-reversibility theory towards partial execution. Utilizing the notion, we can assure the schedulability of a task under a target scheduling algorithm in a divide-and-conquer manner: (i) the first some units of execution guaranteed by a schedulability test for the scheduling algorithm, and (ii) the remaining execution guaranteed by a time-reversible (with respect to partial execution) schedulability test for its time-reversed scheduling algorithm. Such a divide-and-conquer approach has not been directly applied to existing schedulability tests in that they cannot address (ii) effectively. As a case study, this paper develops RTA (Response-Time Analysis) for LCFS, proves its time-reversibility, and applies the divide-and-conquer approach to the test along with an existing EDF schedulability test. Our simulation results show that the time-reversibility theory helps to find up to 13.1% additional EDF-schedulable task sets on a multiprocessor platform.
To account for phenomenological theories and a set of invariants, stress and strain are usually decomposed into a pair of pressure and deviatoric stress and a pair of volumetric strain and deviatoric strain. However, the conventional decomposition method only focuses on individual stress and strain, so that cannot be directly applied to either formulation in Finite Element Method (FEM) or Boundary Element Method (BEM). In this paper, a simpler, more general, and widely applicable decompositionarXiv:1211.2693v1 fatcat:acngdducenawzkaz64jcr34prm
more »... s suggested. A new decomposition method adopts multiplying decomposition tensors or matrices to not only stress and strain but also constitutive and compliance relation. With this, we also show its practical usage on FEM and BEM in terms of tensors and matrices.
Please cite this article in press as: Lee, J., et al., Zero-laxity based real-time multiprocessor scheduling. ... ., 2010; Lee et al., 2011; Stavrinides and Karatza, 2011) . ... Please cite this article in press as: Lee, J., et al., Zero-laxity based real-time multiprocessor scheduling. J. Syst. Software (2011), doi:10.1016/j.jss.2011.07.002 ...doi:10.1016/j.jss.2011.07.002 fatcat:g3tv2bkdfzbrznjpss3skqi6ri
The recent success of the generative model shows that leveraging the multi-modal embedding space can manipulate an image using text information. However, manipulating an image with other sources rather than text, such as sound, is not easy due to the dynamic characteristics of the sources. Especially, sound can convey vivid emotions and dynamic expressions of the real world. Here, we propose a framework that directly encodes sound into the multi-modal (image-text) embedding space andarXiv:2112.00007v1 fatcat:rcv7bt5ppvfihc57kqfa4d2pau
more »... an image from the space. Our audio encoder is trained to produce a latent representation from an audio input, which is forced to be aligned with image and text representations in the multi-modal embedding space. We use a direct latent optimization method based on aligned embeddings for sound-guided image manipulation. We also show that our method can mix text and audio modalities, which enrich the variety of the image modification. We verify the effectiveness of our sound-guided image manipulation quantitatively and qualitatively. We also show that our method can mix different modalities, i.e., text and audio, which enrich the variety of the image modification. The experiments on zero-shot audio classification and semantic-level image classification show that our proposed model outperforms other text and sound-guided state-of-the-art methods.
This paper introduces a tagless cache architecture for large in-package DRAM caches. The conventional die-stacked DRAM cache has both a TLB and a cache tag array, which are responsible for virtual-to-physical and physical-to-cache address translation, respectively. We propose to align the granularity of caching with OS page size and take a unified approach to address translation and cache tag management. To this end, we introduce cache-map TLB (cTLB), which stores virtual-to-cache, instead ofdoi:10.1145/2872887.2750383 fatcat:td3aznb73zfanmsbh774wxndji
more »... rtual-to-physical, address mappings. At a TLB miss, the TLB miss handler allocates the requested block into the cache if it is not cached yet, and updates both the page table and cTLB with the virtual-tocache address mapping. Assuming the availability of large in-package DRAM caches, this ensures that an access to the memory region within the TLB reach always hits in the cache with low hit latency since a TLB access immediately returns the exact location of the requested block in the cache, hence saving a tag-checking operation. The remaining cache space is used as victim cache for memory pages that are recently evicted from cTLB. By completely eliminating data structures for cache tag management, from either on-die SRAM or inpackage DRAM, the proposed DRAM cache achieves best scalability and hit latency, while maintaining high hit rate of a fully associative cache. Our evaluation with 3D Through-Silicon Via (TSV)-based in-package DRAM demonstrates that the proposed cache improves the IPC and energy efficiency by 30.9% and 39.5%, respectively, compared to the baseline with no DRAM cache. These numbers translate to 4.3% and 23.8% improvements over an impractical SRAM-tag cache requiring megabytes of on-die SRAM storage, due to low hit latency and zero energy waste for cache tags.
« Previous Showing results 1 — 15 out of 261 results