A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Memory-Efficient Differentiable Transformer Architecture Search
[article]
2021
arXiv
pre-print
Preserved Fulltext
Differentiable architecture search (DARTS) is successfully applied in many vision tasks. ...
However, directly using DARTS for Transformers is memory-intensive, which renders the search process infeasible. To this end, we propose a multi-split reversible network and combine it with DARTS. ...
Reversible networks
Conclusion We have proposed a memory-efficient differentiable architecture search (DARTS) method on sequence-to-sequence tasks. ...
arXiv:2105.14669v1
fatcat:iqvywprvm5dpvmpwviypy7qhti
TND-NAS: Towards Non-Differentiable Objectives in Differentiable Neural Architecture Search
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Differentiable architecture search has gradually become the mainstream research topic in the field of Neural Architecture Search (NAS) for its high efficiency compared with the early NAS (EA-based, RL-based ...
Recent differentiable NAS also aims at further improving the search performance and reducing the GPU-memory consumption. ...
Even with high search efficiency, differentiable NAS researches rarely involves the non-differentiable objectives, e.g., energy, latency, or memory consumption, and multi-objective are merely jointly considered ...
arXiv:2111.03892v2
fatcat:wgn45wznjvcntp6evov4so25gm
ReconfigISP: Reconfigurable Camera Image Processing Pipeline
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
search and effectively search for the optimal ISP architecture. ...
In particular, we implement several ISP modules, and enable backpropagation for each module by training a differentiable proxy, hence allowing us to leverage the popular differentiable neural architecture ...
With all modules differentiable, the ISP architecture can be efficiently explored with the help of neural architecture search. ...
arXiv:2109.04760v1
fatcat:7khy3l2s5bdfbnp2cxzwd4uukq
Evolved Speech-Transformer: Applying Neural Architecture Search to End-to-End Automatic Speech Recognition
2020
Interspeech 2020
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
With a combination of carefully designed search space and Progressive dynamic hurdles, a genetic algorithm based, our algorithm finds a memory-efficient architecture which outperforms vanilla Transformer ...
Neural architecture search (NAS) has been successfully applied to finding efficient, high-performance deep neural network architectures in a task-adaptive manner without extensive human intervention. ...
In this work, we designed a search space apt for speech recognition and applied PDH to find a memory-efficient architecture which performs well across standard benchmark datasets with reduced training ...
doi:10.21437/interspeech.2020-1233
dblp:conf/interspeech/KimWKL20
fatcat:pseebobwqfgkpgztxmnqnsjvwq
Joint Search of Data Augmentation Policies and Network Architectures
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2012.09407v1 fatcat:hoihigqj3nck7j4myare3rwabi
The proposed method combines differentiable methods for augmentation policy search and network architecture search to jointly optimize them in the end-to-end manner. ...
The core idea of our approach is to make the whole part differentiable. ...
memory and power efficiency at inference time by model compression. ...
arXiv:2012.09407v2
fatcat:otozvh2rz5btfn5hd7cacgei4y
Memory-efficient Embedding for Recommendations
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Most of them empirically allocate a unified dimension to all feature fields, which is memory inefficient. ...
Specifically, we first proposed an end-to-end differentiable framework that can calculate the weights over various dimensions for feature fields in a soft and continuous manner with an AutoML based optimization ...
Inspired by the differentiable architecture search (DARTS) techniques [24] , W and are alternately optimized through gradient descent. ...
arXiv:2006.14827v2
fatcat:2ujor3sqf5aild7r7rkknwxvtm
Mutually-aware Sub-Graphs Differentiable Architecture Search
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Differentiable architecture search is prevalent in the field of NAS because of its simplicity and efficiency, where two paradigms, multi-path algorithms and single-path methods, are dominated. ...
In this paper, we propose a conceptually simple yet efficient method to bridge these two paradigms, referred as Mutually-aware Sub-Graphs Differentiable Architecture Search (MSG-DAS). ...
PC-DARTS [48] reduces the memory usage by performing architecture search in a randomly sampled subset of operations. ...
arXiv:2107.04324v3
fatcat:xi47dor6ljdp3iygqhvgcvucwq
SparseMask: Differentiable Connectivity Learning for Dense Image Prediction
2019
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we aim at automatically searching an efficient network architecture for dense image prediction. ...
We then employ gradient descent to search the optimal connectivity from the dense connections. ...
After training, we prune FDN to obtain the final architecture with sparse connectivity (Figure 1c ), which is time and memory efficient. ...
doi:10.1109/iccv.2019.00687
dblp:conf/iccv/WuZH19
fatcat:wy3x2fdqkzgdtbeesln2jggjje
Searching for A Robust Neural Architecture in Four GPU Hours
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this way, our approach can be trained in an end-to-end fashion by gradient descent, named Gradient-based search using Differentiable Architecture Sampler (GDAS). ...
We propose an efficient NAS approach learning to search by gradient descent. Our approach represents the search space as a directed acyclic graph (DAG). ...
Conclusion In this paper, we propose a Gradient-based neural architecture search approach using Differentiable Architecture Sampler (GDAS). ...
arXiv:1910.04465v2
fatcat:fsuntwbkbngbhl5aopl5rfxkym
An auto-tuning framework for parallel multicore stencil computations
2010
2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
Overall we demonstrate that such domain-specific auto-tuners hold enormous promise for architectural efficiency, programmer productivity, performance portability, and algorithmic adaptability on existing ...
Although stencil auto-tuning has shown tremendous potential in effectively utilizing architectural resources, it has hitherto been limited to single kernel instantiations; in addition, the large variety ...
most power efficient architecture evaluated in this study. ...
doi:10.1109/ipdps.2010.5470421
dblp:conf/ipps/KamilCOSW10
fatcat:endkmhvhjjfuvmnpmuxp7elpm4
Searching for a Robust Neural Architecture in Four GPU Hours
2019
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
In this way, our approach can be trained in an end-to-end fashion by gradient descent, named Gradient-based search using Differentiable Architecture Sampler (GDAS). ...
We propose an efficient NAS approach learning to search by gradient descent. Our approach represents the search space as a directed acyclic graph (DAG). ...
Conclusion In this paper, we propose a Gradient-based neural architecture search approach using Differentiable Architecture Sampler (GDAS). ...
doi:10.1109/cvpr.2019.00186
dblp:conf/cvpr/DongY19
fatcat:lz6dsbsqazddbhqnd2ims27rxu
Representation Sharing for Fast Object Detector Search and Beyond
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
FAD consists of a designed search space and an efficient architecture search algorithm. The search space contains a rich set of diverse transformations designed specifically for object detection. ...
To enhance such capability, we propose an extremely efficient neural architecture search method, named Fast And Diverse (FAD), to better explore the optimal configuration of receptive fields and convolution ...
A stream of efficient NAS methods is the differentiable NAS [26, 23] . ...
arXiv:2007.12075v4
fatcat:wj2hhhnyrfgivg2wr5sx3msgsm
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
2022
ACM Transactions on Design Automation of Electronic Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2022 pre-print arXiv:2204.11786v1 fatcat:mm5ex4bkrbfwxkwmnbdjao6wky
To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. ...
This article provides an overview of efficient deep learning methods, systems, and applications. ...
It then employs differentiable NAS to search for models with low memory usage and low op count. Rusci et al. ...
doi:10.1145/3486618
fatcat:h6xwv2slo5eklift2fl24usine
Learning Algorithmic Solutions to Symbolic Planning Tasks with a Neural Computer Architecture
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Therefore, we present a novel architecture, based on memory-augmented networks, that is inspired by the von Neumann and Harvard architectures of modern computers. ...
This architecture enables the learning of abstract algorithmic solutions via Evolution Strategies in a reinforcement learning setting. ...
Due to the design of the interface and the attention mechanisms, the DNC is independent of the memory size and fully differentiable, allowing gradient-based end-to-end learning. Our architecture. ...
arXiv:1911.00926v2
fatcat:h23q6v2lkvdgrht7ecbnfchviq
DSNAS: Direct Neural Architecture Search without Parameter Retraining
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Seeing that most existing methods do not solve this problem directly, we propose DSNAS, an efficient differentiable NAS framework that simultaneously optimizes architecture and parameters with a low-biased ...
However, performance of the same architecture in the two stages correlates poorly. In this work, we propose a new problem definition for NAS, task-specific end-to-end, based on this observation. ...
Our proposal is an efficient differentiable NAS framework, Discrete Stochastic Neural Architecture Search (DSNAS). ...
arXiv:2002.09128v2
fatcat:lmv7yx6wxfc63ilye5vfjtj6ii
« Previous
Showing results 1 — 15 out of 58,768 results