Efficient Synthesis of Compact Deep Neural Networks [article]

Wenhan Xia, Hongxu Yin, Niraj K. Jha
<span title="2020-04-18">2020</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Deep neural networks (DNNs) have been deployed in myriad machine learning applications. However, advances in their accuracy are often achieved with increasingly complex and deep network architectures. These large, deep models are often unsuitable for real-world applications, due to their massive computational cost, high memory bandwidth, and long latency. For example, autonomous driving requires fast inference based on Internet-of-Things (IoT) edge devices operating under run-time energy and
more &raquo; ... ory storage constraints. In such cases, compact DNNs can facilitate deployment due to their reduced energy consumption, memory requirement, and inference latency. Long short-term memories (LSTMs) are a type of recurrent neural network that have also found widespread use in the context of sequential data modeling. They also face a model size vs. accuracy trade-off. In this paper, we review major approaches for automatically synthesizing compact, yet accurate, DNN/LSTM models suitable for real-world applications. We also outline some challenges and future areas of exploration.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2004.08704v1">arXiv:2004.08704v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/g6gu7ng2zjda7minnahitn455a">fatcat:g6gu7ng2zjda7minnahitn455a</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200430193913/https://arxiv.org/pdf/2004.08704v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2004.08704v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>