Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA [article]

Cheng Fu, Shilin Zhu, Hao Su, Ching-En Lee, Jishen Zhao
2018 arXiv   pre-print
Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN by using a single bit (-1/+1) for network parameters and intermediate representations, which has greatly reduced the off-chip data transfer and storage overhead. However, a large amount of computation redundancy still exists in BNN inference. By analyzing local properties of images and the learned BNN kernel weights, we observe an average of ∼78 and ∼59 metric in common network architectures. Thus there does exist
more » ... ancy that can be exploited to further reduce the amount of on-chip computations. Motivated by the observation, in this paper, we proposed two types of fast and energy-efficient architectures for BNN inference. We also provide analysis and insights to pick the better strategy of these two for different datasets and network models. By reusing the results from previous computation, much cycles for data buffer access and computations can be skipped. By experiments, we demonstrate that 80 skipped by exploiting BNN similarity. Thus, our design can achieve 17 reduction in total power consumption, 54 consumption and 2.4× maximum speedup, compared to the baseline without applying our reuse technique. Our design also shows 1.9× more area-efficiency compared to state-of-the-art BNN inference design. We believe our deployment of BNN on FPGA leads to a promising future of running deep learning models on mobile devices.
arXiv:1810.02068v1 fatcat:6ttblslosnfu5bgpajuvhs3hzy