L3E-HD: A Framework Enabling Efficient Ensemble in High-Dimensional Space for Language Tasks

Fangxin Liu, Haomin Li, Xiaokang Yang, Li Jiang
2022 Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  
Brain-inspired hyperdimensional computing (HDC) has been introduced as an alternative computing paradigm to achieve efficient and robust learning. Language tasks, generally solved using machine learning methods, are widely deployed on low-power embedded devices. However, existing HDC solutions suffer from major challenges that impede the deployment of low-power embedded devices: the storage and computation overhead of HDC models grows dramatically with (i) the number of dimensions and (ii) the
more » ... omplex similarity metric during the inference. In this paper, we proposed a novel ensemble framework for the language task, termed L3E-HD, which enables efficient HDC on low-power edge devices. L3E-HD accelerates the inference by mapping data points to a high-dimensional binary space to simplify similarity search, which dominates costly and frequent operation in HDC. Through marrying HDC with the ensemble technique, L3E-HD also addresses the severe accuracy degradation induced by the compression of the dimension and precision of the model. Our experiments show that the ensemble technique is naturally a perfect fit to boost HDCs. We find that our L3E-HD, which is faster, more efficient, and more accurate than conventional machine learning methods, can even surpass the accuracy of the full-precision model at a smaller model size. Code is released at: https://github.com/ MXHX7199/SIGIR22-EnsembleHDC. CCS CONCEPTS • Computing methodologies → Bio-inspired approaches.
doi:10.1145/3477495.3531761 fatcat:fvpa2jlwyrhezkrelneeudnbta