Enhancing Sentence Embedding with Generalized Pooling [article]

Qian Chen, Zhen-Hua Ling, Xiaodan Zhu
2018 arXiv   pre-print
Pooling is an essential component of a wide variety of sentence representation and embedding models. This paper explores generalized pooling methods to enhance sentence embedding. We propose vector-based multi-head attention that includes the widely used max pooling, mean pooling, and scalar self-attention as special cases. The model benefits from properly designed penalization terms to reduce redundancy in multi-head attention. We evaluate the proposed model on three different tasks: natural
more » ... nguage inference (NLI), author profiling, and sentiment classification. The experiments show that the proposed model achieves significant improvement over strong sentence-encoding-based methods, resulting in state-of-the-art performances on four datasets. The proposed approach can be easily implemented for more problems than we discuss in this paper.
arXiv:1806.09828v1 fatcat:mqkw33awmzfnnhzdpoqgigloze