Extensive Pyramid Networks for Text Classification

Linqing Shi, Zeping Yu, Gongshen Liu
2019 Australian Journal of Intelligent Information Processing Systems  
Text classification is an important task in Natural Language Processing. Traditionally, human-designed features are used on text classifiers. In recent years, methods based on word embeddings and deep learning have made a great breakthrough. Deep neural networks such as convolutional neural networks and recurrent neural networks have achieved very good results. The attention mechanism, especially self-attention, has also gained great performance on many NLP tasks. This paper describes an
more » ... ve pyramid network for text classification. The pyramid architecture is simple and elegant by using max pooling layers in many pyramid blocks. In each pyramid block, self-attention layers are used to capture the relationship between the words in the sequences. In addition, recurrent layers are used to obtain the order information and convolutional layers are applied to get the local contextual information. Experiments on five large-scale text classification datasets show that our network achieves better performance than the previous models.
dblp:journals/ajiips/ShiYL19 fatcat:zqt6eu6mxfaepjitd2qlvoetja