Filters








11 Hits in 3.8 sec

Exploring and Adapting Chinese GPT to Pinyin Input Method

Minghuan Tan, Yong Dai, Duyu Tang, Zhangyin Feng, Guoping Huang, Jing Jiang, Jiwei Li, Shuming Shi
2022 Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)   unpublished
In this work, we make the first exploration to leverage Chinese GPT for pinyin input method. We find that a frozen GPT achieves state-of-the-art performance on perfect pinyin.  ...  While GPT has become the de-facto method for text generation tasks, its application to pinyin input method remains unexplored.  ...  Conclusion In this paper, we explore how to adapt pretrained Chinese GPT to pinyin input method.  ... 
doi:10.18653/v1/2022.acl-long.133 fatcat:ms7cszn2p5dbfaixwzvkch2hsa

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information [article]

Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, Jiwei Li
2021 arXiv   pre-print
Recent pretraining models in Chinese neglect two important aspects specific to the Chinese language: glyph and pinyin, which carry significant syntax and semantic information for language understanding  ...  The glyph embedding is obtained based on different fonts of a Chinese character, being able to capture character semantics from the visual features, and the pinyin embedding characterizes the pronunciation  ...  We also want to acknowledge National Key R&D Program of China (2020AAA0105200) and Beijing Academy of Artificial Intelligence (BAAI).  ... 
arXiv:2106.16038v1 fatcat:rmdmt3dh7fhx3gu4pj4vmnmt4i

Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models [article]

Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, Jian-Guang Lou
2022 arXiv   pre-print
This motivates us to propose input-tuning, which fine-tunes both the continuous prompts and the input representations, leading to a more effective way to adapt unfamiliar inputs to frozen PLMs.  ...  For example, our preliminary exploration reveals a large performance gap between prompt-tuning and fine-tuning when unfamiliar inputs occur frequently in NLG tasks.  ...  E2E−− Translate both the attributes and the val- ues into French. ONR+ We use the canonical utterances for every logic forms. ONR− Convert the function names into Chinese Pinyin.  ... 
arXiv:2203.03131v1 fatcat:uwftri6g3bfo3i2kuikoz4rmw4

How to Fine-Tune BERT for Text Classification? [article]

Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang
2020 arXiv   pre-print
In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning.  ...  Language model pre-training has proven to be useful in learning universal language representations.  ...  Unlike Zhang et al. (2015) , we use the Chinese character directly rather than Pinyin. The dataset is a combination of the So-gouCA and SogouCS news corpora (Wang et al., 2008) .  ... 
arXiv:1905.05583v3 fatcat:6f7ozgdzc5ecpdhh3khd7ejfy4

MEduKG: A Deep-Learning-Based Approach for Multi-Modal Educational Knowledge Graph Construction

Nan Li, Qiang Shen, Rui Song, Yang Chi, Hao Xu
2022 Information  
We propose a speech-fusion method that links these data into the graph as a class of entities.  ...  The popularity of information technology has given rise to a growing interest in smart education and has provided the possibility of combining online and offline education.  ...  Acknowledgments: The authors would like to thank all of anonymous reviewers and editors for their helpful suggestions for the improvement of this paper.  ... 
doi:10.3390/info13020091 doaj:8bcca0df31df408d94e52d6aae0b674d fatcat:2x664jknnrhkfa4qkyfyribxcu

SemAttack: Natural Textual Attacks via Different Semantic Spaces [article]

Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, Bo Li
2022 arXiv   pre-print
We further demonstrate that SemAttack is general and able to generate natural adversarial texts for different languages (e.g., English and Chinese) with high attack success rates.  ...  Thus, the generated adversarial texts are more semantically close to the original inputs.  ...  Acknowledgments We gratefully thank the anonymous reviewers and meta-reviewers for their constructive feedback.  ... 
arXiv:2205.01287v3 fatcat:3nwcjl2ozfc6nlzvlnnfig2s3a

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation [article]

Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Srivastava, Samson Tan, Tongshuang Wu, Jascha Sohl-Dickstein (+113 others)
2021 arXiv   pre-print
(data splits according to specific features).  ...  In this paper, we present NL-Augmenter, a new participatory Python-based natural language augmentation framework which supports the creation of both transformations (modifications to the data) and filters  ...  on com- graph, etc.) by passing the input text to a GPT-2 mon abbreviations present in business communi- model (Radford et al., 2019).  ... 
arXiv:2112.02721v1 fatcat:uqizuxc4wzgxnnfsc6azh6ckpq

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information

Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, Jiwei Li
2021 Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)   unpublished
Recent pretraining models in Chinese neglect two important aspects specific to the Chinese language: glyph and pinyin, which carry significant syntax and semantic information for language understanding  ...  The glyph embedding is obtained based on different fonts of a Chinese character, being able to capture character semantics from the visual features, and the pinyin embedding characterizes the pronunciation  ...  Acknowledgement This work is supported by National Key R&D Program of China (2020AAA0105200) and Beijing Academy of Artificial Intelligence (BAAI).  ... 
doi:10.18653/v1/2021.acl-long.161 fatcat:tnrso3c6afcttgc5mjtbfuwhma

UnifieR: A Unified Retriever for Large-Scale Retrieval [article]

Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Kai Zhang, Daxin Jiang
2022 arXiv   pre-print
According to the encoding space, recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms.  ...  Large-scale retrieval is to recall relevant documents from a huge collection given a query. It relies on representation learning to embed documents and queries into a common semantic encoding space.  ...  Tai chi (simplified Chinese: ; traditional Chinese: ; pinyin: chi, an abbreviation of ;is an internal Chinese martial art (Chinese: ; pinyin: ) practiced for both its defense training and its health benefits  ... 
arXiv:2205.11194v1 fatcat:ttwz7flj75dg7ezchxsos2tfum

SemAttack: Natural Textual Attacks via Different Semantic Spaces

Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, Bo Li
2022 Findings of the Association for Computational Linguistics: NAACL 2022   unpublished
We further demonstrate that SemAttack is general and able to generate natural adversarial texts for different languages (e.g., English and Chinese) with high attack success rates.  ...  Thus, the generated adversarial texts are more semantically close to the original inputs.  ...  Acknowledgments We gratefully thank the anonymous reviewers and meta-reviewers for their constructive feedback.  ... 
doi:10.18653/v1/2022.findings-naacl.14 fatcat:aqs7ojvn7nf7nmig2om6xjwgv4

JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding [article]

Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang, Cong Liu, Ji-Rong Wen
2022 pre-print
This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model (PLM) for effectively understanding and representing mathematical  ...  Finally, we introduce a more difficult pre-training task that enforces the PLM to detect and correct the errors in its generated solutions.  ...  They consider modeling the linguistic characteristics or semantic knowledge of Chinese texts and devise special strategies to improve the task performance, e.g., whole word masking [4] , glyph and pinyin  ... 
doi:10.1145/3534678.3539131 arXiv:2206.06315v1 fatcat:f7jawk2tvnhkdjfgtlqulkj7bu