Filters








12,783 Hits in 5.0 sec

Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention [article]

Yanzeng Li, Bowen Yu, Mengge Xue, Tingwen Liu
2020 arXiv   pre-print
Hence, we propose a novel word-aligned attention to exploit explicit word information, which is complementary to various character-based Chinese pre-trained language models.  ...  Most Chinese pre-trained models take character as the basic unit and learn representation according to character's external contexts, ignoring the semantics expressed in the word, which is the smallest  ...  This work is supported by the Strategic Priority Research Program of Chinese Academy of Sciences, Grant No. XDC02040400.  ... 
arXiv:1911.02821v2 fatcat:m7coyik2eja67g75stxpfcdmoe

Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention

Yanzeng Li, Bowen Yu, Xue Mengge, Tingwen Liu
2020 Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics   unpublished
Most Chinese pre-trained models take character as the basic unit and learn representation according to character's external contexts, ignoring the semantics expressed in the word, which is the smallest  ...  Hence, we propose a novel wordaligned attention to exploit explicit word information, which is complementary to various character-based Chinese pre-trained language models.  ...  This work is supported by the Strategic Priority Research Program of Chinese Academy of Sciences, Grant No. XDC02040400.  ... 
doi:10.18653/v1/2020.acl-main.315 fatcat:lcghqqpxbfezbhoirr7qz7gbuy

Multi-Granularity and Internal-External Correlation Residual Model for Chinese Sentence Semantic Matching

Lan Zhang, Hongmei Chen
2021 Fuzzy Systems and Data Mining  
Then, soft alignment attention is employed to enhance the correlation between characters/words in a sentence, called internal correlation, and the correlation between sentences, called external correlation  ...  Experimental results show that the proposed method achieves state-of-the-art performance for Chinese SSM, and, compared with pre-trained models, the method also achieves better performance with fewer parameters  ...  Comparison with pre-trained models. We compare MGIER with the pre-trained models as shown in Table 5 .  ... 
doi:10.3233/faia210185 dblp:conf/fsdm/ZhangC21 fatcat:eylv5vljofgjfoflkildlkm47i

Exploiting Word Semantics to Enrich Character Representations of Chinese Pre-trained Models [article]

Wenbiao Li, Rui Sun, Yunfang Wu
2022 arXiv   pre-print
Most of the Chinese pre-trained models adopt characters as basic units for downstream tasks.  ...  After that, we apply a word-to-character alignment attention mechanism to emphasize important characters by masking unimportant ones.  ...  with the baseline pre-trained models.  ... 
arXiv:2207.05928v1 fatcat:62bd2sni7bggbgrjjs6erw7kpy

Back Attention Knowledge Transfer for Low-Resource Named Entity Recognition [article]

Shengfei Lyu, Linghao Sun, Huixiong Yi, Yong Liu, Huanhuan Chen, Chunyan Miao
2021 arXiv   pre-print
BAN uses a translation system to translate other language sentences into English and then applies a new mechanism named back attention knowledge transfer to obtain task-specific information from pre-trained  ...  This strategy can transfer high-layer features of well-trained model and enrich the semantic representations of the original language.  ...  The hidden state size of the pre-trained English NER model is also 256. The aligned high-resource semantic feature has the same size as the hidden state of the pre-trained English NER model.  ... 
arXiv:1906.01183v3 fatcat:6h23amjkh5c7li5jixdup2sqqy

A Comparison on Fine-grained Pre-trained Embeddings for the WMT19Chinese-English News Translation Task

Zhenhao Li, Lucia Specia
2019 Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)  
Our systems are based on RNN architectures with pre-trained embeddings which utilize character and subcharacter information.  ...  We find that a finer granularity embeddings can help the model according to character level evaluation and that the pre-trained embeddings can also be beneficial for model performance marginally when the  ...  The two submitted systems use pre-trained embeddings enhanced by character and sub-character information respectively.  ... 
doi:10.18653/v1/w19-5324 dblp:conf/wmt/LiS19 fatcat:zvuqdk6l7vbmdpwztoqiyffa4u

Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter [article]

Wei Liu, Xiyan Fu, Yue Zhang, Wenming Xiao
2021 arXiv   pre-print
Lexicon information and pre-trained models, such as BERT, have been combined to explore Chinese sequence labelling tasks due to their respective strengths.  ...  In this paper, we propose Lexicon Enhanced BERT (LEBERT) for Chinese sequence labelling, which integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer.  ...  Tian et al. (2020b) enhanced the character-based Chinese POS tagging model with a multi-channel attention of N-grams. Pre-trained Model-based.  ... 
arXiv:2105.07148v3 fatcat:drc6vw5iave2plo6ndu4dhvfle

Porous Lattice-based Transformer Encoder for Chinese NER [article]

Xue Mengge, Yu Bowen, Liu Tingwen, Zhang Yue, Meng Erli, Wang Bin
2020 arXiv   pre-print
We first investigate the lattice-aware self-attention coupled with relative position representations to explore effective word information in the lattice structure.  ...  Incorporating lattices into character-level Chinese named entity recognition is an effective method to exploit explicit word information.  ...  PLTE enables the interaction between the matched lexical words and their constituent characters, and proceeds in batches with the lattice-aware self-attention.  ... 
arXiv:1911.02733v3 fatcat:wtntgq4cqjgb5nhgar36f5vbwm

Research on Semantic Similarity of Short Text Based on Bert and Time Warping Distance

Shijie Qiu, Yan Niu, Jun Li, Xing Li
2021 Journal of Web Engineering  
The model first uses the pre trained Bert model to extract the semantic features of the short text from the whole level, and obtains a 768 dimensional short text feature vector.  ...  Compared with other models, it can distinguish the semantic features of ambiguous words more accurately.  ...  The Attention mechanism mainly aims to enhance representation of semantic characteristics of the target character by learning information of characters in the context of the target character.  ... 
doi:10.13052/jwe1540-9589.20814 fatcat:djmadh33b5er7npv432iko2lam

Automatic Transferring between Ancient Chinese and Contemporary Chinese [article]

Zhiyuan Zhang, Wei Li, Xu Sun
2018 arXiv   pre-print
With this method, we build a large parallel corpus. We propose to apply the sequence to sequence model to automatically transfer between ancient and contemporary Chinese sentences.  ...  During the long time of development, Chinese language has evolved a great deal. Native speakers now have difficulty in reading sentences written in ancient Chinese.  ...  To train an automatic translating model, we need to build a sentence-aligned corpus first. Translation alignment is an important pre-step for machine translation.  ... 
arXiv:1803.01557v2 fatcat:ugb7dxy4anhy5caamak3fdm77a

A Hybrid Classification Method via Character Embedding in Chinese Short Text with Few Words

Yi Zhu, Yun Li, Yongzheng Yue, Jipeng Qiang, Yunhao Yuan
2020 IEEE Access  
INDEX TERMS Short text with few words, character embedding, attention mechanism, feature selection.  ...  To address these problems, this paper propose a hybrid classification method of Attention mechanism and Feature selection via Character embedding in Chinese short text with few words, called AFC.  ...  to pre-train a deep bidirectional transformer and text-pair representations.  ... 
doi:10.1109/access.2020.2994450 fatcat:vfcj2ocm55fidp46vsxs4p6ucu

Exploiting Japanese-Chinese Cognates with Shared Private Representations for Neural Machine Translation

Zezhong Li, Fuji Ren, Xiao Sun, Degen Huang, Piao Shi
2022 ACM Transactions on Asian and Low-Resource Language Information Processing  
In this article, we seek to strengthen the semantic correlation between Japanese and Chinese by leveraging cognate words that share common Chinese characters.  ...  words that can be utilized as additional linguistic knowledge to enhance translation performance.  ...  [10] provided a decoder-to-encoder attention mechanism with improved access to source word representations. Denis et al.  ... 
doi:10.1145/3533429 fatcat:y7qfjximirgyppq3uwm54pz6oe

Enhancing Chinese Intent Classification by Dynamically Integrating Character Features into Word Embeddings with Ensemble Techniques [article]

Ruixi Lin, Charles Costello, Charles Jankowski
2018 arXiv   pre-print
Chinese word embeddings alone can be inadequate for representing words, and pre-trained embeddings can suffer from not aligning well with the task at hand.  ...  To account for the inadequacy and leverage Chinese character information, we propose a low-effort and generic way to dynamically integrate character embedding based feature maps with word embedding based  ...  This is explained by the fact that the external source that pre-trains the word embeddings does not align well with the task at hand.  ... 
arXiv:1805.08914v1 fatcat:zu3e56jo2jgutgk6nyekamgw4e

SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task [article]

Zuchao Li, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita
2020 arXiv   pre-print
Based on different conditions of language pairs, we have experimented with diverse neural machine translation (NMT) techniques: document-enhanced NMT, XLM pre-trained language model enhanced NMT, bidirectional  ...  translation as a pre-training, reference language based UNMT, data-dependent gaussian prior objective, and BT-BLEU collaborative filtering self-training.  ...  We normalized punctuation, remove non-printing characters, and tokenize all data with the Moses tokenizer (Koehn et al., 2007) except for the Chinese.  ... 
arXiv:2010.05122v1 fatcat:rlgy4zy7pnawfhly2pn6gi74ce

Early stage visual-orthographic processes predict long-term retention of word form and meaning: A visual encoding training study

Fan Cao, Ben Rickles, Marianne Vu, Ziheng Zhu, Derek Ho Lung Chan, Lindsay N. Harris, Joseph Stafura, Yi Xu, Charles A. Perfetti
2013 Journal of Neurolinguistics  
We hypothesized that the character training effects would be seen in ERP components associated with word recognition and episodic memory.  ...  Adult learners of Chinese learned new characters through writing, visual chunking or reading-only.  ...  The coupling of meaning and sound recall with visual processing aligns with the assumption that a high quality orthographic representation is needed to support lexical identity in Chinese and in reading  ... 
doi:10.1016/j.jneuroling.2013.01.003 pmid:23798804 pmcid:PMC3689543 fatcat:d2rhomo3qngb7f4wgrasemanpq
« Previous Showing results 1 — 15 out of 12,783 results