A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision
2021
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
unpublished
Preserved Fulltext
Limited efforts have been made by previous studies to deal with such problems. In this work, we propose a self-supervised CWS approach with a straightforward and effective architecture. ...
Recent state-of-the-art (SOTA) effective neural network methods and fine-tuning methods based on pre-trained models (PTM) have been used in Chinese word segmentation (CWS), and they achieve great results ...
We thank all anonymous reviewers for their valuable comments and suggestions on this work. ...
doi:10.18653/v1/2021.emnlp-main.158
fatcat:mzqhyi3uxrgyjm5keamixlj27e
Discriminative Self-training for Punctuation Prediction
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we propose a Discriminative Self-Training approach with weighted loss and discriminative label smoothing to exploit unlabeled speech transcripts. ...
Experimental results on the English IWSLT2011 benchmark test set and an internal Chinese spoken language dataset demonstrate that the proposed approach achieves significant improvement on punctuation prediction ...
We also compare the proposed approach with previous methods on a large internal Chinese dataset [1] . We use Jieba 2 for Chinese word segmentation. ...
arXiv:2104.10339v2
fatcat:gfnf5xxkzzej7akmebwulho7bm
How Context Affects Language Models' Factual Predictions
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We report that augmenting pre-trained language models in this way dramatically improves performance and that the resulting system, despite being unsupervised, is competitive with a supervised machine reading ...
Furthermore, processing query and context with different segment tokens allows BERT to utilize its Next Sentence Prediction pre-trained classifier to determine whether the context is relevant or not, substantially ...
One possible reason for this phenomenon resides in the Next Sentence Prediction (NSP) classifier of BERT, learned with self-supervision during pretraining by training the model to distinguish contiguous ...
arXiv:2005.04611v1
fatcat:uylkyzvscve5dgcofihq7g3mfu
SpanBERT: Improving Pre-training by Representing and Predicting Spans
2020
Transactions of the Association for Computational Linguistics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to predict the entire content of the masked span, without ...
We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. ...
We thank the anonymous reviewers, the action editor, and our colleagues at Facebook AI Research and the University of Washington for their insightful feedback that helped improve the paper. ...
doi:10.1162/tacl_a_00300
fatcat:2i52pub37jcghhxp43gjw5cvve
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
2020
International Conference on Language Resources and Evaluation
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Moreover, a word segmentation method called SentencePiece is utilized to further enhance Chinese Bert performance for tasks with long texts. ...
Hence a new pre-training task called Sentence Insertion (SI) is proposed in this paper for Chinese query-passage pairs NLP tasks including answer span prediction, retrieval question answering and sentence ...
In this table, WWM means the Whole Word Masking for Chinese. DA indicates the data augmentation for samples with unpaired queries and passages. ...
dblp:conf/lrec/ChenCJ20
fatcat:b2ub26e3zfbe5anwbsyc7gyili
Polyphone Disambiguition in Mandarin Chinese with Semi-Supervised Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Although the problem has been well explored with both knowledge-based and learning-based approaches, it remains challenging due to the lack of publicly available labeled datasets and the irregular nature ...
In this paper, we propose a novel semi-supervised learning (SSL) framework for Mandarin Chinese polyphone disambiguation that can potentially leverage unlimited unlabeled text data. ...
It is a common practice in the industry to leverage an existing text segmenter and a Chinese word-topinyin dictionary. ...
arXiv:2102.00621v2
fatcat:ahnwiycy4baizgasvvx5uuqbea
From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This design hinders prediction performance in video data from diverse domains and unseen scenarios. ...
However, human trajectory prediction still remains a challenging task, as scene semantics and human intent are difficult to model. ...
In the following methods (Word Hard Matching and Latent Topic with Word Embedding), we rst extract bag-of-words features from di erent modalities for each video and then match them using speci c matching ...
arXiv:2011.10670v3
fatcat:mlom5zqk6jdvjndcsfwimpj7xu
Self-Supervised Speech Representation Learning: A Review
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Although self-supervised speech representation is still a nascent research area, it is closely related to acoustic word embedding and learning with zero lexical resources, both of which have seen active ...
This review presents approaches for self-supervised speech representation learning and their connection to other research areas. ...
Networks are usually pre-trained with SSL techniques, augmented with prediction heads, and fine-tuned (or trained) with labeled data in downstream tasks for benchmarking. ...
arXiv:2205.10643v2
fatcat:6pveqmlbh5ebrhv2wuvb5hcp7q
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Additionally, LayoutLMv3 is pre-trained with a word-patch alignment objective to learn cross-modal alignment by predicting whether the corresponding image patch of a text word is masked. ...
Self-supervised pre-training techniques have achieved remarkable progress in Document AI. ...
The WPA objective is to predict whether the corresponding image patches of a text word are masked. ...
arXiv:2204.08387v3
fatcat:3tgkjyllzzhkdfasf25ytfrrfi
Stroke-Based Autoencoders: Self-Supervised Learners for Efficient Zero-Shot Chinese Character Recognition
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we develop a stroke-based autoencoder(SAE), to model the sophisticated morphology of Chinese characters with the self-supervised method. ...
One is fine-tuned on existing stroke-based method for zero-shot recognition of handwritten Chinese characters, and the other is applied to enrich the Chinese word embeddings from their morphological features ...
Acknowledgement This work is partially supported by the National Natural Science Foundation of China (61573012), China Scholarship Council (201906955038), and National innovation and entrepreneurship training ...
arXiv:2207.08191v1
fatcat:xpngz2xy7vchjlsvxr4xz33kge
A Feasible Framework for Arbitrary-Shaped Scene Text Recognition
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Our method wins the championship on Scene Text Spotting Task (Latin Only, Latin and Chinese) of ICDAR2019 Robust Reading Challenge on ArbitraryShaped Text Competition. ...
In this paper, we propose a feasible framework for multi-lingual arbitrary-shaped STR, including instance segmentation based text detection and language model based attention mechanism for text recognition ...
Then the mask branch predicts the final segmented text region for latter text recognition. ...
arXiv:1912.04561v2
fatcat:5rtipn2hsjefjifv57ouizawoi
Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We further propose a masked segment prediction task to push the model to learn from rich but redundant information inherent in lattices, while avoiding learning unexpected tricks. ...
In this work, we propose a novel pre-training paradigm for Chinese -- Lattice-BERT, which explicitly incorporates word representations along with characters, thus can model a sentence in a multi-granularity ...
We also adopt the whole word masking trick. LBERT is our proposed Lattice-BERT model, with word lattices as inputs, equipping with lattice position attentions and masked segment prediction. ...
arXiv:2104.07204v2
fatcat:2xxwbavxqbc6bbodlmdhfirn5a
Instance Segmentation Network with Selfdistillation for Scene Text Detection
2020
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Segmentation based methods have become the mainstream for detecting scene text with arbitrary orientations and shapes. ...
In this paper, we propose an instance segmentation network (ISNet), which simultaneously generates prototype masks and per-instance mask coefficients. ...
OPTIMIZATION Multiple loss functions, including L mask for instance segmentation, L sd for self-distillation, and L norm , can be back-propagated to optimize the text detection model in a supervised manner ...
doi:10.1109/access.2020.2978225
fatcat:tylrcfwzcvfijn47dgthjnvtui
A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
For Mandarin text inputs, Chinese dialect TTS can only generate partly-meaningful speech with relatively poor prosody and naturalness. ...
To lower the bar of use and make it more practical in commercial, we propose a novel Chinese dialect TTS frontend with a translation module. ...
CWS aims to address that Chinese words have no formal natural segmentation symbols. ...
arXiv:2206.04922v2
fatcat:u46vg7c24rglrd653g4uiv47va
Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2107.09278v1 fatcat:iqyhkzevnbgwpaoakjqyhiotby
We propose a sequence model with self-adaptive sliding window for accurate and efficient paragraph segmentation. ...
Automatically predicting paragraph segmentation for spoken documents may both improve readability and downstream NLP performance such as summarization and machine reading comprehension. ...
StructBERT [15] augments the BERT pre-training objectives with a word structural objective and a sentence structural objective. ...
arXiv:2107.09278v2
fatcat:3t7rya67ofgnzey3ntwfrmuole
« Previous
Showing results 1 — 15 out of 2,028 results