Filters








74,747 Hits in 4.3 sec

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning [article]

Yixuan Su and Fangyu Liu and Zaiqiao Meng and Tian Lan and Lei Shu and Ehsan Shareghi and Nigel Collier
2022 arXiv   pre-print
In this work, we propose TaCL (Token-aware Contrastive Learning), a novel continual pre-training approach that encourages BERT to learn an isotropic and discriminative distribution of token representations  ...  The results show that TaCL brings consistent and notable improvements over the original BERT model. Furthermore, we conduct detailed analysis to reveal the merits and inner-workings of our approach.  ...  Language model pre-training aims to improve the system's performance on downstream NLU tasks. All pre-training corpora and pre-trained models used in this work are publically available.  ... 
arXiv:2111.04198v4 fatcat:dfenijgyqrfxngzyc52il7rn7y

An Auxiliary Classifier Generative Adversarial Framework for Relation Extraction [article]

Yun Zhao
2019 arXiv   pre-print
In AC-GAN, the discriminator gives not only a probability distribution over the real source, but also a probability distribution over the relation labels.  ...  In this paper, we use Auxiliary Classifier Generative Adversarial Networks (AC-GANs) to generate high-quality relational sentences and to improve the performance of relation classifier in end-to-end models  ...  Since the generator needs rewards from discriminator to update, the discriminator gives both a probability distribution over the real source and a probability distribution over the relation labels.  ... 
arXiv:1909.05370v1 fatcat:t76bbzuufbg3ve7ilhi6yea6du

Decoding and Diversity in Machine Translation [article]

Nicholas Roberts, Davis Liang, Graham Neubig, Zachary C. Lipton
2020 arXiv   pre-print
Moreover, search tends to bias the distribution of translated gender pronouns.  ...  To improve systems with respect to these metrics, NLP researchers employ a variety of heuristic techniques, including searching for the conditional mode (vs. sampling) and incorporating various training  ...  Evaluating the L1 distance between the sentence length distributions, we find that both sampling and beam search produce outputs with a similar distribution of sequence lengths to ground truth candidates  ... 
arXiv:2011.13477v1 fatcat:5l4oxdzv7zb7rdxaymn3xdb6dq

Generating Text through Adversarial Training using Skip-Thought Vectors [article]

Afroz Ahamad
2018 arXiv   pre-print
This work presents an approach to text generation using Skip-Thought sentence embeddings in conjunction with GANs based on gradient penalty functions and f-measures.  ...  The results of using sentence embeddings with GANs for generating text conditioned on input information are comparable to the approaches where word embeddings are used.  ...  The generator network produces sentence vectors similar to those from the encoded real distribution.  ... 
arXiv:1808.08703v2 fatcat:yiwkvds4yrepfcn5y7yfzuyzwa

Adversarial Text Generation Without Reinforcement Learning [article]

David Donahue, Anna Rumshisky
2019 arXiv   pre-print
This is largely because sequences of text are discrete, and thus gradients cannot propagate from the discriminator to the generator.  ...  We propose to utilize an autoencoder to learn a low-dimensional representation of sentences. A GAN is then trained to generate its own vectors in this space, which decode to realistic utterances.  ...  Plotted Sentences A quality generator should produce sentence representations that lie in similar neighborhoods to real sentences within the autoencoder-learned latent space.  ... 
arXiv:1810.06640v2 fatcat:tm44ai4nfvgllkpdee74zoywoi

Generating Text through Adversarial Training Using Skip-Thought Vectors

Afroz Ahamad
2019 Proceedings of the 2019 Conference of the North  
This study presents an approach to text generation using Skip-Thought sentence embeddings with GANs based on gradient penalty functions and f-measures.  ...  The proposed architecture aims to reproduce writing style in the generated text by modelling the way of expression at a sentence level across all the works of an author.  ...  Acknowledgements The author would like to thank Aruna Malapati for providing insights and access to an Nvidia Titan X GPU for the experiments; and Pranesh Bhargava, Greg Durrett and Yash Raj Jain for providing  ... 
doi:10.18653/v1/n19-3008 dblp:conf/naacl/Ahamad19 fatcat:2dim2p7k4rfcvlxtxi7vhs5emq

Beyond the Deep Metric Learning: Enhance the Cross-Modal Matching with Adversarial Discriminative Domain Regularization [article]

Li Ren, Kai Li, LiQiang Wang, Kien Hua
2020 arXiv   pre-print
The objective is to find efficient similarity metrics to compare the similarity between visual and textual information.  ...  In this paper, we address this limitation with an efficient learning objective that considers the discriminative feature distributions between the visual objects and sentence words.  ...  Similar to MS-COCO, each image sample also has 5 corresponding sentence labels.  ... 
arXiv:2010.12126v2 fatcat:we74xd3jdzdzlev2fewj7spf7m

GANs for Semi-Supervised Opinion Spam Detection [article]

Gray Stanton, Athirai A. Irissappane
2019 arXiv   pre-print
In this paper, we propose spamGAN, a generative adversarial network which relies on limited set of labeled data as well as unlabeled data for opinion spam detection. spamGAN improves the state-of-the-art  ...  This competition between the generator and discriminator improves the quality of the generated sentence.  ...  The generator, for a given class label, learns to generate new sentences (we call them fake 3 sentences) similar to the real sentences in the train set belonging to the same class.  ... 
arXiv:1903.08289v2 fatcat:nwhhoslqrrdz3adkpcgsfd63fq

Biomedical text summarization using Conditional Generative Adversarial Network(CGAN) [article]

Seyed Vahid Moravvej, Abdolreza Mirzaei, Mehran Safayani
2021 arXiv   pre-print
Unlike previous models, which often use greedy methods to select sentences, we use a new approach for selecting sentences.  ...  Experiments on the medical dataset show that the proposed method works on average 5% better than the competing models and is more similar to the reference summaries.  ...  The generator tries to be similar to the sentences given to the discriminator as a real vector. Summarization After training, only the generator is used to select the sentences.  ... 
arXiv:2110.11870v1 fatcat:sykqjjccafee3figytt4vedby4

DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text [article]

Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu Sun
2018 arXiv   pre-print
Moreover, we propose a novel language-model based discriminator, which can better distinguish novel text from repeated text without the saturation problem compared with existing classifier-based discriminators  ...  Existing text generation methods tend to produce repeated and "boring" expressions.  ...  Third, with the improvement of diversity, the generated data distribution of DP-GAN is closer to the real-world data distribution compared with that of MLE.  ... 
arXiv:1802.01345v3 fatcat:5hznaj46mfffvdygd4zkdqdn3i

Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation

Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu Sun
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
Moreover, we propose a novel languagemodel based discriminator, which can better distinguish novel text from repeated text without the saturation problem compared with existing classifier-based discriminators  ...  Existing text generation methods tend to produce repeated and "boring" expressions.  ...  Third, with the improvement of diversity, the generated data distribution of DP-GAN is closer to the real-world data distribution compared with that of MLE.  ... 
doi:10.18653/v1/d18-1428 dblp:conf/emnlp/XuRL018 fatcat:xoy56r6qazdobbkk7hxax2zwxi

Learning Compact Reward for Image Captioning [article]

Nannan Li, Zhenzhong Chen
2020 arXiv   pre-print
In this paper, we propose a refined Adversarial Inverse Reinforcement Learning (rAIRL) method to handle the reward ambiguity problem by disentangling reward for each word in a sentence, as well as achieve  ...  In addition, we introduce a conditional term in the loss function to mitigate mode collapse and to increase the diversity of the generated descriptions.  ...  For better clarity, policy π ψ is hereinafter referred to as the generated vocabulary distribution Discriminator The objective of the discriminator is to distinguish the true caption from the generated  ... 
arXiv:2003.10925v1 fatcat:se65gwk7fvae3kfpo6jyxvvase

Entity Disambiguation with Freebase

Zhicheng Zheng, Xiance Si, Fangtao Li, Edward Y. Chang, Xiaoyan Zhu
2012 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology  
Specifically, we explore both generative and discriminative models for each iteration.  ...  Experiments on 2, 430, 707 English sentences and 33, 743 Freebase entities show the effectiveness of the two features, where 90% accuracy can be reached without any labeled data.  ...  Accordingly, we update the Eq. 4 as p(i|e) = n(i, e) j n(j, e) (5) To incorporate the rich taxonomy, we assume that the feature distribution of all entities sharing the same type are similar, and they  ... 
doi:10.1109/wi-iat.2012.26 dblp:conf/webi/ZhengSLCZ12 fatcat:qpk4a5343baj7ovkxjunn3v7oq

Personality Privacy Protection Method of Social Users Based on Generative Adversarial Networks

Yi Sui, Xiujuan Wang, Kangfeng Zheng, Yutong Shi, Siwei Cao, Gennaro Vessio
2022 Computational Intelligence and Neuroscience  
In addition, the semantic and personality modules are designed to calculate the semantic similarity and personality distribution distance between the real text and the generated text as a part of the objective  ...  Making use of reinforcement learning, we use the output of the discriminator as a reward signal to guide the training of the generator.  ...  In order to ensure that the synthetic sentence matches the real sentence and to improve the semantic similarity, we feed the source text to the token encoded at each LSTM time step after being processed  ... 
doi:10.1155/2022/2419987 pmid:35463264 pmcid:PMC9020900 fatcat:iucfhwlhhbfptgnsqq35tjjyea

FA-GAN: Feature-Aware GAN for Text to Image Synthesis [article]

Eunyeong Jeon, Kunhee Kim, Daijin Kim
2021 arXiv   pre-print
To address this issue, we propose Feature-Aware Generative Adversarial Network (FA-GAN) to synthesize a high-quality image by integrating two techniques: a self-supervised discriminator and a feature-aware  ...  Secondly, we introduce a feature-aware loss to provide the generator more direct supervision by employing the feature representation from the self-supervised discriminator.  ...  from real distribution P data , and (x,s) is the mismatched image and sentence.  ... 
arXiv:2109.00907v1 fatcat:yzq3s4tbprg7xmslvhdpxtokni
« Previous Showing results 1 — 15 out of 74,747 results