Filters








26 Hits in 4.2 sec

Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

Leon Fröhling, Arkaitz Zubiaga
2021 PeerJ Computer Science  
While most of the current approaches rely on the availability of expensive language models, we propose a simple feature-based classifier for the detection problem, using carefully crafted features that  ...  of language models.  ...  The performance of the detector learned and evaluated on the GPT-3 model is surprisingly good, being even higher than for the GPT-2 xl generations.  ... 
doi:10.7717/peerj-cs.443 pmid:33954234 pmcid:PMC8049133 fatcat:xtmb2i7e4rekbe4z4phaxqu7pi

TweepFake: About detecting deepfake tweets

Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, Maurizio Tesconi, Qingzhong Liu
2021 PLoS ONE  
The recent advances in language modeling significantly improved the generative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a pre-trained language model that can autonomously generate  ...  We collected tweets from a total of 23 bots, imitating 17 human accounts. The bots are based on various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM, GPT-2.  ...  GROVER's authors [28] followed the fine-tuning based detection approach by using BERT, GPT2 and GROVER itself as the pre-trained language model.  ... 
doi:10.1371/journal.pone.0251415 pmid:33984021 fatcat:37wt7l2w3vdpdnr2udwj4fo7wu

TweepFake: about Detecting Deepfake Tweets [article]

Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, Maurizio Tesconi
2021 arXiv   pre-print
The recent advances in language modeling significantly improved the generative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a pre-trained language model that can autonomously generate  ...  We collected tweets from a total of 23 bots, imitating 17 human accounts. The bots are based on various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM, GPT-2.  ...  texts produced by transformer-based text generators such as GPT-2, Grover and CTRL.  ... 
arXiv:2008.00036v2 fatcat:ljuernmicjckblwxdgraolwjy4

Survey of Generative Methods for Social Media Analysis [article]

Stan Matwin, Aristides Milios, Paweł Prałat, Amilcar Soares, François Théberge
2021 arXiv   pre-print
We included two important aspects that currently gain importance in mining and modeling social media: dynamics and networks.  ...  This survey draws a broad-stroke, panoramic picture of the State of the Art (SoTA) of the research in generative methods for the analysis of social media data.  ...  Current SoTA approaches include the GROVER detector [253] and the RoBERTa detector [214] . The GROVER model is approximately the same size as GPT-2.  ... 
arXiv:2112.07041v1 fatcat:xgmduwctpbddfo67y6ack5s2um

The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? [article]

Toby Shevlane, Allan Dafoe
2020 arXiv   pre-print
The AI research community should consider concepts and policies from a broad set of adjacent fields, and ultimately needs to craft policy well-suited to its particular challenges.  ...  However, we show that the existing conversation within AI has imported concepts and conclusions from prior debates within computer security over the disclosure of software vulnerabilities.  ...  Gregory Lewis, Luke Muehlhauser, Carina Prunkl, Jacob Shapiro, Carl Shulman, David Siegel, Steven Weber, Toby Ord, and especially Ben Garfinkel.  ... 
arXiv:2001.00463v2 fatcat:pxc4altffrgdzevh6244f5lmme

Defending Against Neural Fake News [article]

Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, Yejin Choi
2020 arXiv   pre-print
We conclude by discussing ethical issues regarding the technology, and plan to release Grover publicly, helping pave the way for better detection of neural fake news.  ...  We thus present a model for controllable text generation called Grover.  ...  gifts by Google and Facebook.  ... 
arXiv:1905.12616v3 fatcat:dgeqsce3brcyhnege3xzmd2ooi

A Survey on Computational Propaganda Detection [article]

Giovanni Da San Martino, Stefano Cresci, Alberto Barron-Cedeno, Seunghak Yu, Roberto Di Pietro, Preslav Nakov
2020 arXiv   pre-print
In this survey, we review the state of the art on computational propaganda detection from the perspective of Natural Language Processing and Network Analysis, arguing about the need for combined efforts  ...  They exploit the anonymity of the Internet, the micro-profiling ability of social networks, and the ease of automatically creating and managing coordinated networks of accounts, to reach millions of social  ...  For instance, if a pre-trained language model such as GPT-2 is used as an automated propaganda generation method, it may become ineffective to detect propaganda when focusing on linguistic features alone  ... 
arXiv:2007.08024v1 fatcat:i4xhy7tgvbc45mxrjif4apne2a

Mining Disinformation and Fake News: Concepts, Methods, and Recent Advancements [article]

Kai Shu, Suhang Wang, Dongwon Lee, Huan Liu
2020 arXiv   pre-print
; (2) describing important and emerging tasks to combat disinformation for characterization, detection and attribution; and (3) discussing a weak supervision approach to detect disinformation with limited  ...  on detecting and mitigating disinformation; and (3) trending issues such as ethics, blockchain, clickbaits, etc.  ...  Rebecca Goolsby's vision on social bots and disinformation via interdisciplinary research.  ... 
arXiv:2001.00623v1 fatcat:zcmgzbudjvab3fckajrmrbppoy

Reasoning About Inconsistent Formulas

Joao Marques-Silva, Carlos Mencía
2020 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence  
The analysis of inconsistent formulas finds an ever-increasing range of applications, that include axiom pinpointing in description logics, fault localization in software, model-based diagnosis, optimization  ...  This paper overviews approaches for analyzing inconsistent formulas, focusing on finding and enumerating explanations of and corrections for inconsistency, but also on solving optimization problems modeled  ...  For instance, if a pre-trained language model such as GPT-2 is used as an automated propaganda generation method, it may become ineffective to detect propaganda when focusing on linguistic features alone  ... 
doi:10.24963/ijcai.2020/672 dblp:conf/ijcai/MartinoCBYPN20 fatcat:7fdnmougz5cllmi2no3qrd7wia

AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [article]

Yaman Kumar Singla, Swapnil Parekh, Somesh Singh, Junyi Jessy Li, Rajiv Ratn Shah, Changyou Chen
2021 arXiv   pre-print
To deal with these issues, we propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.  ...  Deep-learning based Automatic Essay Scoring (AES) systems are being actively used by states and language testing agencies alike to evaluate millions of candidates for life-changing decisions ranging from  ...  We use a GPT-2 language model (Radford et al., 2019) to do unsupervised language modelling on our training corpus to learn the grammar and structure of normal essays.  ... 
arXiv:2109.11728v3 fatcat:ylkw4ryvpvblxfjwdabctkdyra

How much intelligence is there in artificial intelligence? A 2020 update

Han L.J. van der Maas, Lukas Snoek, Claire E. Stevenson
2021 Intelligence  
We follow with a description of the main techniques these AI breakthroughs were based upon, such as deep learning and reinforcement learning; two techniques that have deep roots in psychology.  ...  We start with a short overview of modern AI and showcase some of the AI breakthroughs in the four decades since Schank's paper.  ...  Extreme examples are the winning network from the ImageNet competition in 2015, which featured over 150 hidden layers (He, Zhang, Ren, & Sun, 2016) and the aforementioned GPT-3 language model featuring  ... 
doi:10.1016/j.intell.2021.101548 fatcat:kiuz5begebgv5enceq7vw7lkte

TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, Dongwon Lee
2021 Findings of the Association for Computational Linguistics: EMNLP 2021   unpublished
We used GROVER-Large discriminator for GROVER detector, the weights of roberta-large fine-tuned on GPT-2 XL outputs for GPT-2 de- tector, and GPT-2 117M model for GLTR.  ...  fake online reviews using neural language models and their human-and machine-based detection.  ... 
doi:10.18653/v1/2021.findings-emnlp.172 fatcat:idnwldoglfdtpgq7kvbljagrry

NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media [article]

Grace Luo, Trevor Darrell, Anna Rohrbach
2021 arXiv   pre-print
We benchmark several state-of-the-art multimodal models on our dataset and analyze their performance across different pretraining domains and visual backbones.  ...  of mismatch between text and image in news that are able to mislead humans.  ...  This work was supported in part by DoD including DARPA's XAI, LwLL, and/or SemaFor programs, as well as BAIR's industrial alliance programs.  ... 
arXiv:2104.05893v2 fatcat:e4fax5rnafdg5hg2p24rjhtwxy

Artificial Intelligence in Drug Discovery: Applications and Techniques [article]

Jianyuan Deng, Zhibo Yang, Iwao Ojima, Dimitris Samaras, Fusheng Wang
2021 arXiv   pre-print
Furthermore, to summarize the progress of AI in drug discovery, we present the relevant AI techniques including model architectures and learning paradigms in the papers surveyed.  ...  Various AI techniques have been used in a wide range of applications, such as virtual screening and drug design.  ...  models, such as GPT, 208 BERT, 209 GPT-2, 210 RoBERTa, 211 and GPT-3, 212 and even in advanced computer vision models, such as DETR 213 and Vision Transformer. 214 Unlike RNNs, transformers  ... 
arXiv:2106.05386v4 fatcat:w2at5y5jyffrxiejsupmwiimhq

Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive [article]

Lirong Wu, Haitao Lin, Zhangyang Gao, Cheng Tan, Stan.Z.Li
2021 arXiv   pre-print
In this survey, we extend the concept of SSL, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of existing SSL techniques  ...  Deep learning on graphs has recently achieved remarkable success on a variety of tasks, while such success relies heavily on the massive and carefully labeled data.  ...  Fig. 3. A comparison of the feature-based, structue-based, sampling-based, and adaptive augmentation.  ... 
arXiv:2105.07342v4 fatcat:iak3xwlx5nci3mlzerxhcylojm
« Previous Showing results 1 — 15 out of 26 results