Filters








44 Hits in 5.1 sec

TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising [article]

Ziyi Yang, Chenguang Zhu, Robert Gmyr, Michael Zeng, Xuedong Huang, Eric Darve
2020 arXiv   pre-print
Next, we finetune TED on target domains through theme modeling and a denoising autoencoder to enhance the quality of generated summaries.  ...  In order to address these issues, we propose TED, a transformer-based unsupervised abstractive summarization system with pretraining on large-scale data.  ...  In this paper, we present TED, a pretrained unsupervised abstractive summarization model which is finetuned with theme modeling and denoising on in-domain data.  ... 
arXiv:2001.00725v3 fatcat:n4penvk2dven3nzj3s4q4c67z4

TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising

Ziyi Yang, Chenguang Zhu, Robert Gmyr, Michael Zeng, Xuedong Huang, Eric Darve
2020 Findings of the Association for Computational Linguistics: EMNLP 2020   unpublished
Next, we finetune TED on target domains through theme modeling and a denoising autoencoder to enhance the quality of generated summaries.  ...  In order to address these issues, we propose TED, a transformerbased unsupervised abstractive summarization system with pretraining on large-scale data.  ...  In this paper, we present TED, a pretrained unsupervised abstractive summarization model which is finetuned with theme modeling and denoising on in-domain data.  ... 
doi:10.18653/v1/2020.findings-emnlp.168 fatcat:tvivth4ufbh2petouoldjbd3i4

Pretrained Language Models for Text Generation: A Survey [article]

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen
2021 arXiv   pre-print
The resurgence of deep learning has greatly advanced this field by neural generation models, especially the paradigm of pretrained language models (PLMs).  ...  Our survey aims to provide text generation researchers a synthesis and pointer to related research.  ...  For example, Fan et al. [2019] proposed an unsupervised approach to pretraining encoder-decoder model with unpaired speech and transcripts.  ... 
arXiv:2105.10311v2 fatcat:euqkvufzlbhwresrodmxf7wivi

Leveraging Lead Bias for Zero-shot Abstractive News Summarization [article]

Chenguang Zhu, Ziyi Yang, Robert Gmyr, Michael Zeng, Xuedong Huang
2021 arXiv   pre-print
We deploy the model in Microsoft News and provide public APIs as well as a demo website for multi-lingual news summarization.  ...  We propose that this lead bias can be leveraged in our favor in a simple and effective way to pre-train abstractive news summarization models on large-scale unlabeled news corpora: predicting the leading  ...  TED [28] employs theme modeling and a denoising autoencoder to enhance the quality of summaries.  ... 
arXiv:1912.11602v4 fatcat:w6nlhfvvojbmfnijgoymeserzy

StreamHover: Livestream Transcript Summarization and Annotation [article]

Sangwoo Cho and Franck Dernoncourt and Tim Ganter and Trung Bui and Nedim Lipka and Walter Chang and Hailin Jin and Jonathan Brandt and Hassan Foroosh and Fei Liu
2021 arXiv   pre-print
With the explosive growth of livestream broadcasting, there is an urgent need for new summarization technology that enables us to create a preview of streamed content and tap into this wealth of knowledge  ...  We explore a neural extractive summarization model that leverages vector-quantized variational autoencoder to learn latent vector representations of spoken utterances and identify salient utterances from  ...  We use pretrained BERT-BASE as our embedder Embed θ (·). The model has 12 layers, 12 heads per layer and a hidden size (H) of 768.  ... 
arXiv:2109.05160v1 fatcat:fwjq6xr4czd4hinsy5xhsw4mfa

TED-MWE: a bilingual parallel corpus with MWE annotation [chapter]

Johanna Monti, Federico Sangati, Mihael Arcan
Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015  
and creation of a dictionary, n. 20105B3HE8), funded by the Italian Ministry of Education, University and Research (MIUR). http://combinet.humnet. unipi.it.  ...  Acknowledgements The authors would like to thank Maria Simi and Roberta Montefusco for providing the EVALITA14 gold standard set, and the two anonymous reviewers who contributed with their valuable feedback  ...  We evaluate our models on the Evalita 2009 Italian dataset for NERs (Speranza, 2009) summarized in Tab. 1.  ... 
doi:10.4000/books.aaccademia.1514 fatcat:blsozns23nawtjt6rdt2pumjzm

Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics [article]

Daniel Deutsch and Rotem Dror and Dan Roth
2022 arXiv   pre-print
We identify two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice and propose changes to rectify this disconnect.  ...  How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by system-level correlations.  ...  TED: 5093-5100, Florence, Italy. Association for Compu- guistics and the 11th International Joint Conference A Pretrained Unsupervised Summarization Model tational Linguistics.  ... 
arXiv:2204.10216v1 fatcat:cs4nxubtjjfnvdi32sc5lzvoii

PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization [article]

Xiaochen Liu, Yu Bai, Jiawei Li, Yinan Hu, Yang Gao
2022 arXiv   pre-print
The first step in the summarization procedure is to conduct prompt pre-training with self-supervised pseudo-data. This teaches the model basic summarizing capabilities.  ...  To support it, we designed a novel soft prompts architecture coupled with a prompt pre-training plus fine-tuning paradigm that is effective and tunes only extremely light parameters.  ...  [Yang et al., 2020] Ziyi Yang, Chenguang Zhu, Robert Gmyr, Michael Zeng, Xuedong Huang, and Eric Darve. Ted: A pretrained unsupervised summarization model with theme modeling and denoising.  ... 
arXiv:2204.04413v1 fatcat:ljkkxqgoyfg3todcvbrj4dedsi

Pretrained Language Model for Text Generation: A Survey

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen
2021 Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence   unpublished
The resurgence of deep learning has greatly advanced this field by neural generation models, especially the paradigm of pretrained language models (PLMs).  ...  Our survey aims to provide text generation researchers a synthesis and pointer to related research.  ...  [2019] proposed an unsupervised approach to pretraining encoder-decoder model with unpaired speech and transcripts.  ... 
doi:10.24963/ijcai.2021/612 fatcat:xiorzbciffetflqxziks7oan7y

StreamHover: Livestream Transcript Summarization and Annotation

Sangwoo Cho, Franck Dernoncourt, Tim Ganter, Trung Bui, Nedim Lipka, Walter Chang, Hailin Jin, Jonathan Brandt, Hassan Foroosh, Fei Liu
2021 Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing   unpublished
With the explosive growth of livestream broadcasting, there is an urgent need for new summarization technology that enables us to create a preview of streamed content and tap into this wealth of knowledge  ...  We explore a neural extractive summarization model that leverages vector-quantized variational autoencoder to learn latent vector representations of spoken utterances and identify salient utterances from  ...  We use pretrained BERT-BASE as our embedder Embed θ (•). The model has 12 layers, 12 heads per layer and a hidden size (H) of 768.  ... 
doi:10.18653/v1/2021.emnlp-main.520 fatcat:i4rgh24i5vgfjfit6w5h2khlze

Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics

Daniel Deutsch, Rotem Dror, Dan Roth
2022 Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies   unpublished
We identify two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice and propose changes to rectify this disconnect.  ...  How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by systemlevel correlations.  ...  A Pretrained Unsupervised Summarization Model Zeng, Xuedong Huang, and Eric Darve. 2020. TED: Ziyi Yang, Chenguang Zhu, Robert Gmyr, Michael tional Linguistics. pages 1355-1362, Online.  ... 
doi:10.18653/v1/2022.naacl-main.442 fatcat:5s67u3bj55db5kfgzakx5oshga

Insights from Deep Representations for Machine Learning Systems and Human Collaborations

Maithreyi Raghu
2020
We conclude with a discussion on the many rich further directions and open questions for future study.  ...  Over the past several years, we have witnessed fundamental breakthroughs in machine learning, largely driven by rapid advancements of the underlying deep neural network models and algorithms.  ...  , executable computer program structure [375] and summarization, where passages of text are summarized by a neural network [188, 378] .  ... 
doi:10.7298/xvk2-m314 fatcat:f3qjq56xyrdbpognytmc6oizsu

Machine Translation of Spontaneous Speech

Eunah Cho
2016
SPONTANEOUS DATA AND EXPERIMENTAL SETUP The neural network is pretrained layerwise using denoising autoencoders with a 20 million mini batches.  ...  Once the TED data is clustered into 1, 000 classes, we build a 9-gram language model and used it as an additional model.  ...  The baseline is a strong phrase-based system whose performance is boosted using a POS-based language model, a cluster-based language model using MKCLS and a DWL with source context.  ... 
doi:10.5445/ir/1000056203 fatcat:mznlepsb3ndexf2qgjugmc5sh4

Yahi_columbia_0054D_15472.pdf [article]

2019
Simulating drug responses in laboratory test time series with deep generative modeling Alexandre Yahi Drug e ects can be unpredictable and vary widely among patients with environmental, genetic, and clinical  ...  models are more exible with multi-modal inputs and can make sense of large amounts of features without extensive engineering.  ...  and unsupervised tasks that could be used in every paper for evaluation and comparison of models.  ... 
doi:10.7916/d8-nr56-nk24 fatcat:tdbnwan4dnbhja7vty2xu4khgu

ItEM: A Vector Space Model to Bootstrap an Italian Emotive Lexicon [chapter]

Lucia C. Passaro, Laura Pollacci, Alessandro Lenci
Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015  
and creation of a dictionary, n. 20105B3HE8), funded by the Italian Ministry of Education, University and Research (MIUR). http://combinet.humnet. unipi.it.  ...  Acknowledgements The authors would like to thank Maria Simi and Roberta Montefusco for providing the EVALITA14 gold standard set, and the two anonymous reviewers who contributed with their valuable feedback  ...  We evaluate our models on the Evalita 2009 Italian dataset for NERs (Speranza, 2009) summarized in Tab. 1.  ... 
doi:10.4000/books.aaccademia.1530 fatcat:hn5na225mba7xbvlwimzbm4ddm
« Previous Showing results 1 — 15 out of 44 results