Filters








2 Hits in 1.5 sec

Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity [article]

Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, Lav R. Varshney
<span title="2021-01-14">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We use this analysis to design a feedback-based adaptive top-k text decoding algorithm called mirostat that generates text (of any length) with a predetermined value of perplexity, and thereby high-quality  ...  Neural text decoding is important for generating high-quality texts using language models.  ...  Building on this analysis, we developed mirostat, a neural text decoding algorithm that directly controls the perplexity of the generated text over a wide range of text length.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.14966v2">arXiv:2007.14966v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/sug5pzvpsfb43lzxbc4eqdil4e">fatcat:sug5pzvpsfb43lzxbc4eqdil4e</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210119193639/https://arxiv.org/pdf/2007.14966v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/87/8a/878ae18ae23d89400cc28991ad4c2fc9d42e7123.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.14966v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Improving Diversity of Neural Text Generation via Inverse Probability Weighting [article]

Xinran Zhang, Maosong Sun, Jiafeng Liu, Xiaobing Li
<span title="2021-08-26">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The neural text generation suffers from the text degeneration issue such as repetition.  ...  Results show that our algorithm can effectively increase the diversity of generated samples while achieving close resemblance to human text.  ...  decoding method that generates samples with low perplexity actually deviates from human text behavior (see Figure 2 , Holtzman et al., 2020) .  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.07649v3">arXiv:2103.07649v3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/26zaiodd7zahra6an3dq7h63py">fatcat:26zaiodd7zahra6an3dq7h63py</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210831022215/https://arxiv.org/pdf/2103.07649v3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/ed/b6/edb632dfefa1ae68550ce0ec3db7e13ed9e4a226.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.07649v3" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>