A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit <a rel="external noopener" href="https://arxiv.org/pdf/2107.06505v1.pdf">the original URL</a>. The file type is <code>application/pdf</code>.
Few-shot Neural Human Performance Rendering from Sparse RGBD Videos
[article]
<span title="2021-07-14">2021</span>
<i >
arXiv
</i>
<span class="release-stage" >pre-print</span>
Preserved Fulltext
Other Versions
<span title="2021-09-28">2021</span>
<span class="release-stage">accepted</span>
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.06505v2">arXiv:2107.06505v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/rob562mfxbh4rfzzxrmhgy2vaq">fatcat:rob562mfxbh4rfzzxrmhgy2vaq</a> </span>
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.06505v2">arXiv:2107.06505v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/rob562mfxbh4rfzzxrmhgy2vaq">fatcat:rob562mfxbh4rfzzxrmhgy2vaq</a> </span>
Recent neural rendering approaches for human activities achieve remarkable view synthesis results, but still rely on dense input views or dense training with all the capture frames, leading to deployment difficulty and inefficient training overload. However, existing advances will be ill-posed if the input is both spatially and temporally sparse. To fill this gap, in this paper we propose a few-shot neural human rendering approach (FNHR) from only sparse RGBD inputs, which exploits the temporal
<span class="external-identifiers">
<a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.06505v1">arXiv:2107.06505v1</a>
<a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ilzslk4kvvgrri7svtvjhqwfze">fatcat:ilzslk4kvvgrri7svtvjhqwfze</a>
</span>
more »
... and spatial redundancy to generate photo-realistic free-view output of human activities. Our FNHR is trained only on the key-frames which expand the motion manifold in the input sequences. We introduce a two-branch neural blending to combine the neural point render and classical graphics texturing pipeline, which integrates reliable observations over sparse key-frames. Furthermore, we adopt a patch-based adversarial training process to make use of the local redundancy and avoids over-fitting to the key-frames, which generates fine-detailed rendering results. Extensive experiments demonstrate the effectiveness of our approach to generate high-quality free view-point results for challenging human performances under the sparse setting.
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210720083644/https://arxiv.org/pdf/2107.06505v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext">
<button class="ui simple right pointing dropdown compact black labeled icon button serp-button">
<i class="icon ia-icon"></i>
Web Archive
[PDF]
<div class="menu fulltext-thumbnail">
<img src="https://blobs.fatcat.wiki/thumbnail/pdf/a8/10/a810133f99b8e844c7381cdff1798d0e721891e6.180px.jpg" alt="fulltext thumbnail" loading="lazy">
</div>
</button>
</a>
<a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2107.06505v1" title="arxiv.org access">
<button class="ui compact blue labeled icon button serp-button">
<i class="file alternate outline icon"></i>
arxiv.org
</button>
</a>