Fast Sketch Segmentation and Labeling With Deep Learning

Lei Li, Hongbo Fu, Chiew-Lan Tai
<span title="2019-03-01">2019</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="" style="color: black;">IEEE Computer Graphics and Applications</a> </i> &nbsp;
We present a simple and efficient method based on deep learning to automatically decompose sketched objects into semantically valid parts. We train a deep neural network to transfer existing segmentations and labelings from 3D models to freehand sketches without requiring numerous well-annotated sketches as training data. The network takes the binary image of a sketched object as input and produces a corresponding segmentation map with per-pixel labelings as output. A subsequent post-process
more &raquo; ... cedure with multi-label graph cuts further refines the segmentation and labeling result. We validate our proposed method on two sketch datasets. Experiments show that our method outperforms the state-of-the-art method in terms of segmentation and labeling accuracy and is significantly faster, enabling further integration in interactive drawing systems. We demonstrate the efficiency of our method in a sketch-based modeling application that automatically transforms input sketches into 3D models by part assembly. Freehand sketching is frequently adopted as an efficient means of visual communication. Nowadays, the wide adoption of touch devices, together with the development of well-designed drawing software (e.g., Autodesk SketchBook), notably gives rise to easy creation of digital sketches without pen and paper. Unlike photos, which are faithful captures of the real world from cameras, sketches are artistic depictions from humans. Due to various levels of abstraction and distortion existing in sketches, the computer is still far from being able to robustly interpret their underlying semantics conveyed by humans. Existing studies on sketch analysis, such as sketch classification or sketch-based retrieval, have mainly focused on interpreting an input sketch globally, lacking further understanding of its constituent parts. Sketch segmentation is a step towards finer-level sketch analysis. 1-3 Its goal is to decompose an input sketch into several semantically meaningful components, to which
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1109/mcg.2018.2884192</a> <a target="_blank" rel="external noopener" href="">fatcat:aqjs5nukyneblp7o2lnt45id2u</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>