Human Object Sketches: Datasets, Descriptors, Computational Recognition and 3d Shape Retrieval [article]

Mathias Eitz, Marc Alexa, Technische Universität Berlin, Technische Universität Berlin
2012
Since prehistoric times, humans have used sketching to depict our visual world. Even today, sketching is possibly the only rendering technique readily available to all humans. To understand how humans sketch objects, we perform two experiments. In the first experiment, we analyze the distribution of non-expert sketches of everyday objects such as 'teapot' or 'car'. We ask participants to sketch objects of a given category and gather 20,000 unique sketches evenly distributed over 250 object
more » ... ver 250 object categories. The second experiment targets 3d shape retrieval, and we gather 1,814 sketches that are related to the categories in an existing dataset of 3d shapes. The sketches in both datasets turn out to be generally quite abstract with large local and global deviations from the original shape. Based on the first sketch dataset, we perform a perceptual study and find that humans can correctly identify the object category of a sketch 73% of the time. We develop a targeted feature transform for sketches that is based on a bag-of-features approach, yields a compact representation and comes with suitable invariance properties. Using this representation, we develop the first computational recognition method for classifying human object sketches. We compare human performance against the computational model for which we use multi-class support vector machines, trained on the sketch dataset, to classify sketches. The resulting recognition method is able to identify unknown sketches with 56% accuracy (chance is 0.4%). Using the computational model, we demonstrate an interactive sketch recognition system. Based on the second dataset, we develop a system for 3d object retrieval using sketched feature lines as input. The system employs a view-based approach, matching the input against computer generated line drawings of the objects, using the bag-of-features representation developed earlier. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting compari [...]
doi:10.14279/depositonce-3451 fatcat:cooi6o7w3jf4zfve5eqilw5bhe