Automatic Discoveries of Physical and Semantic Concepts via Association Priors of Neuron Groups [article]

Shuai Li, Kui Jia, Xiaogang Wang
2017 arXiv   pre-print
The recent successful deep neural networks are largely trained in a supervised manner. It associates complex patterns of input samples with neurons in the last layer, which form representations of concepts. In spite of their successes, the properties of complex patterns associated a learned concept remain elusive. In this work, by analyzing how neurons are associated with concepts in supervised networks, we hypothesize that with proper priors to regulate learning, neural networks can
more » ... ly associate neurons in the intermediate layers with concepts that are aligned with real world concepts, when trained only with labels that associate concepts with top level neurons, which is a plausible way for unsupervised learning. We develop a prior to verify the hypothesis and experimentally find the proposed prior help neural networks automatically learn both basic physical concepts at the lower layers, e.g., rotation of filters, and highly semantic concepts at the higher layers, e.g., fine-grained categories of an entry-level category.
arXiv:1612.09438v2 fatcat:eogrgrxogfa4nlgyreu422pa7u