Feature Imitating Networks [article]

Sari Saba-Sadiya, Tuka Alhanai, Mohammad M Ghassemi
2021 arXiv   pre-print
In this paper, we introduce a novel approach to neural learning: the Feature-Imitating-Network (FIN). A FIN is a neural network with weights that are initialized to reliably approximate one or more closed-form statistical features, such as Shannon's entropy. In this paper, we demonstrate that FINs (and FIN ensembles) provide best-in-class performance for a variety of downstream signal processing and inference tasks, while using less data and requiring less fine-tuning compared to other networks
more » ... of similar (or even greater) representational power. We conclude that FINs can help bridge the gap between domain experts and machine learning practitioners by enabling researchers to harness insights from feature-engineering to enhance the performance of contemporary representation learning approaches.
arXiv:2110.04831v2 fatcat:65t6d7td4jf7pfgc4kc5ebh34e