Sparse coding with memristor networks

Patrick M. Sheridan, Fuxi Cai, Chao Du, Wen Ma, Zhengya Zhang, Wei D. Lu
2017 Nature Nanotechnology  
Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms
more » ... a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary. M emristors are two-terminal devices whose resistance values depend on an internal state variable and can be modulated by the history of external stimulation 1-4 . Unlike conventional charge-based electronic devices, a memristor's state is determined by the internal ion (either cation or anion) configuration, and the redistribution of oxygen ions or metal cations inside the device modulates the local resistivity and overall device resistance 2-4 . Memristors have been extensively studied for both digital memory and analog logic circuit applications 3-7 . At the device level, memristors have been shown to be able to emulate synaptic functions by storing the analog synaptic weights and implementing synaptic learning rules 8-12 . When constructed into a crossbar form, memristor networks offer the desired density and connectivity that are required for hardware implementation of neuromorphic computing systems [13] [14] [15] . Recently, memristor arrays and phase-change memory devices have been used as artificial neural networks to perform pattern classification tasks [16] [17] [18] . Other studies have shown memristors can be used in recurrent artificial neural networks for applications such as analog-to-digital convertors 19 . Memristor-based architectures have also been proposed and analysed for tasks such as sparse coding and dictionary learning 20,21 . The ability to sparsely encode data is believed to be a key mechanism by which biological neural systems can efficiently process large amounts of complex sensory data 22-24 and can enable the implementation of efficient bio-inspired neuromorphic systems for data representation and analysis [25] [26] [27] [28] . In this Article, we experimentally demonstrate the implementation of a sparse coding algorithm in a memristor crossbar, and show that this network can be used to perform applications such as natural image analysis using learned dictionaries.
doi:10.1038/nnano.2017.83 pmid:28530717 fatcat:yh4wz7empre5nckqjdjm554uxm