Energy efficient biomorphic pulse information coding in electronic neurons for the entrance unit of the neuroprocessor

About the author:

Alexander D. Pisarev, Cand. Sci. (Tech.), Associate Professor, Department of Applied and Technical Physics, School of Natural Sciences, University of Tyumen, Tyumen, Russia; Senior Researcher, Memristive Materials Laboratory, Center for Nature-Inspired Engineering, University of Tyumen, Tyumen, Russia; spcb.doc@utmn.ru, https://orcid.org/0000-0002-5602-3880

Abstract:

This article studies the implementation of some well-known principles of information work of biological systems in the input unit of the neuroprocessor, including spike coding of information used in models of neural networks of the latest generation.

The development of modern neural network IT gives rise to a number of urgent tasks at the junction of several scientific disciplines. One of them is to create a hardware platform — a neuroprocessor for energy-efficient operation of neural networks. Recently, the development of nanotechnology of the main units of the neuroprocessor relies on combined memristor super-large logical and storage matrices. The matrix topology is built on the principle of maximum integration of programmable links between nodes. This article describes a method for implementing biomorphic neural functionality based on programmable links of a highly integrated 3D logic matrix.

This paper focuses on the problem of achieving energy efficiency of the hardware used to model neural networks. The main part analyzes the known facts of the principles of information transfer and processing in biological systems from the point of view of their implementation in the input unit of the neuroprocessor. The author deals with the scheme of an electronic neuron implemented based on elements of a 3D logical matrix. A pulsed method of encoding input information is presented, which most realistically reflects the principle of operation of a sensory biological neural system. The model of an electronic neuron for selecting ranges of technological parameters in a real 3D logic matrix scheme is analyzed. The implementation of disjunctively normal forms is shown, using the logic function in the input unit of a neuroprocessor as an example. The results of modeling fragments of electric circuits with memristors of a 3D logical matrix in programming mode are presented.

The author concludes that biomorphic pulse coding of standard digital signals allows achieving a high degree of energy efficiency of the logic elements of the neuroprocessor by reducing the number of valve operations. Energy efficiency makes it possible to overcome the thermal limitation of the scalable technology of three-dimensional layout of elements in memristor crossbars.

References:

1. Barsky A. B. 2004. Neural Networks: Recognition, Management, Decision Making. Moscow: Finance and Statistics. [In Russian]
2. Bloom F., Leiserson Α., Hofstedter L. 1988. Brain, Mind, and Behavior. Translated from English. Moscow: Mir. [In Russian]
3. Bobylev A. N., Udovichenko S. Yu., Busygin A. N., Ebrahim A. H. 2019. “Increase of switching range of resistive memristor for realization of a greater number of synaptic states in a neuroprocessor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 5, no 2, pp. 124-136. DOI: 10.21684/2411-7978-2019-5-2-124-136 [In Russian]
4. Backus J. 1993. “Is programming free from von Neumann style? Functional style and associated program algebra”. In: Lectures by Turing Prize Laureates. Moscow: Mir. [In Russian]
5. Dubrovsky D. I. (ed.). 2013. Global Future 2045. Converged Technologies (NBICS) and Transhumanist Evolution. Moscow: IBA. [In Russian]
6. Gusev A. V. 2017. “Prospects for neural networks and deep machine learning in creating solutions for healthcare”. Information Technologies for the Physician, no 3, pp. 92-105. [In Russian]
7. Kalichkin V. K., Pavlova A. I. 2014. “The use of a neural expert system for the classification of erosive lands”. Siberian Herald of Agricultural Science, no 6, pp. 5-11. [In Russian]
8. Pisarev A. D., Mayevsky O. V., Busygin A. N., Udovichenko S. Yu. 2018. “A multilayer logic matrix based on a memristor switching cell”. Invention patent No 2017122704. Applied 27 June 2017. Issued 10 April 2018. [In Russian]
9. Pisarev A. D., Busygin A. N., Bobylev A. N., Udovichenko S. Yu. 2017. “Combined memristor-diode crossbar as a memory storage base”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 3, no 4, pp. 142-149. DOI: 10.21684/2411-7978-2017-3-4-142-149 [In Russian]
10. Pisarev A. D. 2019. “Implementation of discrete cosinus transformation in the input block of the memristor neural processor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 5, no 1, pp. 147-161. DOI: 10.21684/2411-7978-2019-5-1-147-161 [In Russian]
11. Pisarev A. D. 2018. “SPICE-Modeling of the Processes of Associative Self Learning and Unconditional Discrimination in the Logic Unit of a Neuroprocessor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 4, no 3, pp. 132-145. DOI: 10.21684/2411-7978-2018-4-3-132-145 [In Russian]
12. Trofimova E. A., Mazurov V. D., Gilev D. V. 2017. Neural Networks in Applied Economics. Edited by E. A. Trofimova. Commisioned by the RF Ministry of Educationa and Science. Yekaterinburg: Ural Federal University. [In Russian]
13. Udovichenko S. Yu., Pisarev A. D., Busygin A. N., Mayevsky O. V. 2018. “Neuroprocessor based on a combined memristor-diode crossbar”. Nanoindustry, no 5 (84), pp. 344-355. DOI: 10.22184/1993-8578.2018.84.5.344.355 [In Russian]
14. Udovichenko S. Yu., Pisarev A. D., Busygin A. N., Mayevsky O. V. 2017. “3D CMOS-memristor nanotechnology for creating a logical and memory matrix of a neuroprocessor”. Nanoindustry, no 5 (76), pp. 26-34. DOI: 10.22184/1993-8578.2017.76.5.26.34 [In Russian]
15. Biolek D., Di Ventra M., Pershin Y. V. 2013. “Reliable SPICE simulations of memristors, memcapacitors and meminductors”. Radioengineering, vol. 22, no 4, pp. 945-968. 
16. Bobylev A. N., Busygin A. N., Pisarev A. D., Udovichenko S. Yu., Filippov V. A. 2017. “Neuromorphic coprocessor prototype based on mixed metal oxide memristors”. International Journal of Nanotechnology, vol. 14, no 7/8, pp. 698-704. DOI: 10.1504/IJNT.2017.083444
17. Bobylev A. N., Udovichenko S. Yu. 2016. “The electrical properties of memristor devices TiN/TixAl1−xOy/TiN produced by magnetron sputtering”. Russian Microelectronics, vol. 45, no 6, pp. 396-401. DOI: 10.1134/S1063739716060020
18. Chen Y.-H., Krishna T., Emer J. S., Sze V. 2017. “Eyeriss: an energy-efficient reconfigurable accelerator for deep convolutional neural networks”. IEEE Journal of Solid-State Circuits, vol. 52, no 1, pp. 127-138. DOI: 10.1109/JSSC.2016.2616357
19. Filippov V. A., Bobylev A. N., Busygin A. N., Pisarev A. D., Udovichenko S. Yu. 2019. “A biomorphic neuron model and principles of designing a neural network with memristor synapses for a biomorphic neuroprocessor”. Neural Computing and Applications. Published online. DOI: 10.1007/s00521-019-04383-7
20. Gardner B, Grüning A. 2016. “Supervised learning in spiking neural networks for precise temporal encoding”. PLoS ONE, vol. 11, no 8, art. e0161335, pp. 1-28. DOI: 10.1371/journal.pone.0161335
21. Gollisch T., Meister M. 2008. “Rapid neural coding in the retina with relative spike latencies”. Science, vol. 319, no 5866, pp. 1108-1111. DOI: 10.1126/science.1149639
22. Gütig R. 2014. “To spike, or when to spike?”. Current Opinion in Neurobiology, vol. 25, pp. 134-139. DOI: 10.1016/j.conb.2014.01.004
23. Hadiyawarman, Budiman F., Hernowo D. G. O., Pandey R. R., Tanaka H. 2018. “Recentprogress on fabrication of memristor and transistor-based neuromorphic devices for high signal processing speed with low power consumption”. Japanese Journal of Applied Physics, vol. 52, no 3S2, art. 03EA06. DOI: 10.7567/JJAP.57.03EA06
24. Hodgkin A. L., Huxley A. F. 1952. “A quantitative description of membrane current and its application to conduction and excitation in nerve”. The Journal of Physiology, vol. 117, no 4, pp. 500-544. DOI: 10.1113/jphysiol.1952.sp004764
25. Johansson R. S, Birznieks I. 2004. “First spikes in ensembles of human tactile afferents code complex spatial fingertip events”. Nature Neuroscience, vol. 7, no 2, pp. 170-177. DOI: 10.1038/nn1177
26. Kasinski A., Ponulak F. 2006. “Comparison of supervised learning methods for spike time coding in spiking neural networks”. International Journal of Applied Mathematics and Computer Science, vol. 16, no 1, pp. 101-113.
27. Larkum M. E., Zhu J. J., Sakmann B. 2001. “Dendritic mechanisms underlying the coupling of the dendritic with the axonal action potential initiation zone of adult rat layer 5 pyramidal neurons”. The Journal of Physiology, vol. 533, no 2, pp. 447-466. DOI: 10.1111/j.1469-7793.2001.0447a.x
28. Mainen Z. F, Sejnowski T. J. 1995. “Reliability of spike timing in neocortical neurons”. Science, vol. 268, no 5216, pp. 1503-1506. DOI: 10.1126/science.7770778
29. Mohemmed A., Schliebs S., Matsuda S., Kasabov N. 2012. “SPAN: spike pattern association neuron for learning spatio-temporal spike patterns”. International Journal of Neural Systems, vol. 22, no 04, art. 1250012. DOI: 10.1142/S0129065712500128
30. Pavlova A. I., Morozov D. A. 2017. “The use of neural networks in forecasting the load of the call-center”. International journal of advanced studies, vol. 7, no 2-2, pp. 108-112. DOI: 10.12731/2227-930X-2017-2-2-108-112
31. Peng C.-S., Chang W.-Y., Lee Y.-H., Lin M.-H., Chen F., Tsai M.-J. 2012. “Improvement of resistive switching stability of HfO2 films with al doping by atomic layer deposition”. Electrochemical and Solid-State Letters, vol. 15, no 4, pp. H88-H90. DOI: 10.1149/2.011204esl
32. Pisarev A. D., Busygin A. N., Udovichenko S. Yu., Maevsky O. V. 2018. “3D memory matrix based on a composite memristor-diode crossbar for a neuromorphic processor”. Microelectronic Engineering, vol. 198, pp. 1-7. DOI: 10.1016/j.mee.2018.06.008
33. Reich D. S, Victor J. D, Knight B. W, Ozaki T., Kaplan E. 1997. “Response variability and timing precision of neuronal spike trains in vivo”. Journal of Neurophysiology, vol. 77, no 5, pp. 2836-2841. DOI: 10.1152/jn.1997.77.5.2836
34. Takeuchi T., Duszkiewicz A. J., Morris R. G. M. 2014. “The synaptic plasticity and memory hypothesis: encoding, storage and persistence”. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, vol. 369, no 1633. DOI: 10.1098/rstb.2013.0288
35. Uzzell V. J., Chichilnisky E. J. 2004. “Precision of spike trains in primate retinal ganglion cells”. Journal of Neurophysiology, vol. 92, no 2, pp. 780-789. DOI: 10.1152/jn.01171.2003
36. Van Rullen R., Guyonneau R., Thorpe S. J. 2005. “Spike times make sense”. Trends in Neurosciences, vol. 28, no 1, pp. 1-4. DOI: 10.1016/j.tins.2004.10.010
37. Yu Q., Tang H., Tan K. C, Li H. 2013. “Precise-spike-driven synaptic plasticity: Learning hetero-association of spatiotemporal spike patterns”. PLoS ONE, vol. 8, no 11, art. e78318. DOI: 10.1371/journal.pone.0078318