Release:

2019, Vol. 5. №3Title:

Energy efficient biomorphic pulse information coding in electronic neurons for the entrance unit of the neuroprocessor
Author:
Alexander D. Pisarev

For citation:
Pisarev A. D. 2019. “Energy efficient biomorphic pulse information coding in electronic neurons for the entrance unit of the neuroprocessor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 5, no 3, pp. 186-212. DOI: 10.21684/2411-7978-2019-5-3-186-212

About the author:

Alexander D. Pisarev, Cand. Sci. (Tech.), Associate Professor, Department of Applied and Technical Physics, Head of Laboratory of Beam-Plasma Technologies, SEC “Nanotechnologies”, University of Tyumen; eLibrary AuthorID, ORCID, Scopus AuthorID, spcb.doc@gmail.comAbstract:

This article studies the implementation of some well-known principles of information work of biological systems in the input unit of the neuroprocessor, including spike coding of information used in models of neural networks of the latest generation.

The development of modern neural network IT gives rise to a number of urgent tasks at the junction of several scientific disciplines. One of them is to create a hardware platform — a neuroprocessor for energy-efficient operation of neural networks. Recently, the development of nanotechnology of the main units of the neuroprocessor relies on combined memristor super-large logical and storage matrices. The matrix topology is built on the principle of maximum integration of programmable links between nodes. This article describes a method for implementing biomorphic neural functionality based on programmable links of a highly integrated 3D logic matrix.

This paper focuses on the problem of achieving energy efficiency of the hardware used to model neural networks. The main part analyzes the known facts of the principles of information transfer and processing in biological systems from the point of view of their implementation in the input unit of the neuroprocessor. The author deals with the scheme of an electronic neuron implemented based on elements of a 3D logical matrix. A pulsed method of encoding input information is presented, which most realistically reflects the principle of operation of a sensory biological neural system. The model of an electronic neuron for selecting ranges of technological parameters in a real 3D logic matrix scheme is analyzed. The implementation of disjunctively normal forms is shown, using the logic function in the input unit of a neuroprocessor as an example. The results of modeling fragments of electric circuits with memristors of a 3D logical matrix in programming mode are presented.

The author concludes that biomorphic pulse coding of standard digital signals allows achieving a high degree of energy efficiency of the logic elements of the neuroprocessor by reducing the number of valve operations. Energy efficiency makes it possible to overcome the thermal limitation of the scalable technology of three-dimensional layout of elements in memristor crossbars.

Keywords:

References:

- Barsky A. B. 2004. Neural Networks: Recognition, Management, Decision Making. Moscow: Finance and Statistics. [In Russian]
- Bloom F., Leiserson Α., Hofstedter L. 1988. Brain, Mind, and Behavior. Translated from English. Moscow: Mir. [In Russian]
- Bobylev A. N., Udovichenko S. Yu., Busygin A. N., Ebrahim A. H. 2019. “Increase of switching range of resistive memristor for realization of a greater number of synaptic states in a neuroprocessor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 5, no 2, pp. 124-136. DOI: 10.21684/2411-7978-2019-5-2-124-136 [In Russian]
- Backus J. 1993. “Is programming free from von Neumann style? Functional style and associated program algebra”. In: Lectures by Turing Prize Laureates. Moscow: Mir. [In Russian]
- Dubrovsky D. I. (ed.). 2013. Global Future 2045. Converged Technologies (NBICS) and Transhumanist Evolution. Moscow: IBA. [In Russian]
- Gusev A. V. 2017. “Prospects for neural networks and deep machine learning in creating solutions for healthcare”. Information Technologies for the Physician, no 3, pp. 92-105. [In Russian]
- Kalichkin V. K., Pavlova A. I. 2014. “The use of a neural expert system for the classification of erosive lands”. Siberian Herald of Agricultural Science, no 6, pp. 5-11. [In Russian]
- Pisarev A. D., Mayevsky O. V., Busygin A. N., Udovichenko S. Yu. 2018. “A multilayer logic matrix based on a memristor switching cell”. Invention patent No 2017122704. Applied 27 June 2017. Issued 10 April 2018. [In Russian]
- Pisarev A. D., Busygin A. N., Bobylev A. N., Udovichenko S. Yu. 2017. “Combined memristor-diode crossbar as a memory storage base”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 3, no 4, pp. 142-149. DOI: 10.21684/2411-7978-2017-3-4-142-149 [In Russian]
- Pisarev A. D. 2019. “Implementation of discrete cosinus transformation in the input block of the memristor neural processor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 5, no 1, pp. 147-161. DOI: 10.21684/2411-7978-2019-5-1-147-161 [In Russian]
- Pisarev A. D. 2018. “SPICE-Modeling of the Processes of Associative Self Learning and Unconditional Discrimination in the Logic Unit of a Neuroprocessor”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 4, no 3, pp. 132-145. DOI: 10.21684/2411-7978-2018-4-3-132-145 [In Russian]
- Trofimova E. A., Mazurov V. D., Gilev D. V. 2017. Neural Networks in Applied Economics. Edited by E. A. Trofimova. Commisioned by the RF Ministry of Educationa and Science. Yekaterinburg: Ural Federal University. [In Russian]
- Udovichenko S. Yu., Pisarev A. D., Busygin A. N., Mayevsky O. V. 2018. “Neuroprocessor based on a combined memristor-diode crossbar”. Nanoindustry, no 5 (84), pp. 344-355. DOI: 10.22184/1993-8578.2018.84.5.344.355 [In Russian]
- Udovichenko S. Yu., Pisarev A. D., Busygin A. N., Mayevsky O. V. 2017. “3D CMOS-memristor nanotechnology for creating a logical and memory matrix of a neuroprocessor”. Nanoindustry, no 5 (76), pp. 26-34. DOI: 10.22184/1993-8578.2017.76.5.26.34 [In Russian]
- Biolek D., Di Ventra M., Pershin Y. V. 2013. “Reliable SPICE simulations of memristors, memcapacitors and meminductors”. Radioengineering, vol. 22, no 4, pp. 945-968.
- Bobylev A. N., Busygin A. N., Pisarev A. D., Udovichenko S. Yu., Filippov V. A. 2017. “Neuromorphic coprocessor prototype based on mixed metal oxide memristors”. International Journal of Nanotechnology, vol. 14, no 7/8, pp. 698-704. DOI: 10.1504/IJNT.2017.083444
- Bobylev A. N., Udovichenko S. Yu. 2016. “The electrical properties of memristor devices TiN/TixAl1−xOy/TiN produced by magnetron sputtering”. Russian Microelectronics, vol. 45, no 6, pp. 396-401. DOI: 10.1134/S1063739716060020
- Chen Y.-H., Krishna T., Emer J. S., Sze V. 2017. “Eyeriss: an energy-efficient reconfigurable accelerator for deep convolutional neural networks”. IEEE Journal of Solid-State Circuits, vol. 52, no 1, pp. 127-138. DOI: 10.1109/JSSC.2016.2616357
- Filippov V. A., Bobylev A. N., Busygin A. N., Pisarev A. D., Udovichenko S. Yu. 2019. “A biomorphic neuron model and principles of designing a neural network with memristor synapses for a biomorphic neuroprocessor”. Neural Computing and Applications. Published online. DOI: 10.1007/s00521-019-04383-7
- Gardner B, Grüning A. 2016. “Supervised learning in spiking neural networks for precise temporal encoding”. PLoS ONE, vol. 11, no 8, art. e0161335, pp. 1-28. DOI: 10.1371/journal.pone.0161335
- Gollisch T., Meister M. 2008. “Rapid neural coding in the retina with relative spike latencies”. Science, vol. 319, no 5866, pp. 1108-1111. DOI: 10.1126/science.1149639
- Gütig R. 2014. “To spike, or when to spike?”. Current Opinion in Neurobiology, vol. 25, pp. 134-139. DOI: 10.1016/j.conb.2014.01.004
- Hadiyawarman, Budiman F., Hernowo D. G. O., Pandey R. R., Tanaka H. 2018. “Recentprogress on fabrication of memristor and transistor-based neuromorphic devices for high signal processing speed with low power consumption”. Japanese Journal of Applied Physics, vol. 52, no 3S2, art. 03EA06. DOI: 10.7567/JJAP.57.03EA06
- Hodgkin A. L., Huxley A. F. 1952. “A quantitative description of membrane current and its application to conduction and excitation in nerve”. The Journal of Physiology, vol. 117, no 4, pp. 500-544. DOI: 10.1113/jphysiol.1952.sp004764
- Johansson R. S, Birznieks I. 2004. “First spikes in ensembles of human tactile afferents code complex spatial fingertip events”. Nature Neuroscience, vol. 7, no 2, pp. 170-177. DOI: 10.1038/nn1177
- Kasinski A., Ponulak F. 2006. “Comparison of supervised learning methods for spike time coding in spiking neural networks”. International Journal of Applied Mathematics and Computer Science, vol. 16, no 1, pp. 101-113.
- Larkum M. E., Zhu J. J., Sakmann B. 2001. “Dendritic mechanisms underlying the coupling of the dendritic with the axonal action potential initiation zone of adult rat layer 5 pyramidal neurons”. The Journal of Physiology, vol. 533, no 2, pp. 447-466. DOI: 10.1111/j.1469-7793.2001.0447a.x
- Mainen Z. F, Sejnowski T. J. 1995. “Reliability of spike timing in neocortical neurons”. Science, vol. 268, no 5216, pp. 1503-1506. DOI: 10.1126/science.7770778
- Mohemmed A., Schliebs S., Matsuda S., Kasabov N. 2012. “SPAN: spike pattern association neuron for learning spatio-temporal spike patterns”. International Journal of Neural Systems, vol. 22, no 04, art. 1250012. DOI: 10.1142/S0129065712500128
- Pavlova A. I., Morozov D. A. 2017. “The use of neural networks in forecasting the load of the call-center”. International journal of advanced studies, vol. 7, no 2-2, pp. 108-112. DOI: 10.12731/2227-930X-2017-2-2-108-112
- Peng C.-S., Chang W.-Y., Lee Y.-H., Lin M.-H., Chen F., Tsai M.-J. 2012. “Improvement of resistive switching stability of HfO2 films with al doping by atomic layer deposition”. Electrochemical and Solid-State Letters, vol. 15, no 4, pp. H88-H90. DOI: 10.1149/2.011204esl
- Pisarev A. D., Busygin A. N., Udovichenko S. Yu., Maevsky O. V. 2018. “3D memory matrix based on a composite memristor-diode crossbar for a neuromorphic processor”. Microelectronic Engineering, vol. 198, pp. 1-7. DOI: 10.1016/j.mee.2018.06.008
- Reich D. S, Victor J. D, Knight B. W, Ozaki T., Kaplan E. 1997. “Response variability and timing precision of neuronal spike trains in vivo”. Journal of Neurophysiology, vol. 77, no 5, pp. 2836-2841. DOI: 10.1152/jn.1997.77.5.2836
- Takeuchi T., Duszkiewicz A. J., Morris R. G. M. 2014. “The synaptic plasticity and memory hypothesis: encoding, storage and persistence”. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, vol. 369, no 1633. DOI: 10.1098/rstb.2013.0288
- Uzzell V. J., Chichilnisky E. J. 2004. “Precision of spike trains in primate retinal ganglion cells”. Journal of Neurophysiology, vol. 92, no 2, pp. 780-789. DOI: 10.1152/jn.01171.2003
- Van Rullen R., Guyonneau R., Thorpe S. J. 2005. “Spike times make sense”. Trends in Neurosciences, vol. 28, no 1, pp. 1-4. DOI: 10.1016/j.tins.2004.10.010
- Yu Q., Tang H., Tan K. C, Li H. 2013. “Precise-spike-driven synaptic plasticity: Learning hetero-association of spatiotemporal spike patterns”. PLoS ONE, vol. 8, no 11, art. e78318. DOI: 10.1371/journal.pone.0078318