Numerical simulation and experimental study of a hardware pulse neural network with memristor synapses

About the authors:

Alexander N. Busygin, Cand. Sci. (Phys.-Math.), Associate Professor, Department of Applied and Technical Physics, School of Natural Sciences, University of Tyumen, Tyumen, Russia; Senior Researcher, Memristive Materials Laboratory, Center for Nature-Inspired Engineering, University of Tyumen, Tyumen, Russia; a.n.busygin@utmn.ru, https://orcid.org/0000-0002-3439-8067

Andrey N. Bobylev, Head of the Laboratory of Electronic and Probe Microscopy. REC “Nanotechnology”, University of Tyumen; eLibrary AuthorID, ScopusID, andreaubobylev@gmail.com; ORCID: 0000-0001-5488-8736

Alexey A. Gubin, Postgraduate Student, Department of Applied and Technical Physics, Engineer, REC “Nanotechnology”, University of Tyumen; a.a.gubin@utmn.ru

Alexander D. Pisarev, Cand. Sci. (Tech.), Associate Professor, Department of Applied and Technical Physics, School of Natural Sciences, University of Tyumen, Tyumen, Russia; Senior Researcher, Memristive Materials Laboratory, Center for Nature-Inspired Engineering, University of Tyumen, Tyumen, Russia; spcb.doc@utmn.ru, https://orcid.org/0000-0002-5602-3880

Sergey Yu. Udovichenko, Dr. Sci. (Phys.-Math.), Professor, Department of Applied and Tech­nical Physics, School of Natural Sciences, University of Tyumen, Tyumen, Russia; Scientific Director of the Memristive Materials Laboratory, Center for Nature-Inspired Engineering, University of Tyumen, Tyumen, Russia; udotgu@mail.ru, https://orcid.org/0000-0003-3583-7081

Abstract:

This article presents the results of a numerical simulation and an experimental study of the electrical circuit of a hardware spiking perceptron based on a memristor-diode crossbar. That has required developing and manufacturing a measuring bench, the electrical circuit of which consists of a hardware perceptron circuit and an input peripheral electrical circuit to implement the activation functions of the neurons and ensure the operation of the memory matrix in a spiking mode. The authors have performed a study of the operation of the hardware spiking neural network with memristor synapses in the form of a memory matrix in the mode of a single-layer perceptron synapses. The perceptron can be considered as the first layer of a biomorphic neural network that performs primary processing of incoming information in a biomorphic neuroprocessor. The obtained experimental and simulation learning curves show the expected increase in the proportion of correct classifications with an increase in the number of training epochs. The authors demonstrate generating a new association during retraining caused by the presence of new input information. Comparison of the results of modeling and an experiment on training a small neural network with a small crossbar will allow creating adequate models of hardware neural networks with a large memristor-diode crossbar. The arrival of new unknown information at the input of the hardware spiking neural network can be related with the generation of new associations in the biomorphic neuroprocessor. With further improvement of the neural network, this information will be comprehended and, therefore, will allow the transition from weak to strong artificial intelligence.

References:

1. Pisarev A. D., Busygin A. N., Bobylev A. N., Gubin A. A., Udovichenko S. Yu. 2020. “The study of the electrophysical properties of a composite memristor-diode crossbar as a basis of the neuroprocessor hardware implementation”. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, vol. 6, no. 3 (23), pp. 93-109. DOI: 10.21684/2411-7978-2020-6-3-93-109 [In Russian]

2. Biolek D., Di Ventra M., Pershin Y. V. 2013. “Reliable spice simulations of memristors, memcapacitors and meminductors”. Radioengineering, vol. 22, no. 4, pp. 945-968. https://arxiv.org/abs/1307.2717.

3. Cai F., Kumar S., Van Vaerenbergh T. et al. 2020. “Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks”. Nature Electronics, vol.3, pp. 409-418. DOI: 10.1038/s41928-020-0436-6

4. Demin V. A. Nekhaev D. V., Surazhevsky I. A. et al. 2021. “Necessary conditions for STDP-based pattern recognition learning in a memristive spiking neural network”. Neural Networks, vol. 134, pp. 64-75. DOI: 10.1016/j.neunet.2020.11.005

5. Filippov V. A., Bobylev A. N., Busygin A. N., Pisarev A. D., Udovichenko S. Yu. 2020. “A biomorphic neuron model and principles of designing a neural network with memristor synapses for a biomorphic neuroprocessor”. Neural Computing and Applications, vol. 32, pp. 2471-2485. DOI:10.1007/s00521-019-04383-7

6. Khacef L., Abderrahmane N., Miramond B. 2018. “Confronting machine-learning with neuroscience for neuromorphic architectures design”. Proceedings of the International Joint Conference on Neural Networks (IJCNN). DOI: 10.1109/ijcnn.2018.8489241

7. Lobo J. L., Ser J. D., Bifet A., Kasabov N. 2020 “Spiking Neural Networks and online learning: an overview and perspectives”. Neural Networks, vol. 121, pp. 88-100. DOI: 10.1016/j.neunet.2019.09.004

8. Miao H., Graves C. E., Li C. et al. 2018. “Memristor-based analog computation and neural network classification with a dot product engine”. Advanced Materials, vol. 30, no. 9, art. 1705914. DOI: 10.1002/adma.201705914

9. Minnekhanov A. A., Emelyanov A. V., Lapkin D. A. et al. 2019. “Parylene based memristive devices with multilevel resistive switching for neuromorphic applications”. Scientific Reports, vol. 9, art. 10800. DOI: 10.1038/s41598-019-47263-9

10. Pershin Y. V., Di Ventra M. 2010. “Experimental demonstration of associative memory with memristive neural networks”. Neural Networks, vol. 23, no 7, pp.881-886. DOI: 10.1016/j.neunet.2010.05.001

11. Pisarev A., Busygin A., Bobylev A., Gubin A., Udovichenko S. 2021. “Fabrication technology and electrophysical properties of a composite memristor-diode crossbar used as a basis for hardware implementation of a biomorphic neuroprocessor”. Microelectronic Engineering, vol. 236, art. 111471. DOI: 10.1016/j.mee.2020.111471 [In Russian]

12. Pisarev A. D., Busygin A. N., Udovichenko S. Yu., Maevsky O. V. 2020. “A biomorphic neuroprocessor based on a composite memristor-diode crossbar”. Microelectronics Journal, vol. 102, art. 104827. DOI: 10.1016/j.mejo.2020.104827

13. Prezioso M., Merrikh-Bayat F., Hoskins B. D., Adam G. C., Likharev K. K., Strukov D. B. 2015. “Training and operation of an integrated neuromorphic network based on metal-oxide memristors”. Nature, vol. 521, pp. 61-64. DOI: 10.1038/nature14441

14. Wang Z., Rao M., Han J.-W. et al. 2018. “Capacitive neural network with neuro-transistors”. Nature Communications, vol. 9, art. 3208. DOI: 10.1038/s41467-018-05677-5

15. Wang Z., Wang X. 2018. “A novel memristor-based circuit implementation of full-function pavlov associative memory accorded with biological feature”. IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 7, pp. 2210-2220. DOI: 10.1109/TCSI.2017.2780826

16. Yang L., Zeng Z., Huang Y., Wen S. 2018. “Memristor-based circuit implementations of recognition network and recall network with forgetting stages”. IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 4, pp. 1133-1142. DOI: 10.1109/TCDS.2018.2859303.

17. Zhang X., Long K. 2019. “Improved learning experience memristor model and application as neural network synapse”. IEEE Access, vol. 7, pp. 15262-15271. DOI: 10.1109/ACCESS.2019.2894634