Memristor-based neural network hardware with hybrid stochastic neuron for fully in-situ training

Abstract

This paper introduces a hybrid stochastic neuron (HSN) chip designed for fully in-situ training of memristorbased neural networks, addressing limitations of traditional systems that rely on software for backpropagation. By performing backpropagation computations, including gradient calculation and weight updates, directly in hardware, the HSN chip eliminates external dependencies and enhances computational efficiency. Fabricated using 28 nm CMOS technology, the HSN chip integrates seamlessly with a 32 x 32 TiOx-based memristor vector matrix multiplication (VMM). Forward propagation tasks are managed by the VMM, while the HSN chip handles backpropagation processes, ensuring a streamlined and effective training framework. Extensive simulations and experimental evaluations validate the performance and feasibility of the system. Tested on the MNIST and CIFAR-10 datasets, it achieved validation accuracies of 94.13 % and 79.88 %, respectively, with a throughput 1.78 tera-operations per second. By enabling fully hardware-based forward and backward propagation, the proposed architecture offers a scalable and efficient solution for softwareindependent neural network training.