Visual Stimulus Images Classification and Reconstruction using Brain Signals

Abstract

We always have wondered about how can we understand people’s thought and intent. Especially, rich studies about reconstructing images what humans see using generative adversarial network(GAN) conjugated with brain signals. In this work, we considered classification of the images the person sees and then their reconstruction. Recently, deep learning techniques have been applied to classify brain signals in various experiments, such as motor imagery and steady state visual evoked potential. First, we proposed a channel attention network and investigated the way the deep learning network may determine which channels contain more important information that represents brainwaves’ characteristics and the way it may visualize that information. We found that our proposed deep learning architecture outperforms basic approaches to classifying categorized images from visual evoked magnetoencephalographic (MEG) brain signals. Second, we proposed brain-signal-to-image model for reconstructing exact given images based on deep convolutional GAN. Additionally, generated images were evaluated by pre-trained convolutional neural network(CNN) and found how to efficiently compress the brain signals.