Generalized Eigenvalue Beamforming Using Covariance Matrices Estimated by Complex Multi-Task U-Net

Abstract

Acoustic beamforming is a signal processing technique that focuses on sound sources using a multi-channel microphone, and is effective in suppressing background noise, interference and reverberation. However, design parameters for the direction of arrival (DoA) based beamforming systems have disadvantages that must be used in known configurations of microphone arrays. As an alternative, the generalized eigenvalue beamforming has been used, requiring only spatial covariance matrices of speech and noise without spatial information. Recently, many studies have been conducted to separate speech and noise by estimating masks using a deep neural network. However, since only real-valued mask was investigated in these studies, spatial covariance matrix estimation errors may occur.

In this dissertation, the GEV beamforming using covariance matrices estimated by complex multi-task U-Net (CMTU-Net) is proposed. The CMTU-Net is a phase-aware speech separation technique, which is an extension of MTU-Net, the magnitude separation technique. In addition, we shows an attempt to reduce error of the parameter estimation using convolutional neural network (CNN).

In performance evaluation, the proposed methods are evaluated using two datasets and compared to conventional mask and MTU-Net based GEV beamforming. As a result, in simulated environments, the CMTU-Net based GEV beamforming performed better than the conventional one in the scale-invariant source-to-distortion ratio (SI-SDR), perceptual of speech quality (PESQ), cepstral distances (CD), and frequency-weighted segmental signal-to-noise ratio (fwSegSNR) on average.