LSTM-based Verbal Aggression Detection Using Audio and Recognized Text

Abstract

In this paper, we propose a deep neural network-based aggression detection method that uses audio information and recognized text information together using a recognizer. First, the proposed method uses a long short-term memory (LSTM) model that inputs feature information of audio data. Recognized text information is extracted using BERT (Bidirectional Encoder Representations from Transformers), and the extracted features use LSTM model. After that, it is composed of a deep neural network (DNN) that can merge the outputs of each configured model and detect aggression. In order to evaluate the performance of the proposed aggressiveness detection model, an objective performance evaluation is performed. In objective evaluation, performance evaluation is conducted by measuring the F1-score between the actual and predicted aggression. It was confirmed that the proposed method shows relatively high F1-score in the model using audio and text data compared to the model using only audio data.