Probabilistic Forecasting based Joint Detection and Imputation of Clustered Bad Data in Residential Electricity Loads

Abstract

As the energy consumption increases due to the growth of population and economy, various solution are being sought to preserve the global environment. One of such efforts is energy management system (EMS), a system that reduces energy consumption by managing energy demand and optimizing facility operation, and can be applied to various area such as factories, buildings, communities, and households. In the EMS, energy consumption analysis, prediction, and optimization are performed based on data, thus data reliability is highly necessary to enhance the EMS performance. Among the data-driven imputation method for energy data, auto-regressive integrated moving average (ARIMA), recurrent neural network (RNN), and auto-encoder were used as conventional methods. However, there are several types of anomalies in the energy load metering data such as not-a-number (NaN), zero points, and even the summation of actual values during the NaN series. In order to utilize the energy load data including such defective data for prediction and optimization, anomaly detection and adaptive imputation should be implemented.

In this paper, probabilistic forecasting based detection and adaptive imputation for residential electrical load dataset is proposed. Especially, the summation point in front of the cluster of NaNs is focused because its information can be applied to impute the NaN series. Probabilistic forecasting based detection is performed for points where such values can be appeared, and the result is obtained whether such point is normal or anomaly. After detection, the imputation is applied differently according to the detection result. If it is estimated anomaly, the accumulated value is applied as a constraint to the imputation result of the NaN series range and a better result is extracted. Through the evaluation of applying the proposed method to actual residential electrical load data, it was shown that it can achieve better performance than the conventional method in both detection and imputation.