Deep Reinforcement Learning based Smart Joint Control Scheme for On/Off Pump Systems in Wastewater Treatment Plants

Abstract

In this paper, we propose a smart joint control scheme for reducing the amount and cost of energy consumption of pump systems in wastewater treatment plants (WWTP), where the pumps are operated in a binary mode by on/off signals. As the global energy consumption increases, efficient operation in energy-intensive facilities has also become important. The WWTP is one of these large power consuming facilities, and this study investigates a novel control scheme for energy efficient coordination of the pumping station which accounts for a significant portion of the energy consumption of WWTP. The proposed scheme consists of deep neural networks (DNN) model for forecasting wastewater inflow and deep reinforcement learning (DRL) for controlling on/off of pump system. Proximal policy optimization (PPO) and deep Q-neural network (DQN) are used as the DRL agents, where the forecasted inflows generated by the DNN model are utilized as elements of the state from the environment in the DRL framework. To implement smart control with DRL, the reward function is designed to take into account the energy consumption amount and cost by the customized utilization of the pumps while maintaining the operating rules. In particular, penalties for pump switching, which are essential for applying the control scheme to the actual site, are also considered. The performance of the proposed control scheme is evaluated with the captured operation from a WWTP in Republic of Korea, of which results show a sensible reduction not only in energy consumption but also in energy cost, compared to the conventional control method such as binary integer linear programming (BILP).