Adaptive Resource Management for End-to-End Network Slicing in 5G

Abstract

In recent years, data traffic over mobile networks has experienced an exponential rise due to the advances in the internet of things (IoTs) and smart devices and the popularization of mobile services, such as augmented reality, face recognition, and high-definition multimedia streaming. Moreover, emerging edge applications with strong latency requirements, such as cyber-physical systems (CPS) and autonomous driving applications, require new communication networks that can provide near-zero latency. To meet very different service-level agreements (SLAs) of the heterogeneous services, the 5G network needs a flexible and efficient network resource management architecture. 5G network slicing is a network architecture that enables the multiplexing of virtualized and independent logical networks on the physical end-to-end network infrastructures from the edge to the core. Multiple network slices can meet the requirements of various services through dedicated networks, including their own resource capacity and resource control policies customized for on-demand service applications. Traditional static resource allocation methods based on fixed network shares result in resource under-utilization due to continuous fluctuations in traffic demand, leading to high operating costs and severe resource waste. However, dynamically managing resources to reduce resource waste is a very challenging task because the end-to-end 5G network consists of various types of resources, as well as the state of the resources changes over time and is difficult to predict. Therefore, many researchers have been interested in resource management in end-to-end network slicing to obtain better solutions under dynamic environments. Meanwhile, reinforcement learning (RL)-based methods are drawing keen attention as a way to overcome the challenges that can be encountered in a dynamic environment by learning from historical data to anticipate upcoming future information and adaptively make good decisions. In this dissertation, I propose adaptive resource management techniques of network slicing, including a dynamic buffer management algorithm for QoS satisfaction, a reinforcement learning-based resource management algorithm to reduce operating costs, and multi-agent reinforcement learning-based resource management algorithm in more complex end-to-end network slicing environments. In the first part of this dissertation, I focus on managing buffer resources of network slices with different QoS requirements. The impact of buffer size changes on QoS performance depends on the traffic characteristics within the network slice. Therefore, I propose a buffer sizing algorithm based on the QoS sensitivity of buffer allocation. The experimental results using the Mininet emulator indicate that the proposed buffer sizing method achieves buffer allocation that improves QoS satisfaction. In the second part of this dissertation, I developed a dynamic resource management mechanism based on variations of the traffic mix using a Q-learning algorithm. First, I define a business model of the resource allocation process between a slice provider and multiple slices tenants. Second, I propose taking a ratio of the number of inelastic flows to the total number of flows in each network slice as an MDP state parameter to characterize different QoS satisfaction properties of each slice to the change of slice resource allocation. Third, I propose a Q-learning-based dynamic resource management strategy to maximize tenant's profit while satisfying the QoS of end-users in each slice from each tenant's point of view. The numerical simulation results demonstrate that the proposed algorithm can significantly increase the profit of tenants compared to existing fixed resource allocation methods while satisfying the QoS requirements of end-users. In the third part of this dissertation, I develop a RL-based dynamic resource allocation framework for complex end-to-end network slicing with heterogeneous requirements in multi-layer MEC environments. I first design a hierarchical MEC architecture and formulate a resource allocation problem for the end-to-end network slicing as an optimization problem using the Markov Decision Process (MDP). Using proximal policy optimization (PPO), I develop independently-collaborative and jointly-collaborative dynamic resource allocation algorithms to maximize resource efficiency while satisfying the QoS of slices. Experimental results show that the proposed algorithms can recognize the characteristics of slice requests and coming resource demands and efficiently allocate resources with a high QoS satisfaction rate.