Enhanced Multi-clustering and Multi-tenancy for ML Workflows over Cloud-native based SmartX AI Cluster

Abstract

The recent increase in demand for cloud services has led to a significant increase in the use of container technologies that make up a computing environment based on lightweight virtual environment. In addition, the number of AI technology support tools being utilized in various areas has been gradually increased and flexible distribution and operation has been made possible. However, in order to efficiently use the above tools, there is a need for a resource configuration specialized for a specific workload. In response, the concept of a multi-cluster has been proposed, but it requires a component and a concept that manages the structure along with the construction of a multi-cluster. In response, this paper introduces SmartX AI Cluster and Tenants Portal built on K8S-based and operated by this research team. Extended SmartX AI Cluster provides a multi-K8S cluster environment that combined single K8S clusters with different types of resources, unlike the previous method that operated as a single K8S cluster. In addition, Tenants Portal is an entity that supports multiple users to use the environment at the same time without building a separate K8S environment. Finally, in the proposed environment, machine learning (ML) workload verification is performed through parallel and distributed processing and inference service cases, and related studies are discussed.