Edge AI with Kubernetes: Deploying machine learning models at scale
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V3I2P104Keywords:
Edge AI, Kubernetes, KubeEdge, Federated Learning, Model Optimization, Real-time InferenceAbstract
The use of Edge AI has grown rapidly over the years, and thus, there has been a need for deployment frameworks that can efficiently and scalable deploy the edge models. As a platform for container orchestration, Kubernetes has been adopted as an effective solution for managing a number of ML tasks on the edge. Kubernetes adds value in the scenario of deploying new AI models at the edge nodes due to the features of containerization, microservices, and auto-scaling. This paper aims to discuss the integration of Kubernetes with Edge AI; it is a broad area that deals with deployment architectures, the management of resources, and optimization. In this paper, different categories of model deployment are reviewed and highlighted, such as containerized inference, serverless AI, and federated learning, with the understanding of latency, security, and scalability challenges. Ways in which Edge AI has been implemented to solve real-life problems have been presented by two appealing use cases: distributed image processing utilizing Kubernetes and real-time face recognition with KubeEdge. When comparing the figures, one can also find that latency and resource usage have been reduced significantly, making Edge AI a suitable replacement for Cloud ML solutions.
Additionally, the paper also presents research opportunities in model optimization, real-time artificial intelligence scheduling, private artificial intelligence, and decentralized learning architecture. Thus, it has been pointed out that future excellent, autonomous, and intelligent AI applications will be developed based on Kubernetes at the Edge. Despite the transferring of AI work in a distributed edge environment, Kubernetes will be the keyframe for the scalability and efficiency of AI at scale
Downloads
References
[1] Toka, L., Dobreff, G., Fodor, B., & Sonkoly, B. (2021). Machine learning-based scaling management for Kubernetes edge clusters. IEEE Transactions on Network and Service Management, 18(1), 958-972.
[2] Merenda, M., Porcaro, C., & Iero, D. (2020). Edge machine learning for AI-enabled IoT devices: A review. Sensors, 20(9), 2533.
[3] Liu, B. (2019). Study and benchmarking of Artificial Intelligence (AI) model serving systems on edge computation units and cloud environments.
[4] Rao, D. (2019). Keras to Kubernetes: The journey of a machine learning model to production. John Wiley & Sons.
[5] Toka, L., Dobreff, G., Fodor, B., & Sonkoly, B. (2020, May). Adaptive AI-based auto-scaling for Kubernetes. In 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID) (pp. 599-608). IEEE.
[6] Huang, Y., Cai, K., Zong, R., & Mao, Y. (2019, March). Design and implement an edge computing platform architecture using Docker and Kubernetes for machine learning, in Proceedings of the 3rd International Conference on High-Performance Compilation, Computing and Communications (pp. 29-32).
[7] Jim Davis, Study shows AI/ML is the top workload for Kubernetes; edge IoT projects are on the rise, edgeir, online. https://www.edgeir.com/study-shows-ai-ml-is-top-workload-for-kubernetes-edge-iot-projects-on-the-rise-20220109
[8] Pääkkönen, P., Pakkala, D., Kiljander, J., & Sarala, R. (2020). Architecture for enabling edge inference via model transfer from the cloud domain in a Kubernetes environment. Future Internet, 13(1), 5.
[9] Kubernetes on the Edge: Getting Started with KubeEdge and Kubernetes for Edge Computing, influxdata, online. https://www.influxdata.com/blog/kubernetes-edge-kubeedge-edge-computing/
[10] Kubernetes deployment models for edge applications, 2022. online. https://www.redhat.com/en/blog/kubernetes-edge-applications
[11] Kubernetes for the Edge: Key Developments & Implementations, rancher, online. https://www.suse.com/c/rancher_blog/kubernetes-for-the-edge-key-developments-implementations/
[12] Kum, S., Kim, Y., Siracusa, D., & Moon, J. (2020, November). Artificial intelligence service architecture for edge devices. In 2020 IEEE 10th International Conference on Consumer Electronics (ICCE-Berlin) (pp. 1-3). IEEE.
[13] Harnessing Kubernetes for Machine Learning: Scaling and Orchestrating ML Projects, Medium, online. https://medium.com/@neeraztiwari/harnessing-kubernetes-for-machine-learning-scaling-and-orchestrating-ml-projects-f2d08ea77ede
[14] Böhm, S., & Wirtz, G. (2022). Cloud-edge orchestration for smart cities: A review of Kubernetes-based orchestration architectures. EAI Endorsed Trans. Smart Cities, 6(18), e2.
[15] Kristiani, E., Yang, C. T., Huang, C. Y., Wang, Y. T., & Ko, P. C. (2021). Implementing a cloud-edge computing architecture using OpenStack and Kubernetes for air quality monitoring applications. Mobile Networks and Applications, 26(3), 1070-1092.
[16] Kim, E., Lee, K., & Yoo, C. (2021, January). On the resource management of Kubernetes. In 2021 International Conference on Information Networking (ICOIN) (pp. 154-158). IEEE.
[17] Medel, V., Rana, O., Bañares, J. Á., & Arronategui, U. (2016, December). Modelling performance & resource management in Kubernetes. In Proceedings of the 9th International Conference on Utility and Cloud Computing (pp. 257-262).
[18] Mao, Y., Fu, Y., Gu, S., Vhaduri, S., Cheng, L., & Liu, Q. (2020). Resource management schemes for cloud-native platforms with computing containers of docker and Kubernetes. arXiv preprint arXiv:2010.10350.
[19] Medel, V., Tolosana-Calasanz, R., Bañares, J. Á., Arronategui, U., & Rana, O. F. (2018). Characterising resource management performance in Kubernetes. Computers & Electrical Engineering, 68, 286-297.
[20] Verreydt, S., Beni, E. H., Truyen, E., Lagaisse, B., & Joosen, W. (2019, December). Leveraging Kubernetes for adaptive and cost-efficient resource management. In Proceedings of the 5th International Workshop on Container Technologies and Container Clouds (pp. 37-42).
