Optimization of Multi-Task Efficient Offloading Strategy for Heterogeneous Edge Computing Based on Deep Reinforcement Learning

Sen Zhu

doi:10.54097/cpmvcr09

Authors

Sen Zhu

DOI:

https://doi.org/10.54097/cpmvcr09

Keywords:

Edge Computing, Deep Reinforcement Learning, Resource Adaptation, Task Scheduling

Abstract

Edge computing is a computing paradigm that provides fast and efficient computing services to mobile devices by bringing resources closer to the edge of the network. However, current edge servers lack the computational power to handle the large volume of tasks generated by connected devices. Moreover, some mobile devices may not fully utilize their computational resources. In order to maximize the use of resources, a more sustainable and energy efficient computing paradigm was developed. Minimize the use of Mobile Edge Computing (MEC) servers for applications that run in random environments with unpredictable workloads. And utilizing available resource-constrained edge devices to keep resource-rich servers idle to handle any unpredictable larger workloads. Our focus is on optimizing device computation offloading to minimize the maximum task processing delay in the system. This problem has been proven to be NP-hard. To address this problem, we propose an efficient offloading algorithm based on deep Q-network (DQNEO). The algorithm effectively utilizes computational resources in the system and makes intelligent scheduling decisions based on system state information using deep reinforcement learning. Experimental results show that the DQNEO algorithm can reduce the system processing delay by converging to the optimal solution.

Downloads

Download data is not yet available.

References

[1] Wu Y, Ni K, Zhang C, Qian LP, Tsang DH. NOMA-assisted multi-access mobile edge computing: A joint optimization of computation offloading and time allocation. IEEE Transactions on Vehicular Technology. 2018; 67(12):12244-58.

[2] Li J, Qiu M, Ming Z, Quan G, Qin X, Gu Z. Online optimization for scheduling preemptable tasks on IaaS cloud systems. Journal of Parallel and Distributed Computing. 2012; 72(5):666-77.

[3] Calheiros RN, Buyya R. Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Transactions on Parallel & Distributed Systems. 2013; 25(7):1787-96.

[4] Abrishami S, Naghibzadeh M, Epema DH. Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Generation Computer Systems. 2013; 29(1):158-69.

[5] Cai Z, Li X, Ruiz R, Li Q. A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds. Future Generation Computer Systems. 2017; 71:57-72.

[6] Jiang H, Song M. Dynamic scheduling of workflow for makespan and robustness improvement in the IaaS cloud. IEEE Transactions on Information and Systems. 2017; 100(4):813-21.

[7] Zuo X, Zhang G, Tan W. Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IaaS cloud. IEEE Transactions on Automation Science and Engineering. 2013; 11(2):564-73.

[8] Fu Z, Tang Z, Yang L, Liu C. An optimal locality-aware task scheduling algorithm based on bipartite graph modelling for spark applications. IEEE Transactions on Parallel and Distributed Systems. 2020; 31(10):2406-20.

[9] Yuan H, Bi J, Zhou M. Energy-efficient and QoS-optimized adaptive task scheduling and management in clouds. IEEE Transactions on Automation Science and Engineering. 2020; 19(2):1233-44.

[10] Bezdan T, Zivkovic M, Bacanin N, Strumberger I, Tuba E, Tuba M. Multi-objective task scheduling in cloud computing environment by hybridized bat algorithm. Journal of intelligent & fuzzy systems. 2022; 42(1):411-23.

[11] Attiya I, Abd Elaziz M, Abualigah L, Nguyen TN, Abd El-Latif AA. An improved hybrid swarm intelligence for scheduling iot application tasks in the cloud. IEEE Transactions on Industrial Informatics. 2022; 18(9):6264-72.

[12] Yuan H, Bi J, Zhou M. Spatial task scheduling for cost minimization in distributed green cloud data centers. IEEE Transactions on Automation Science and Engineering. 2018; 16(2):729-40.

[13] Verma A, Kaushal S. A hybrid multi-objective particle swarm optimization for scientific workflow scheduling. Parallel Computing. 2017; 62:1-19.

[14] Wu Z, Xiong J. A novel task-scheduling algorithm of cloud computing based on particle swarm optimization. International Journal of Gaming and Computer-Mediated Simulations. 2021; 13(2):1-15.

[15] Domanal SG, Guddeti RMR, Buyya R. A hybrid bio-inspired algorithm for scheduling and resource management in cloud environment. IEEE Transactions on Services Computing. 2017; 13(1):3-15.

[16] Shiue Y-R, Lee K-C, Su C-T. Real-time scheduling for a smart factory using a reinforcement learning approach. Computers & Industrial Engineering. 2018; 125:604-14.

[17] Sinde R, Begum F, Njau K, Kaijage S. Refining network lifetime of wireless sensor network using energy-efficient clustering and DRL-based sleep scheduling. Sensors. 2020; 20(5):1540.

[18] Qi Q, Zhang L, Wang J, Sun H, Zhuang Z, Liao J, et al. Scalable parallel task scheduling for autonomous driving using multi-task deep reinforcement learning. IEEE Transactions on Vehicular Technology. 2020; 69(11):13861-74.

[19] Zeng D, Gu L, Pan S, Cai J, Guo S. Resource management at the network edge: A deep reinforcement learning approach. IEEE Network. 2019; 33(3):26-33.

[20] Sheng S, editor A resource collaborative scheduling strategy based on cloud edge framework. 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC); 2021: IEEE.

[21] Jalali F, Hinton K, Ayre R, Alpcan T, Tucker RS. Fog computing may help to save energy in cloud computing. IEEE Journal on Selected Areas in Communications. 2016; 34(5):1728-39.