Hybrid Actor-Critic Based Low-Overhead Scheduling Using MDP for Large-Scale Edge Computing Networks

Abstract

The inconsistency of application requirements in large-scale edge computing networks leads to the need for edge computing scheduling to optimize multiple objectives at the same time. There are significant inherent conflicts between these objectives, and optimizing a single objective often comes at the expense of sacrificing the performance of other objectives, making it difficult to achieve global optimality in large-scale dynamic edge environments. Therefore, a hybrid Actor-Critic Based Low-Overhead Scheduling Using MDP for Large-Scale is proposed. Build a three-layer architecture of "end edge cloud", including terminal collection, edge processing upload, and cloud storage analysis. Enable data flow and processing at different levels, comprehensively analyze the performance and interrelationships of various targets at different levels, and reduce bandwidth and cloud load. Set targets for transmission efficiency, computation delay, transmission delay, and energy consumption, and comprehensively evaluate scheduling performance through cumulative weighted overhead. Considering the impact of various objectives on a global scale, scheduling strategies need to comprehensively consider all objectives when formulating. To achieve this goal, a Markov decision process is used to model the system dynamics, balancing unloading and resource allocation through state space, action space, and reward function, accurately capturing the characteristics of large-scale dynamic edge environments. Considering the changes in system state over time and the impact of different actions on system state, long-term weighted overhead is minimized; At the same time, the hybrid actor critic algorithm is introduced to process different types of actions through discrete and continuous actor networks, and the experience playback mechanism is used to solve the problem of real-time reward acquisition in asynchronous environments. In addition, the critical network is combined to optimize resource allocation, effectively handle complex action space, further balance the relationship between multiple targets, and achieve efficient and low-cost scheduling of large-scale edge computing network computing resources. Experimental results show that the proposed method performs well in large-scale edge computing network resource scheduling. In terms of resource utilization, the average is 93.2%, and the response time is always below 4ms. In terms of energy efficiency, the average task energy consumption is the lowest, saving more than one-third of energy compared to the comparative method. In scalability evaluation, when facing large-scale tasks, the cumulative weighted overhead increases almost linearly, with a growth rate far lower than the comparative method, demonstrating excellent scalability.