Approximate SARSA Algorithm for Dimensionality-Challenged Resource Allocation Optimization in MIMO Communication Systems

Abstract

An approximate sate-action-reward-state-action (ASARSA) algorithm is proposed to solve the resource allocation optimization in multiple-input multiple-output (MIMO) communication systems, especially in the context of energy harvesting (EH) wireless communication systems. ASARSA algorithm aims to overcome the dimensional disaster problem faced by traditional SARSA algorithm in high-dimensional state space. By transforming the resource allocation optimization problem into a Markov decision-making problem and applying reinforcement learning, this study realizes the resource allocation optimization of EH-MIMO system. The experimental results showed that the system throughput of ASARSA algorithm reached 15.0×105 bits under the condition of 100 slots, which was 0.2×105 bits and 3.6×105 bits higher than that of SARSA and Q-Learning (QL) algorithms, respectively. In terms of convergence speed, ASARSA algorithm was close to the target accuracy after 76 iterations, which was 25 iterations and 77 iterations less than SARSA and QL algorithms, respectively. In addition, the average absolute error and root mean square error of ASARSA algorithm were 3.54% and 3.10%, which were 1.27% and 0.58%, 2.01% and 1.12% lower than those of SARSA and QL algorithms, respectively. These results show that ASARSA algorithm has higher efficiency and better optimization effect in resource allocation optimization. It is also found that ASARSA algorithm can maintain high computational efficiency and low approximate error, which proves its effectiveness and reliability in practical applications. Therefore, ASARSA algorithm can effectively optimize the allocation of EH-MIMO resources, solve the shortage of spectrum resources to some extent, and promote the development of EH-MIMO technology.