HDRL-MDIB: A Hierarchical Deep Reinforcement Learning-Based Multi-Dimensional Billing System for Computing Power Networks

Abstract

Developing an intelligent billing system for computing power networks, where dynamic pricing, accurate demand forecasting, and low-carbon strategies are crucial, this paper introduces the Hierarchical Deep Reinforcement Learning-Based Multi-Dimensional Intelligent Billing (HDRL-MDIB) Algorithm. The algorithm integrates a Transformer-based task feature extraction network, a hybrid market modeling method using Graph Neural Networks and Variational Autoencoders, and a Hierarchical Reinforcement Learning framework to optimize pricing decisions. Robustness enhancement mechanisms, such as adversarial training and adaptive strategies, ensure stability in dynamic environments. Experiments show that HDRLMDIB outperforms existing methods in prediction accuracy, operational efficiency, and real-world business scenario deployments. The intelligent billing system suffers from inaccurate demand forecasts and fixed pricing, making it difficult to adapt to dynamic changes in user demand. This paper introduces theHierarchical Deep Reinforcement Learning-Based Multi-Dimensional Intelligent Billing (HDRL-MDIB) algorithm. Specifically, we propose a Transformer-based task feature extraction network with multi-head attention mechanisms for accurate temporal pattern recognition. Secondly, we introduce a hybrid market modeling approach that combines Graph Attention Networks (GAT) for user relationship modeling withConditional Variational Autoencoders (CVAE) to accurately predict user needs. Additionally, we propose a Hierarchical Reinforcement Learning framework with high-level pricing strategy selection and low-level parameter optimization. The robustness of the system is further enhanced through adversarial training and adaptive strategy mechanisms. Experiments conducted on two real-world datasets demonstrate thatHDRL-MDIB achieves 15.3% higher prediction accuracy compared to state-of-the-art methods, reduces operational costs by 22.7%, improves revenue optimization by 18.4%, and decreases carbon emissions by 12.6% compared to traditional rule-based and single-agent reinforcement learning approaches.