Deep Reinforcement Learning-Based Real-Time Trading Decision Support for Virtual Power Plants with Intelligent Assistant Integration
Abstract
With the profound transformation of the energy structure and the advancement of the “dual carbon” goal, the virtual power plant (VPP), centered on distributed energy resources, has emerged as a key technology for enhancing the flexibility and integration capacity of modern power grids. However, the diversity and volatility of VPP internal resources, coupled with the complexity of the electricity market, impose significant challenges on the response speed and economic efficiency of real-time trading decisions.To address these challenges, this paper proposes and develops a real-time trading decision support system for VPPs driven by an Intelligent Assistant (IA). The system leverages hydropower plants—with their fast response and energy storage capabilities—as the core regulating resources and coordinates multiple distributed energy sources, including photovoltaics, wind power, and energy storage systems. At its core, the IA integrates deep learning–based forecasting models, reinforcement learning–based decision modules, and a natural language processing (NLP)–based interaction component. The IA assists operators in real time by analyzing multidimensional data such as market prices, grid loads, meteorological information, and hydropower inflows, accurately predicting generation and price trends, and dynamically optimizing bidding and regulation strategies through reinforcement learning algorithms to maximize overall benefits.This paper details the overall architecture and key technological components of the proposed system and conducts a simulation case study using a regional VPP containing multiple hydropower plants. Specifically, the core decision-making module employs the Soft Actor-Critic (SAC) deep reinforcement learning algorithm. The system was trained for 2 million steps over 72 hours on a V100 GPU, utilizing one year of real historical operational and market data from a VPP in Southwest China. The simulation results demonstrate that the proposed IA-DRL strategy outperforms a traditional rolling-horizon Mixed-Integer Linear Programming (MILP) method, achieving a 14.7% increase in net profit, a 47.8% reduction in deviation assessment costs, and a remarkable 99.7% acceleration in decision-making time. These results confirm the significant technical and economic advantages of the proposed framework, providing a new theoretical foundation and practical solution for intelligent VPP operation and business model innovation, while also offering enhanced interpretability through the intelligent assistant.References
Zare, A., & Shafiyi, M. A. (2025). Virtual power plant models and market participation: A deep dive into optimization and real-world applications. Results in Engineering, 26, 105548. https://doi.org/10.1016/j.rineng.2025.105548
Abdelkader, S., Amissah, J., & Abdel-Rahim, O. (2024). Virtual power plants: an in-depth analysis of their advancements and importance as crucial players in modern power systems. Energy, Sustainability and Society, 14(1), 52. https://doi.org/10.1186/s13705-024-00483-y
Ruan, G., Qiu, D., Sivaranjani, S., Awad, A. S. A., & Strbac, G. (2024). Data-driven energy management of virtual power plants: A review. Advances in Applied Energy, 14, 100170. https://doi.org/10.1016/j.adapen.2024.100170
Al-Shetwi, A. Q., Al-Shaalan, A. M., El-Ela, A. A., & El-Sehiemy, R. A. (2023). Adaptive power management strategy for microgrids considering virtual power plants. Sustainable Energy Technologies and Assessments, 59, 103328. https://doi.org/10.1016/j.seta.2023.103328
Juma, S. A., Ayeng’o, S. P., & Kimambo, C. Z. M. (2024). A review of control strategies for optimized microgrid operations. IET Renewable Power Generation, 18(14), 2785–2818. https://doi.org/10.1049/rpg2.13056
Alam, M. M., Hossain, M. J., Habib, M. A., Arafat, M. Y., & Hannan, M. A. (2025). Artificial intelligence integrated grid systems: Technologies, potential frameworks, challenges, and research directions. Renewable and Sustainable Energy Reviews, 211, 115251. https://doi.org/10.1016/j.rser.2024.115251
Kumar, A., Maulik, A., & Chinmaya, K. A. (2025). Energy Management Strategies for Active Distribution Networks and Microgrids – A Comprehensive Survey. IETE Technical Review, 1-20. https://doi.org/10.1080/02564602.2025.2522083
Alsaigh, R., Mehmood, R., & Katib, I. (2023). AI explainability and governance in smart energy systems: A review. Frontiers in Energy Research, 11. https://doi.org/10.3389/fenrg.2023.1071291
Li, Y., Chang, W., & Yang, Q. (2025). Deep reinforcement learning based hierarchical energy management for virtual power plant with aggregated multiple heterogeneous microgrids. Applied Energy, 382, 125333.https://doi.org/10.1016/j.apenergy.2025.125333
Gong, X., Li, X., & Zhong, Z. (2025). Strategic bidding of virtual power plants in integrated electricity-carbon-green certificate market with renewable energy uncertainties. Sustainable Cities and Society, 121, 106176.https://doi.org/10.1016/j.scs.2025.106176
Qiu, D., Wang, Y., Hua, W., & Strbac, G. (2023). Reinforcement learning for electric vehicle applications in power systems: A critical review. Renewable and Sustainable Energy Reviews, 173, 113052. https://doi.org/10.1016/j.rser.2022.113052
Xu, Y., Liao, Y., Kuang, S., Ma, J., & Wen, T. (2025). Virtual Power Plant Optimization Process Under the Electricity–Carbon–Certificate Multi-Market: A Case Study in Southern China. Processes, 13(7), 2148.https://doi.org/10.3390/pr13072148
Wang, S., Sheng, W., Shang, Y., & Liu, K. (2024). Distribution network voltage control considering virtual power plants cooperative optimization with transactive energy. Applied Energy, 371, 123680. https://doi.org/10.1016/j.apenergy.2024.123680
Mahmood, M., Chowdhury, P., Yeassin, R., Hasan, M., Ahmad, T., & Chowdhury, N. U. R. (2024). Impacts of digitalization on smart grids, renewable energy, and demand response: An updated review of current applications. Energy Conversion and Management: X, 24, 100790.https://doi.org/10.1016/j.ecmx.2024.100790
Sun, Z., & Lu, T. (2024). Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism. IET Generation, Transmission & Distribution, 18(1), 39-49.https://doi.org/10.1049/gtd2.13037
Tang, X., & Wang, J. (2025). Deep Reinforcement Learning-Based Multi-Objective Optimization for Virtual Power Plants and Smart Grids: Maximizing Renewable Energy Integration and Grid Efficiency. Processes, 13(6), 1809.https://doi.org/10.3390/pr13061809
Feng, B., Liu, Z., Huang, G., & Guo, C. (2023). Robust federated deep reinforcement learning for optimal control in multiple virtual power plants with electric vehicles. Applied Energy, 349, 121615. https://doi.org/10.1016/j.apenergy.2023.121615
Li, G., Zhang, R., Bu, S., Zhang, J., & Gao, J. (2024). Probabilistic prediction-based multi-objective optimization approach for multi-energy virtual power plant. International Journal of Electrical Power & Energy Systems, 161, 110200.https://doi.org/10.1016/j.ijepes.2024.110200
Yan, C., & Qiu, Z. (2025). Review of Power Market Optimization Strategies Based on Industrial Load Flexibility. Energies, 18(7), 1569. https://doi.org/10.3390/en18071569
Boulkroune, A., Hamel, S., Zouari, F., Boukabou, A., & Ibeas, A. (2017). Output‐Feedback Controller Based Projective Lag‐Synchronization of Uncertain Chaotic Systems in the Presence of Input Nonlinearities. Mathematical Problems in Engineering, 2017(1), 8045803.https://doi.org/10.1155/2017/8045803
Boulkroune, A., Zouari, F., & Boubellouta, A. (2025). Adaptive fuzzy control for practical fixed-time synchronization of fractional-order chaotic systems. Journal of Vibration and Control, 10775463251320258. https://doi.org/10.1177/10775463251320258
Zouari, F., Saad, K. B., & Benrejeb, M. (2013, March). Adaptive backstepping control for a class of uncertain single input single output nonlinear systems. In 10th International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13) (pp. 1-6).IEEE.https://doi.org/10.1109/SSD.2013.6564134
Rigatos, G., Abbaszadeh, M., Sari, B., Siano, P., Cuccurullo, G., & Zouari, F. (2023). Nonlinear optimal control for a gas compressor driven by an induction motor. Results in Control and Optimization, 11, 100226.https://doi.org/10.1016/j.rico.2023.100226
Merazka, L., Zouari, F., & Boulkroune, A. (2017, May). High-gain observer-based adaptive fuzzy control for a class of multivariable nonlinear systems. In 2017 6th International Conference on Systems and Control (ICSC) (pp. 96-102). IEEE.https://doi.org/ 10.1109/ICoSC.2017.7958728
Baur, L., Ditschuneit, K., Schambach, M., Kaymakci, C., Wollmann, T., & Sauer, A. (2024). Explainability and Interpretability in Electric Load Forecasting Using Machine Learning Techniques – A Review. Energy and AI, 16, 100358. https://doi.org/10.1016/j.egyai.2024.100358
DOI:
https://doi.org/10.31449/inf.v49i35.12599Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
