Topology-Aware, Performance-Driven Adaptive Routing in Software-Defined Networks Using Dual-Agent Reinforcement Learning

Deepthi Goteti, Vurrury Krishna Reddy

Abstract


This research explores adaptive routing in Software-Defined Networks (SDNs) using reinforcement learning. Two models—R-Learner (Q-learning) and R-Optimizer (policy-gradient)—are evaluated against the Dijkstra baseline across four topologies: Fat Tree, Abilene, Custom, and Dense Adaptive Mesh. Experiments run over 100 TCP/UDP traffic episodes using Mininet and the Ryu controller. Key metrics include throughput, jitter, round-trip time (RTT), and packet loss ratio (PLR). Statistically validated results show R-Optimizer outperforms R-Learner, achieving ~7.4% higher throughput, 44% lower jitter, 19.5% lower RTT, and >50% lower packet loss. Both models also surpass Dijkstra in throughput and delay reduction. These results support reinforcement learning as a viable approach for real-time SDN routing and future controller integration.

Full Text:

PDF

References


G. Wu, Deep reinforcement learning based multi-layered traffic scheduling scheme in data center networks, Wireless Networks, vol. 30, pp. 4133–4144, 2024. https://doi.org/10.1007/s11276-021-02883-w.

S. Sezer et al., Are we ready for SDN? Implementation challenges for software-defined networks, IEEE Communications Magazine, vol. 51, no. 7, pp. 36–43, 2013. https://doi.org/10.1109/MCOM.2013.6553676

D. Kreutz, F. M. V. Ramos, P. E. Verissimo, C. E. Rothenberg, S. Azodolmolky, and S. Uhlig, Software-Defined Networking: A Comprehensive Survey, Proceedings of the IEEE, vol. 103, no. 1, pp. 14–76, 2015. https://doi.org/10.1109/JPROC.2014.2371999.

Open Networking Foundation (ONF), Software-Defined Networking: The New Norm for Networks, White Paper, 2018. [Online]. Available: https://opennetworking.org/wp-content/uploads/2011/09/wp-sdn-newnorm.pdf.

P. Kamboj, S. Pal, S. Bera, and S. Misra, QoS-Aware Multipath Routing in Software-Defined Networks, IEEE Transactions on Network Science and Engineering, vol. 10, no. 2, pp. 723–732, 2023. https://doi.org/10.1109/TNSE.2022.3219417.

A. Dixit, F. Hao, S. Mukherjee, T. Lakshman, and R. Kompella, Towards an Elastic Distributed SDN Controller, ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN), pp. 7–12, 2013. https://doi.org/10.1145/2491185.2491193.

S. Yasmine, U. Prabu, Y. S. D. Phaneendra, and V. Geetha, An Effective Deployment of Controllers in Software-Defined Networks, Procedia Computer Science, vol. 233, pp. 77–86, 2024. https://doi.org/10.1016/j.procs.2024.03.197

K. Suh, S. Kim, Y. Ahn, S. Kim, H. Ju, and B. Shim, Deep Reinforcement Learning-Based Network Slicing for Beyond 5G, IEEE Access, vol. 10, pp. 7384–7395, 2022. https://doi.org/10.1109/ACCESS.2022.3141789.

R. Boutaba, M. A. Salahuddin, N. Limam, S. Ayoubi, N. Shahriar, F. Estrada-Solano, and O. M. Caicedo, A Comprehensive Survey on Machine Learning for Networking: Evolution, Applications and Research Opportunities, Journal of Internet Services and Applications, vol. 9, no. 1, pp. 1–99, 2018. https://doi.org/10.1186/s13174-018-0087-2.

G. Kim, Y. Kim, and H. Lim, Deep Reinforcement Learning-Based Routing on Software-Defined Networks, IEEE Access, vol. 10, pp. 18121–18133, 2022. https://doi.org/10.1109/ACCESS.2022.3151081.

S. Park, S. Ju, and J.-Y. Lee, Efficient Routing for Traffic Offloading in Software-defined Network, Procedia Computer Science, vol. 34, pp. 674–679, 2014. https://doi.org/10.1016/j.procs.2014.07.096.

X. Li, J. Li, J. Zhou, and J. Liu, Towards Robust Routing: Enabling Long-Range Perception with the Power of Graph Transformers and Deep Reinforcement Learning in Software-Defined Networks, Electronics, vol. 14, no. 3, p. 476, 2025. https://doi.org/10.3390/electronics14030476.

Y. Al-Dunainawi, B. R. Al-Kaseem, and H. S. Al-Raweshidy, Optimized Artificial Intelligence Model for DDoS Detection in SDN Environment, IEEE Access, vol. 11, pp. 6733–6745, 2023. https://doi.org/10.1109/ACCESS.2023.3319214.

J. Bhayo, S. Hameed, S. Shah, J. Nasir, A. Ahmed, and D. Draheim, A Novel DDoS Attack Detection Framework for Software-Defined IoT (SD-IoT) Networks Using Machine Learning, Engineering Applications of Artificial Intelligence, vol. 123, p. 106432, 2023. https://doi.org/10.1016/j.engappai.2023.106432.

L. Lei, Y. Tan, K. Zheng, S. Liu, K. Zhang, and X. Shen, Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges, IEEE Communications Surveys & Tutorials, vol. 22, no. 3, pp. 1722–1760, 2020. https://doi.org/10.48550/arXiv.1907.09059.

J. Xie et al., A Survey of Machine Learning Techniques Applied to Software Defined Networking (SDN): Research Issues and Challenges, IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 393–430, 2019. https://doi.org/10.1109/COMST.2018.2866942.

W. Teka and N. Arappali, Software Defined Networking (SDN) Control Plane in Evaluation for Reliability and Fault Tolerance, International Journal of Engineering and Advanced Technology, vol. 7, 2020.

D. B. Prakoso, M. Salman, and R. F. Sari, A survey of deep reinforcement learning based routing optimization in SDN, AIP Conference Proceedings, vol. 3215, p. 080009, 2024. https://doi.org/10.1063/5.0235840.

B. Lantz, B. Heller, and N. McKeown, A Network in a Laptop: Rapid Prototyping for Software-Defined Networks, Proceedings of the 9th ACM Workshop on Hot Topics in Networks (HotNets-X), pp. 1–6, 2010. https://doi.org/10.1145/1868447.1868466.

Y. Li, X. Guo, X. Pang, B. Peng, X. Li, and P. Zhang, Performance Analysis of Floodlight and Ryu SDN Controllers under Mininet Simulator, IEEE/CIC International Conference on Communications in China (ICCC Workshops), pp. 85–90, 2020. https://doi.org/10.1109/ICCCWorkshops49972.2020.9209935.

A. Tirumala, M. Kamran, and K. Malik, Iperf: Bandwidth Measurement Tool, [Online]. Available: https://iperf.fr/.

M. Jarschel, T. Zinner, T. Hossfeld, P. Tran-Gia, and W. Kellerer, Interfaces, attributes, and use cases: A compass for SDN, IEEE Communications Magazine, vol. 52, no. 6, pp. 210–217, 2014. https://doi.org/10.1109/MCOM.2014.6829966

Ryu SDN Framework Official Documentation, [Online]. Available: https://osrg.github.io/ryu/.

V. Pereira, M. Rocha, and P. Sousa, Traffic Engineering With Three-Segments Routing, IEEE Transactions on Network and Service Management, vol. 17, no. 3, pp. 1896–1909, 2020. https://doi.org/10.1109/TNSM.2020.2993207.

B. Fortz and M. Thorup, Internet Traffic Engineering by Optimizing OSPF Weights, IEEE/ACM Transactions on Networking, vol. 10, no. 2, pp. 245–252, 2002. https://doi.org/10.1109/INFCOM.2000.832225.

W. Liu, Intelligent Routing based on Deep Reinforcement Learning in Software-Defined Data-Center Networks, IEEE Symposium on Computers and Communications (ISCC), pp. 1–6, 2019. https://doi.org/10.1109/ISCC47284.2019.8969579.

S. Yan, X. Zhang, L. Zhao, and H. Zhang, Intelligent Routing Based on Deep Deterministic Policy Gradient in SDN, IEEE Communications Letters, vol. 25, no. 1, pp. 104–108, 2021. https://doi.org/10.1109/LCOMM.2020.3021333.

G. Kim, Y. Kim, and H. Lim, Deep Reinforcement Learning-Based Routing on Software-Defined Networks, IEEE Access, vol. 10, pp. 18121–18133, 2022. https://doi.org/10.1109/ACCESS.2022.3151081.

W. Wang, X. Zhang, L. Zhang, and L. Zhao, Reusable Reinforcement Learning for Intelligent Routing in SDN, arXiv preprint arXiv:2409.15226, 2024. https://arxiv.org/abs/2409.15226.

L. Chen, B. Hu, Z.-H. Guan, L. Zhao, and X. Shen, Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization, IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 10, pp. 5374–5386, 2022. https://doi.org/10.1109/TNNLS.2021.3070584

K. Rusek, J. Suarez-Varela, P. Almasan, P. Barlet-Ros, and A. Cabello, RouteNet: Leveraging Graph Neural Networks for Network Modeling and Optimization in SDN, IEEE Journal on Selected Areas in Communications, 2020. https://doi.org/10.1109/JSAC.2020.3000405.

X. Li, J. Li, J. Zhou, and J. Liu, Towards Robust Routing: Enabling Long-Range Perception with the Power of Graph Transformers and Deep Reinforcement Learning in Software-Defined Networks, Electronics, vol. 14, no. 3, p. 476, 2025. https://doi.org/10.3390/electronics14030476.

Y. Al-Dunainawi, B. Al-Kaseem, and H. S. Al-Raweshidy, Optimized Artificial Intelligence Model for DDoS Detection in SDN Environment, IEEE Access, vol. 11, pp. 6733–6745, 2023. https://doi.org/10.1109/ACCESS.2023.3319214.

Y. Guo, Q. Tang, Y. Ma, H. Tian, and K. Chen, Distributed Traffic Engineering in Hybrid Software Defined Networks: A Multi-agent Reinforcement Learning Framework, 2023. https://doi.org/10.48550/arXiv.2307.15922.

J. Wang, L. Codecà, and Z. Li, Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control, IEEE Transactions on Intelligent Transportation Systems, 2019. https://doi.org/10.1109/TITS.2019.2901791.

Mininet Official Repository, [Online]. Available: https://github.com/mininet/mininet.

S. Bhardwaj and S. N. Panda, Performance Evaluation Using RYU SDN Controller in Software-Defined Networking Environment, Wireless Personal Communications, vol. 122, pp. 701–723, 2022. https://doi.org/10.1007/s11277-021-08920-3.

Learn SDN with Ryu GitHub Repository, [Online]. Available: https://github.com/knetsolutions/learn-sdn-with-ryu.

Ryu-Mininet Custom Topology Example GitHub Repository, [Online]. Available: https://github.com/byaussy/ryu-mininet-custom.




DOI: https://doi.org/10.31449/inf.v49i15.9125

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.