Intelligent Traffic Signal Control Based on Reinforcement Learning with Edge Computing and Intelligent Reflecting Surface

Guanghua Zhang; Youchen Yue; Jinghao Yang; Jie Tang

doi:10.54097/713j5n26

Authors

Guanghua Zhang
Youchen Yue
Jinghao Yang
Jie Tang

DOI:

https://doi.org/10.54097/713j5n26

Keywords:

Proximal Policy Optimization, Edge Computing, Intelligent Reflecting Surfaces

Abstract

With increasing traffic demand and growing environmental complexity, traditional signal control methods struggle with low efficiency and poor adaptability. This study proposes a reinforcement learning-based framework integrating edge computing and intelligent reflecting surfaces (IRS) to enhance traffic signal control. The framework is validated through simulation and optimized via an improved Proximal Policy Optimization (PPO) algorithm with multi-step returns and entropy-based adaptive regularization, ensuring better convergence and stability. Experiments on a 5×5 intersection network in SUMO, under varying traffic scenarios, including ablation and comparative studies, show that the proposed method significantly reduces average delay and improves throughput, demonstrating its effectiveness and reliability.

Downloads

Download data is not yet available.

References

[1] Xu Dongwei, Zhou Lei, Wang Da, et al. Review of Urban Traffic Signal Control Based on Deep Reinforcement Learning [J]. Journal of Transportation Engineering and Information, 2022, 20(01): 15-30.

[2] Liu Zhimin, Ye Baolin, Zhu Yaodong, et al. Traffic Signal Control Methods Based on Deep Reinforcement Learning [J]. Journal of Zhejiang University: Engineering Edition, 2022, 56(6): 8.

[3] Haydari A, Yilmaz Y. Deep Reinforcement Learning for Intelligent Transportation Systems: Survey [J]. IEEE Transactions on Intelligent Transportation Systems. 2022, 23(1):11-32.

[4] Liang, X., Du, X., Wang, G., Han, Z.: A deep reinforcement learning network for traffic light cycle control. IEEE Trans. Veh. Technol. 68, 1243–1253 (2019).

[5] Alemzadeh, S., Moslemi, R., Sharma, R., Mesbahi, M.: Adaptive traffic control with deep reinforcement learning towards state-of-the-art and beyond. arXiv:2007.10960 (2020).

[6] Vidali, A., Crociani, L., Vizzari, G., Bandini, S.: A deep reinforce-ment learning approach adaptive traffic lights management. In: WOA (2019).

[7] Mousavi SS, Schukat M, Howley E. Traffic light control using deep policy-gradient and value-function-based reinforcement learning. IET Intelligent Transport Systems 2017, 11(7): 417-423. DOI:10.1049/iet-its.2017.0153.

[8] Ma ZB, Cui TC, Deng WX, et al. Adaptive optimization of traffic signal timing via deep reinforcement learning. Journal of Advanced Transportation, 2021, 2021:6616702. DOI:10.1155/2021/6616702.

[9] Huang LB, Qu XH. Improving traffic signal control operations using proximal policy optimization. IET Intelligent Transport Systems, 2023, 17(3): 592-605. DOI:10.1049/itr2.12286.

[10] Chen Y, Chang Z, Min G, et al. Joint Optimization of Sensing and Computation for Status Update in Mobile Edge Computing Systems [J]. IEEE Transactions on Wireless Communications, 2023, 22(11): 8230-8243.

[11] Cheng Yaodong, Tian Runxin. A Deep Reinforcement Learning-Based Computation Offloading Strategy for Edge Computing [J]. Wireless Interconnection Technology, 2024, 21(13): 19-21.

[12] Deng RQ, Di BY, Zhang HL, et al. Reconfigurable Holographic Surface: Holographic Beamforming for Metasurface-Aided Wireless Communications [J]. IEEE Transactions on Vehicular Technology, 2021, 70(6): 6255-6259.

[13] Zhu YS, Mao BM, Kato N. Intelligent Reflecting Surface in 6G Vehicular Communications: A Survey [J]. IEEE Open Journal of Vehicular Technology, 2022, 3: 266-277.

[14] Gao, J., Shen, Y., Liu, J., lto, M., hiratori, N.: Adaptive traffic signal control deep reinforcement learning algorithm with experience replay and target network. arXiv: 1705.02755(2017).

[15] Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection. International Journal of Intelligent Transportation Systems Research (2023) 21:192–206 arXiv:2109.14337.