The Current Status and Prospects of Reinforcement Learning in the Field of Automation Control

Yiwei Zhang

doi:10.54097/w7ptd039

Authors

Yiwei Zhang Viterbi School of Engineering, University of Southern California, Los Angeles, 90089, USA

DOI:

https://doi.org/10.54097/w7ptd039

Keywords:

Reinforcement learning, Automated control, Application status, Technical bottlenecks, Development Prospects

Abstract

The implementation of Industry 4.0 and Intelligent Manufacturing 2035 plan promotes the transformation of the automation control field from traditional fixed strategies to autonomous and adaptive control. Reinforcement learning relies on the core advantages of trial and error learning and online adaptation to address the control challenges of nonlinear and uncertain systems, becoming a key support for technological upgrades in this field. This article combines authoritative industry data and practical application cases to systematically summarize its application status, analyze the adaptability, application scenarios, technological breakthroughs, and existing bottlenecks of automation control, explore the constraints of large-scale applications, and look forward to development trends. Research has found that reinforcement learning has been implemented in control scenarios such as industrial processes, robots, and intelligent energy. However, issues such as sample efficiency, robustness, and implementation costs still constrain its large-scale promotion. In the future, with the deep integration of algorithm optimization and industry, it will promote the development of automation control towards higher levels of intelligence.

Downloads

Download data is not yet available.

References

[1] Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2017). Optimal and autonomous control using reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2042–2062. https://doi.org/10.1109/TNNLS.2017.2673554

[2] Shah, H., & Gopal, M. (2016). Model-free predictive control of nonlinear processes based on reinforcement learning. IFAC-PapersOnLine, 49(1), 89–94. https://doi.org/10.1016/j.ifacol.2016.04.016

[3] Powell, K. M., Machalek, D., & Quah, T. (2020). Real-time optimization using reinforcement learning. Computers & Chemical Engineering, 143, 107077. https://doi.org/10.1016/j.compchemeng.2020.107077

[4] Pane, Y. P., Nageshrao, S. P., Kober, J., & Babuška, R. (2019). Reinforcement learning based compensation methods for robot manipulators. Engineering Applications of Artificial Intelligence, 78, 236–247. https://doi.org/10.1016/j.engappai.2018.11.004

[5] Xia, K., Sacco, C., Kirkpatrick, M., Saidy, C., Nguyen, L., Kircaliali, A., & Harik, R. (2021). A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence. Journal of Manufacturing Systems, 58, 210–230. https://doi.org/10.1016/j.jmsy.2020.12.012

[6] Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2021). Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110(9), 2419–2468. https://doi.org/10.1007/s10994-021-05961-4

[7] Gheisarnejad, M., & Khooban, M. H. (2020). An intelligent non-integer PID controller-based deep reinforcement learning: Implementation and experimental results. IEEE Transactions on Industrial Electronics, 68(4), 3609–3618. https://doi.org/10.1109/TIE.2020.2984443

[8] Lwakatare, L. E., Raj, A., Crnkovic, I., Bosch, J., & Olsson, H. H. (2020). Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions. Information and Software Technology, 127, 106368. https://doi.org/10.1016/j.infsof.2020.106368

[9] Grosse, K., Bieringer, L., Besold, T. R., Biggio, B., & Krombholz, K. (2023). Machine learning security in industry: A quantitative survey. IEEE Transactions on Information Forensics and Security, 18, 1749–1762. https://doi.org/10.1109/TIFS.2023.3241227

[10] Chen, X., Hu, J., Chen, Z., Lin, B., Xiong, N., & Min, G. (2021). A reinforcement learning-empowered feedback control system for industrial internet of things. IEEE Transactions on Industrial Informatics, 18(4), 2724–2733. https://doi.org/10.1109/TII.2021.3098153

[11] Iyelolu, T. V., Agu, E. E., Idemudia, C., & Ijomah, T. I. (2024). Driving SME innovation with AI solutions: overcoming adoption barriers and future growth opportunities. International Journal of Science and Technology Research Archive, 7(1), 036–054.