Reinforcement learning based - sliding mode control for trajectory tracking of quadrotor unmanned aerial vehicles under disturbances
2 viewsDOI:
https://doi.org/10.54939/1859-1043.j.mst.101.2025.39-46Keywords:
Reinforcement learning; Sliding mode control; Optimal control; Actor/critic structure; Quadrotor unmanned aerial vehicle (QUAV).Abstract
In this article, a reinforcement learning (RL)-based sliding mode control (SMC) is proposed for trajectory tracking of a quadrotor unmanned aerial vehicle (QUAV) under external disturbances. First, an actor-critic RL framework sliding mode control is provided to tackle the optimal control problem without external disturbances. Secondly, the simulation in an environment with disturbances is carried out to show the robustness of the proposed controller. Theoretical analysis shows that the position and attitude tracking errors converge to a preset region, and the weight estimation errors of the actor-critic networks are uniformly ultimately bounded (UUB). Finally, a comparison of the numerical simulations between the proposed controller and traditional sliding mode controller and the Backstepping (BSP) technique is provided to indicate the advantages and improved performance of the RL-based SMC.
References
[1]. Mohsan, Syed Agha Hassnain, et al. "Unmanned aerial vehicles (UAVs): Practical aspects, applications, open challenges, security issues, and future trends." Intelligent Service Robotics, 16.1, 109-137, (2023). DOI: https://doi.org/10.1007/s11370-022-00452-4
[2]. Liu, Hui, et al. "Reinforcement learning‐based tracking control for a quadrotor unmanned aerial vehicle under external disturbances." International Journal of Robust and Nonlinear Control, 33.17, 10360-10377, (2023). DOI: https://doi.org/10.1002/rnc.6334
[3]. Li, Bo, et al. "Fixed-time integral sliding mode control of a high-order nonlinear system." Nonlinear Dynamics, 107, 909-920, (2022). DOI: https://doi.org/10.1007/s11071-021-06984-1
[4]. Qi, Wenhai, Guangdeng Zong, and Wei Xing Zheng. "Adaptive event-triggered SMC for stochastic switching systems with semi-Markov process and application to boost converter circuit model." IEEE Transactions on Circuits and Systems I: Regular Papers, 68.2, 786-796, (2020). DOI: https://doi.org/10.1109/TCSI.2020.3036847
[5]. Yong, Jiongmin, et al. "Dynamic programming and HJB equations." Stochastic controls: Hamiltonian systems and HJB equations, 157-215, (1999). DOI: https://doi.org/10.1007/978-1-4612-1466-3_4
[6]. Evans, L. C., and M. R. James. "The Hamiltonian–Jacobi–Bellman equation for time-optimal control." SIAM journal on control and optimization, 27.6, 1477-1489, (1989). DOI: https://doi.org/10.1137/0327076
[7]. Werbos, Paul. "Approximate dynamic programming for real-time control and neural modeling." Handbook of intelligent control, (1992).
[8]. Vamvoudakis, Kyriakos G., and Frank L. Lewis. "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem." Automatica, 46.5, 878-888, (2010). DOI: https://doi.org/10.1016/j.automatica.2010.02.018
[9]. Ma, Zhiqiang, Panfeng Huang, and Yuxin Lin. "Learning-based sliding-mode control for underactuated deployment of tethered space robot with limited input." IEEE Transactions on Aerospace and Electronic Systems, 58.3, 2026-2038, (2021). DOI: https://doi.org/10.1109/TAES.2021.3126569
[10]. Modares, Hamidreza, Mohammad-Bagher Naghibi Sistani, and Frank L. Lewis. "A policy iteration approach to online optimal control of continuous-time constrained-input systems." ISA transactions, 52.5, 611-621, (2013). DOI: https://doi.org/10.1016/j.isatra.2013.04.004
[11]. Tan, Jian, and Shijun Guo. "Backstepping control with fixed-time prescribed performance for fixed wing UAV under model uncertainties and external disturbances." International Journal of Control, 95.4, 934-951, (2022). DOI: https://doi.org/10.1080/00207179.2020.1831700
[12]. Wen, Guoxing, et al. "Optimized backstepping tracking control using reinforcement learning for quadrotor unmanned aerial vehicle system." IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52.8, 5004-5015, (2021). DOI: https://doi.org/10.1109/TSMC.2021.3112688
[13]. Xu, Shihao, et al. "Reinforcement-learning-based tracking control with fixed-time prescribed performance for reusable launch vehicle under input constraints." Applied Sciences, 12.15, 7436, (2022). DOI: https://doi.org/10.3390/app12157436
[14]. Nguyen, H. S., et al. "Advanced Motion Control of a Quadrotor Unmanned Aerial Vehicle based on Extended State Observer." International Conference on System Science and Engineering. IEEE, (2023). DOI: https://doi.org/10.1109/ICSSE58758.2023.10227212
[15]. Liu, K., Wang, R., Zheng, S., Dong, S., & Sun, G. "Fixed-time disturbance observer-based robust fault-tolerant tracking control for uncertain quadrotor UAV subject to input delay". Nonlinear Dynamics, 107(3), 2363-2390, (2022). DOI: https://doi.org/10.1007/s11071-021-07080-0
[16]. Chen, Fuyang, et al. "Robust backstepping sliding-mode control and observer-based fault estimation for a quadrotor UAV." IEEE Transactions on Industrial Electronics, 63.8, 5044-5056, (2016). DOI: https://doi.org/10.1109/TIE.2016.2552151
[17]. Tan, Lingwei, et al. "Super-twisting sliding mode control with defined boundary layer for chattering reduction of permanent magnet linear synchronous motor." Journal of Mechanical Science and Technology, 35: 1829-1840, (2021). DOI: https://doi.org/10.1007/s12206-021-0403-9
