-
摘要: 面向離散時間非線性動態系統，提出一種基于事件的迭代神經控制框架。主要目標是將迭代自適應評判方法與事件驅動機制結合起來，以解決離散時間非線性系統的近似最優調節問題。首先，構造兩個迭代序列并建立一種事件觸發的值學習策略。其次，詳細給出迭代算法的收斂性分析和新型框架的神經網絡實現。這里是在基于事件的迭代環境下實施啟發式動態規劃技術。此外，通過設計適當的閾值以確定事件驅動方法的觸發條件。最后，借助兩個仿真實例驗證本文控制方案的優越性能，尤其是在通信資源的利用方面。本文的工作有助于構建一類事件驅動機制下的智能控制系統.Abstract: With the widespread popularity of network-based techniques and extension of computer control scales, more dynamical systems, particularly complex nonlinear dynamics, including increasing communication burdens, increasing difficulties in building accurate mathematical models, and different uncertain factors are encountered. Consequently, in contrast to the linear case, the optimization of the design of these uncertain complex systems is difficult to achieve. By combining reinforcement learning, neural networks, and dynamic programming, the adaptive critic method is regarded as an advanced approach to address intelligent control problems. The adaptive critic method has been currently used to solve the optimal regulation, trajectory tracking, robust control, disturbance attenuation, and zero-sum game problems. It has been considered a promising direction within the artificial intelligence field. However, many traditional design processes of the adaptive critic method are conducted based on the time-based mechanism, where the control signals are updated at each time step. Thus, the related control efficiencies are often low, which results in poor performance when considering practical updating times. Hence, more improvements are needed to enhance the control efficiency of adaptive-critic-based nonlinear control design. In this study, we developed an event-based iterative neural control framework for discrete-time nonlinear dynamics. The iterative adaptive critic method was combined with the event-driven mechanism to address the approximate optimal regulation problem in discrete-time nonlinear plants. An event-triggered value learning strategy was established with two iterative sequences. The convergence analysis of the iterative algorithm and the neural network implementation of the new framework were presented in detail. Therein, the heuristic dynamic programming technique was employed under the event-based iterative environment. Moreover, the triggering condition of the event-driven approach was determined with the appropriate threshold. Finally, simulation examples were provided to illustrate the excellent control performance, particularly in utilizing the communication resource. Thus, constructing a class of intelligent control systems based on the event-based mechanism will be helpful.
-
圖 1 離散動態系統基于事件的迭代HDP框架簡圖
Figure 1. Simple diagram of the event-based iterative heuristic dynamic programming (HDP) framework with discrete dynamic plants
中文字幕在线观看
圖 2 執行迭代HDP算法之后的事件驅動控制實現過程
Figure 2. Event-based control implementation process after conducting the iterative HDP algorithm
-
參考文獻
[1] Werbos P J. Approximate dynamic programming for real-time control and neural modeling. In White D A and Sofge D A (Eds. ) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches. New York, NY: Van Nostrand Reinhold, 1992 [2] Li J N, Chai T Y, Lewis F L, et al. Off-policy interleaved Q-learning: Optimal control for affine nonlinear discrete-time systems. IEEE Trans Neural Netw Learn Syst, 2019, 30(5): 1308 doi: 10.1109/TNNLS.2018.2861945 [3] Zhang H G, Liu Y, Xiao G Y, et al. Data-based adaptive dynamic programming for a class of discrete-time systems with multiple delays. IEEE Trans Syst Man Cybern:Syst, 2020, 50(2): 432 doi: 10.1109/TSMC.2017.2758849 [4] Zhang H G, Jiang H, Luo Y H, et al. Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans on Ind Electron, 2017, 64(5): 4091 doi: 10.1109/TIE.2016.2542134 [5] Ha M M, Wang D, Liu D R. Generalized value iteration for discounted optimal control with stability analysis. Syst Control Lett, 2021, 147: 104847 doi: 10.1016/j.sysconle.2020.104847 [6] Wang D, Ha M M, Qiao J F. Data-driven iterative adaptive critic control towards an urban wastewater treatment plant. IEEE Trans Ind Electron, 2021, 68(8): 7362 doi: 10.1109/TIE.2020.3001840 [7] Wang D, Ha M M, Qiao J F, et al. Data-based composite control design with critic intelligence for a wastewater treatment platform. Artif Intell Rev, 2020, 53(5): 3773 doi: 10.1007/s10462-019-09778-5 [8] Liang M M, Wang D, Liu D R. Improved value iteration for neural-network-based stochastic optimal control design. Neural Netw, 2020, 124: 280 doi: 10.1016/j.neunet.2020.01.004 [9] Liang M M, Wang D, Liu D R. Neuro-optimal control for discrete stochastic processes via a novel policy iteration algorithm. IEEE Trans Syst Man Cybern:Syst, 2020, 50(11): 3972 doi: 10.1109/TSMC.2019.2907991 [10] Hou J X, Wang D, Liu D R, et al. Model-free H∞ optimal tracking control of constrained nonlinear systems via an iterative adaptive learning algorithm. IEEE Trans Syst Man Cybern:Syst, 2020, 50(11): 4097 doi: 10.1109/TSMC.2018.2863708 [11] Luo B, Liu D R, Huang T W, et al. Model-free optimal tracking control via critic-only Q-learning. IEEE Trans Neural Netw Learn Syst, 2016, 27(10): 2134 doi: 10.1109/TNNLS.2016.2585520 [12] Al-Tamimi A, Lewis F L, Abu-Khalaf M. Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans Syst Man Cybern B:Cybern, 2008, 38(4): 943 doi: 10.1109/TSMCB.2008.926614 [13] Zhang H G, Luo Y H, Liu D R. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw, 2009, 20(9): 1490 doi: 10.1109/TNN.2009.2027233 [14] Wang D, Liu D R, Wei Q L, et al. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica, 2012, 48(8): 1825 doi: 10.1016/j.automatica.2012.05.049 [15] Zhong X, Ni Z, He H. A theoretical foundation of goal representation heuristic dynamic programming. IEEE Trans Neural Netw Learn Syst, 2016, 27(12): 2513 doi: 10.1109/TNNLS.2015.2490698 [16] Yang X, Liu D R, Wang D, et al. Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning. Neural Netw, 2014, 55: 30 doi: 10.1016/j.neunet.2014.03.008 [17] Tabuada P. Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans Autom Control, 2007, 52(9): 1680 doi: 10.1109/TAC.2007.904277 [18] Fan Q Y, Yang G H. Event-based fuzzy adaptive fault-tolerant control for a class of nonlinear systems. IEEE Trans Fuzzy Syst, 2018, 26(5): 2686 doi: 10.1109/TFUZZ.2018.2800724 [19] Zhou Y, Zeng Z. Event-triggered impulsive control on quasi-synchronization of memristive neural networks with time-varying delays. Neural Netw, 2019, 110: 55 doi: 10.1016/j.neunet.2018.09.014 [20] Wang D, Zhong X N. Advanced policy learning near-optimal regulation. IEEE/CAA J Autom Sin, 2019, 6(3): 743 doi: 10.1109/JAS.2019.1911489 [21] Wang D. Research progress on learning-based robust adaptive critic control. Acta Autom Sin, 2019, 45(6): 1031王鼎. 基于學習的魯棒自適應評判控制研究進展. 自動化學報, 2019, 45(6):1031 [22] Zhang H G, Su H G, Zhang K, et al. Event-triggered adaptive dynamic programming for non-zero-sum games of unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Trans Fuzzy Syst, 2019, 27(11): 2202 doi: 10.1109/TFUZZ.2019.2896544 [23] Eqtami A, Dimarogonas D V, Kyriakopoulos K J. Event-triggered control for discrete-time systems // Proceedings of the 2010 American Control Conference, Baltimore, 2010: 4719 [24] Dong L, Zhong X N, Sun C Y, et al. Adaptive event-triggered control based on heuristic dynamic programming for nonlinear discrete-time systems. IEEE Trans Neural Netw Learn Syst, 2017, 28(7): 1594 doi: 10.1109/TNNLS.2016.2541020 [25] Ha M M, Wang D, Liu D R. Event-triggered adaptive critic control design for discrete-time constrained nonlinear systems. IEEE Trans Syst Man Cybern:Syst, 2020, 50(9): 3158 doi: 10.1109/TSMC.2018.2868510 [26] Dhar N K, Verma N K, Behera L. Adaptive critic-based event-triggered control for HVAC system. IEEE Trans Ind Inform, 2018, 14(1): 178 doi: 10.1109/TII.2017.2725899 -
下載:
點擊查看大圖
圖(12)
計量
- 文章訪問數: 856
- HTML全文瀏覽量: 404
- PDF下載量: 73
- 被引次數: 0
