随着传感、信号处理和通信技术的快速发展,关于网络控制系统(Networked control systems,NCSs)的研究引起了极大的关注.本文拟回顾关于网络控制系统的最新研究进展.为揭示反馈通信网络对控制系统的影响,主要讨论了为满足不同控制目的所需的各种网络条件,例如:在时变信道的环境下,保证线性系统可镇定性所需的最低编码率;在间断观测的环境下,保证卡尔曼滤波器稳定性的临界丢包条件;在时不变连接图的环境下,达到线性多自主体系统趋同性所需的网络拓扑结构;在通信资源有限的情况下,基于事件驱动的采样与控制方法等.
This paper introduces a model-free reinforcement learning technique that is used to solve a class of dynamic games known as dynamic graphical games. The graphical game results from to make all the agents synchronize to the state of a command multi-agent dynamical systems, where pinning control is used generator or a leader agent. Novel coupled Bellman equations and Hamiltonian functions are developed for the dynamic graphical games. The Hamiltonian mechanics are used to derive the necessary conditions for optimality. The solution for the dynamic graphical game is given in terms of the solution to a set of coupled Hamilton-Jacobi-Bellman equations developed herein. Nash equilibrium solution for the graphical game is given in terms of the solution to the underlying coupled Hamilton-Jacobi-Bellman equations. An online model-free policy iteration algorithm is developed to learn the Nash solution for the dynamic graphical game. This algorithm does not require any knowledge of the agents＇ dynamics. A proof of convergence for this multi-agent learning algorithm is given under mild assumption about the inter-connectivity properties of the graph. A gradient descent technique with critic network structures is used to implement the policy iteration algorithm to solve the graphical game online in real-time.
Mohammed I.ABOUHEAFFrank L.LEWISMagdi S.MAHMOUDDariusz G.MIKULSKI
