In this paper, the pursuit-evasion game with state and control constraints
is solved to achieve the Nash equilibrium of both the pursuer and the evader with an iterative self-play technique. Under the condition where the Hamiltonian formed by means of Pontryagin’s Maximum Principle has the unique solution, it can be proved that the iterative control law is convergent to the Nash equilibrium solution. However, the strong nonlinearity of the ordinary differential equations formulated by Pontryagin's Maximum Principle makes the control policy hard to figured
out. Moreover the system dynamics employed in this manuscript contains a high dimensional state vector with constraints. To address the challenges, the optimal control problems are transformed into nonlinear programming problems through direct collocation method. Finally, two numerical cases of aircraft pursuit-evasion scenario are given and analyzed to demonstrate the effectiveness of the presented method.