In this paper, a novel robust cooperative tracking control algorithm is proposed for nonlinear multi-agent graphical games with disturbances based on adaptive dynamic programming approach. The robust cooperative tracking control policy is obtained by multiplying the coupling gains to the interactive Nash solution of nominal multi-agent systems. The consensus error dynamics and cost function for each node depend only on the information of itself and its neighbors. Through Lyapunov approach, the robust stability conditions are derived to guarantee all the followers synchronize to the leader. A cooperative policy iteration algorithm is utilized to approximatly solve the coupled Hamilton-Jacobi-Isaacs equations, and only critic neural network is employed to approximate the value function and control policy for each node. A novel network weight tuning laws is proposed to guarantee uniform ultimate boundedness of closed-loop systems. Simulation results are utilized to demonstrate the effectiveness of the theoretical results.