Musculoskeletal robots with human-like joints, muscles, and actuation mechanisms are characterized by exceptional dexterity, compliance, and versatility. However, existing reinforcement learning methods for such robots rely on precise and sufficient state observation, rendering them vulnerable to perturbations. To address this limitation, this paper proposes a robust motion learning method based on a recurrent neural network (RNN) and muscle synergy. First, the proposed method utilizes task-joint-muscle space states to create an RNN-based neuromuscular controller. Furthermore, a motion learning method with a synergistic constraint of muscles is developed. Additionally, theoretical analysis confirms that the RNN-based controller is more robust to perturbations of state observation than a Multilayer Perceptron (MLP) based controller. The proposed method is evaluated on a simulated musculoskeletal robot and demonstrates superior robustness to other MLP-based reinforcement learning methods. Furthermore, the proposed method is also validated on a musculoskeletal robot hardware system, indicating its potential for real-world applications.