This paper proposes a task-based control optimization method for the robotic fish. It is essentially an adversarial structured control consisting of a global control module and a local compensation control module. In detail, the global control module emulates an optimized central pattern generator with Evolutionary Strategy, while the local control module produces targeted compensation control signals with Soft Actor-Critic. The linear summation of two control laws works for the final robotic fish control. Considering that the evolutionary computation optimization algorithms generally have the defect of falling into the local optimum, we propose a method of antagonistic training to improve the optimization performance. The effectiveness of the designed controller is validated by simulation on agents in Mujoco. Noticeably, the simulation results demonstrate that the proposed method teaches the agent fish to move to any target point with a low energy consumption, which lays a good foundation for application of reinforcement learning in real robotic fish control.