As a popular control rhythm of the multi-joint robotic fish, Center Pattern Generators (CPGs) plays an important role for motion performance. However, its optimal parameters are tough to seek through traditional methods. In order to address this problem, we propose an online optimization method for CPG parameters, including a novel normalized CPGs (N-CPGs) and a learning-based optimization algorithm. Via N-CPGs, the network parameters can be fully decoupled, which provides a great convenience for model parameter optimization. In particular, by applying the established dynamic model of the robotic fish, we use the deep Q network (DQN) to optimize the N-CPGs, aiming at improving the speed performance. Finally, extensive simulation results verify the effectiveness of proposed method, laying a solid foundation for real-time online control optimization of versatile motion modes in complex application scenarios.