In this study, we introduce an innovative gait learning methodology for three-dimensional bipedal robots, integrating Hybrid Zero Dynamics (HZD) priors with periodic reward functions to enhance gait stability, symmetry, and smooth action transitions. Notably, we employ a data-driven Bezier curve parameterization technique optimized through Reinforcement Learning (RL) to significantly improve learning efficiency and dynamic stability of the gait. The effectiveness of our approach is systematically validated across three key metrics: lateral deviation, training speed, and robustness in challenging environments. The results demonstrate our method's superiority in maintaining path stability, accelerating the learning process, and adapting to complex terrains over traditional gait learning approaches. This research not only advances the efficiency and stability of gait learning for three-dimensional bipedal robots but also provides valuable insights for future studies on robotic gait optimization.