This paper introduces an optimized control approach for bipedal robots, merging Bayesian optimization for reward function weights and a novel neural network structure combining LSTM and Transformer-based attention. Bayesian optimization enhances training stability and efficiency, while the hybrid network captures temporal patterns and long-range dependencies, outperforming traditional architectures in reward stability and performance. Simulated evaluations show our model's superior robustness against challenges like varying ground friction and external disturbances. Future work will focus on domain randomization and real-world robot fine-tuning to bridge the simulation-reality gap.