Epilepsy is a prevalent chronic disorder of the central nervous system. The timely and accurate seizure prediction using the scalp Electroencephalography (EEG) signal can make patients adopt reasonable preventive measures before seizures occur and thus reduce harm to patients. In recent years, deep learning-based methods have made significant progress in solving the problem of epileptic seizure prediction. However, most current methods mainly focus on modeling short- or long-term dependence in EEG, while neglecting to consider both. In this study, we propose a Parallel Dual-Branch Fusion Network (PDBFusNet) which aims to combine the complementary advantages of Convolutional Neural Network (CNN) and Transformer. Specifically, the features of the EEG signal are first extracted using Mel Frequency Cepstral Coefficients (MFCC). Then, the extracted features are delivered into the parallel dual-branches to simultaneously capture the short- and long-term dependencies of EEG signal. Further, regarding the Transformer branch, a novel feature fusion module is developed to enhance the ability of utilizing time, frequency, and channel information. To evaluate our proposal, we perform sufficient experiments on the public epileptic EEG dataset CHB-MIT, where the accuracy, sensitivity, specificity and precision are 95.76%, 95.81%, 95.71% and 95.71%, respectively. PDBFusNet shows superior performance compared to state-of-the-art competitors, which confirms the effectiveness of our proposal.