(1) Background: Epilepsy is a neurological disorder that causes repeated seizures. Since electroencephalogram (EEG) patterns differ in different states (inter-ictal, pre-ictal, and ictal), a seizure can be detected and predicted by extracting various features. However, the brain connectivity network, a two-dimensional feature, is rarely studied. We aim to investigate its effectiveness for seizure detection and prediction. (2) Methods: Two time-window lengths, five frequency bands, and five connectivity measures were used to extract image-like features, which were fed into a support vector machine for the subject-specific model (SSM) and a convolutional neural networks meet transformers (CMT) classifier for the subject-independent model (SIM) and cross-subject model (CSM). Finally, feature selection and efficiency analyses were conducted. (3) Results: The classification results on the CHB-MIT dataset showed that a long window indicated better performance. The best detection accuracies of SSM, SIM, and CSM were 100.00, 99.98, and 99.27%, respectively. The highest prediction accuracies were 99.72, 99.38, and 86.17%, respectively. In addition, Pearson Correlation Coefficient and Phase Lock Value connectivity in the beta and gamma bands showed good performance and high efficiency. (4) Conclusions: The proposed brain connectivity features showed good reliability and practical value for automatic seizure detection and prediction, which expects to develop portable real-time monitoring equipment.