Quantitative estimation of a driver's vigilance level has a great value for improving driving safety and preventing accidents. Previous studies have identified correlations between electroencephalogram (EEG) spectrum power and a driver's mental states such as vigilance and alertness. Studies have also built classification models that can estimate vigilance state changes based on data collected from drivers. In the present study, we propose a system to detect vigilance level using not only a driver's EEG signals but also driving contexts as inputs. We combined a support vector machine with particle swarm optimization methods to improve classification accuracy. A simulated driving task was conducted to demonstrate the reliability of the proposed system. Twenty participants were assigned a 2-h sustained-attention driving task to identify a lead car's brake events. Our system was able to account for 84.1% of experimental reaction times with 162-ms prediction errors. A newly introduced driving context factor, road curves, improved the prediction accuracy by 2-5% with 30-80 ms smaller errors. These findings demonstrated the potential value of the proposed system for estimating driver vigilance level on a time scale of seconds.