Infectious diseases have always been a major threat to the survival of humanity. Additionally, they bring an enormous economic burden to society. The conventional methods for bacteria identification are expensive, time-consuming and laborious. Therefore, it is of great importance to automatically rapidly identify pathogenic bacteria in a short time. Here, we constructed an AI-assisted system for automating rapid bacteria genus identification, combining the hyperspectral microscopic technology and a deep-learning-based algorithm Buffer Net. After being trained and validated in the self-built dataset, which consists of 11 genera with over 130,000 hyperspectral images, the accuracy of the algorithm could achieve 94.9%, which outperformed 1D-CNN, 2D-CNN and 3D-ResNet. The AI-assisted system we developed has great potential in assisting clinicians in identifying pathogenic bacteria at the single-cell level with high accuracy in a cheap, rapid and automatic way. Since the AI-assisted system can identify the pathogenic genus rapidly (about 30 s per hyperspectral microscopic image) at the single-cell level, it can shorten the time or even eliminate the demand for cultivating. Additionally, the system is user-friendly for novices.