In this paper, we introduce a simplified end-to-end framework for impurity detection in opaque glass bottles with liquor that learns to directly distinguish between small impurities and backgrounds. Despite promising results using convolutional neural networks in various vision tasks, few works have provided specific solutions under inadequate exposures and large background fluctuations. Two contributions are made for this problem. Firstly, we have built a feasible detection system with a cascade hardware structure, and each FPGA provides a host computer with 12 images which are most confident for containing potential impurities respectively. Secondly, most previous convolutional network architectures generally work in large-scale notable object detection benchmarks, however, such networks cannot transfer well when detecting small objects in gray images. Therefore, we propose a superpixel proposal generation method for image augmentation and a fast convolutional network with an overlapped grid structure to detect small impurities, and experiments show that our binary detection results are comparable with human checkers.