Font recognition is an important part in the field of painting and calligraphy style recognition. Traditional font classification methods are mainly based on texture feature extraction and other methods, which need to be improved in classification accuracy. The mainstream classification methods mainly use convolutional neural networks, but such methods have poor interpretability and may face the problem that some detailed features cannot be accurately extracted. Based on convolutional neural network, the gray-level images, Local Binary Pattern (LBP) feature and Histogram of Oriented Gradient (HOG) of the images in the font dataset are respectively trained. Finally, the results of the three networks are fused by means of average decision fusion. The experimental results of font recognition show that the proposed method can extract the detailed features of fonts more accurately and obtain higher classification accuracy.