It’s important to recognize facial expressions in social communication. To solve the problem that facial expression recognition by visible light is vulnerable to interference, we built a model from the perspective of thermal infrared. Based on the distribution characteristics of thermal infrared, the face region is firstly located by building a multi-projection model toward color. Then, the level set function of the local Gaussian fitting model was optimized, the regular term was removed, and the larger iteration step size was selected to achieve accurate face segmentation on the premise of segmentation accuracy. Finally, based on the structure of traditional deep learning network, the characteristics of DPN and CBAM network are given full play to realize expression recognition by thermal infrared images. © 2021, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.