Classification is a hot topic in hyperspectral remote sensing community. In the last decades, numerous effort has been concentrate on the classification problem. However, most of the methods accuracy is not high enough due to the fact that they do not extract features in a deep manner. In this paper, a new hyperspectral data classification skeleton based on exponential flexible momentum deep convolution neural network (EFM-CNN) is proposed. First, the fitness of convolution neural network is substantiated by following classical spectral information-based classification. Then, a novel deep architecture is proposed, which is a hybrid of principle component analysis (PCA), improved convolution neural network based on exponential flexible momentum and support vector machine (SVM). Experimental results indicate that the classifier can effectively improve the accuracy with the state-of-the-art algorithms. And compared with homologous parameters momentum updating methods such as adaptive momentum method, standard momentum gradient method and elastic momentum method, on LeNet5 net and multiple neural network, the accuracy obtained of proposed algorithm increases by 2.6% and 6.5% on average respectively.