Due to hardware limitations, existing hyperspectral (HS) camera often suffer from low spatial/temporal resolution. Recently, it has been prevalent to super-resolve a low resolution (LR) HS image into a high resolution (HR) HS image with a HR RGB (or multispectral) image guidance. Previous approaches for this guided super-resolution task often model the intrinsic characteristic of the desired HR HS image using hand-crafted priors. Recently, researchers pay more attention to deep learning methods with direct supervised or unsupervised learning, which exploit deep prior only from training dataset or testing data. In this article, an efficient convolutional neural network-based method is presented to progressively super-resolve HS image with RGB image guidance. Specifically, a progressive HS image super-resolution network is proposed, which progressively super-resolve the LR HS image with pixel shuffled HR RGB image guidance. Then, the super-resolution network is progressively trained with supervised pre-training and unsupervised adaption, where supervised pre-training learns the general prior on training data and unsupervised adaptation generalises the general prior to specific prior for variant testing scenes. The proposed method can effectively exploit prior from training dataset and testing HS and RGB images with spectral-spatial constraint. It has a good generalisation capability, especially for blind HS image super-resolution. Comprehensive experimental results show that the proposed deep progressive learning method outperforms the existing state-of-the-art methods for HS image super-resolution in non-blind and blind cases.