To improve their adaptability to various kinds of driving situations, deep learning-based vision algorithms need images from rare scenes, such as extreme weather conditions and traffic congestions. However, most datasets collected from physical driving environments are lack of such images, making vision models trained on these datasets do not work well in scarce scenes. Thus, we design an SST-GAN method for controllably generating realistic images of scarce driving scenes based on the framework of parallel vision. Trained on only a single sample, SST-GAN can produce hundreds of rare scene images from two directions: style transfer and content generation. Specifically, a transition retraining method is designed to transfer the weather and lighting styles from common scenes to scarce scenes, and a structural similarity index loss is used as reconstruction loss to guarantee the trained network can obtain more realistic content modification and generation during the image reconstruction. Experimental results show that SST-GAN outperforms the state-of-the-art method on expanding the amount of scarce scene images from both style and content. The method is highly adaptable and works flexibly on handling image generation problems for various types of rare scenes.