Semantic 3D models have shown their importance in many fields such as autonomous driving. However, it remains a tough task to assign semantic labels to various scenes. In this paper, we propose an Active Learning based method for semantic labeling of street view point clouds with a small amount of annotated data samples. The proposed method takes a point cloud and registrated images as the input, and yields a point cloud with semantic labels. We iteratively fine-tunes a network with the ever-enlarging training set to exploit the semantic information of the scene, and fuse the semantic labels in 3D space. To deal with the imbalanced data in street view scenes, a label biased criterion for query selection is proposed to help select images to efficiently improve the performance of the network and the quality of the semantic model. Experimental result shows that the proposed method demands limited human labor and works well in assigning semantic labels to the imbalanced scenes like street view scenes.