accurately distinguishing aerial photographs from different categories is a promising technique in computer vision. it can facilitate a series of applications, such as video surveillance and vehicle navigation. in this paper, a new image kernel is proposed for effectively recognizing aerial photographs. the key is to encode high-level semantic cues into local image patches in a weakly supervised way, and integrate multimodal visual features using a newly developed hashing algorithm. the flowchart can be elaborated as follows. given an aerial photo, we first extract a number of graphlets to describe its topological structure. for each graphlet, we utilize color and texture to capture its appearance, and a weakly supervised algorithm to capture its semantics. thereafter, aerial photo categorization can be naturally formulated as graphlet-to-graphlet matching. as the number of graphlets from each aerial photo is huge, to accelerate matching, we present a hashing algorithm to seamlessly fuze the multiple visual features into binary codes. finally, an image kernel is calculated by fast matching the binary codes corresponding to each graphlet. and a multi-class svm is learned for aerial photo categorization. we demonstrate the advantage of our proposed model by comparing it with state-of-the-art image descriptors. moreover, an in-depth study of the descriptiveness of the hash-based graphlet is presented.