Modern advancements in deep learning provide a powerful framework for disease classification based on neuroimaging data. However, interpreting the classification decision of convolutional neural network remains a challenging task. It is crucial to track the attention of neural network and provide valuable information about which brain areas are particularly related to the diagnosis of disease. In this paper, we propose a novel attention-based 3D ResNet architecture to diagnose explore potential biological markers. Experiments are conducted on 532 subjects (227 of patients with AD and 305 of normal controls). By introducing the attention mechanism, the proposed approach further improves the classification performance and identifies important brain regions for AD classification simultaneously. The experiments also show that significant brain regions for AD diagnosis captured by our attention-based network are accompanied by significant changes in gray matter.