Deep learning techniques have been widely adopted for learning task-adaptive features in image segmentation applications, such as brain tumor segmentation. However, most of existing brain tumor segmentation methods based on deep learning are not able to ensure appearance and spatial consistency of segmentation results. In this study we propose a novel brain tumor segmentation method by integrating a Fully Convolutional Neural Network (FCNN) and Conditional Random Fields (CRF), rather than adopting CRF as a post-processing step of the FCNN. We trained our network in three stages based on image patches and slices respectively. We evaluated our method on BRATS 2013 dataset, obtaining the second position on its Challenge dataset and first position on its Leaderboard dataset. Compared with other top ranking methods, our method could achieve competitive performance with only three imaging modalities (Flair, T1c, T2), rather than four (Flair, T1, T1c, T2), which could reduce the cost of data acquisition and storage. Besides, our method could segment brain images slice-by-slice, much faster than the methods patch-by-patch. We also took part in BRATS 2016 and got satisfactory results. As the testing cases in BRATS 2016 are more challenging, we added a manual intervention post-processing system during our participation.