We propose a new low-rank based image recovery method and embed it into an existing Groupwise Image Registration (GIR) framework to achieve accurate GIR of Magnetic Resonance (MR) brain images containing tumors. In our method, brain tumor regions in the input images are recovered with population-consistent normal brain appearance to produce low-rank images. The GIR framework is then applied to the tumor-free low-rank images. With no influence from the brain tumor, accurate GIR can be achieved. Unlike conventional low-rank based image recovery methods, a spatial constraint is added to the low-rank framework in our method, by which the quality of the resulting low-rank images can be improved. Particularly, the low-rank images produced by our method contain both effectively recovered brain tumor regions and well-preserved normal brain regions of input images, which are two key factors for accurate GIR. By contrast, in conventional low-rank based image recovery methods, these two factors are mutually exclusive and a good balance is difficult to achieve. Synthetic and real MR brain images are used to evaluate our method. The results show that based on our method, image recovery quality and GIR accuracy are improved in comparison to the state-of-the-art method.