Recognizing a person across different non-overlapping camera views, is the task of person re-identification. For achieving the task, an effective way is to learn a discriminative metric by minimizing the within-class variance and maximizing the between-class variance simultaneously. However, the dimension of sample feature vector is usually greater than the number of training samples, as a result, the within-class scatter matrix is singular and the metric cannot be learned. In this paper, we propose to solve the singularity problem by employing the pseudo-inverse of the within-class scatter matrix and learning an orthogonal transformation for the metric. The proposed method can be effectively solved with a closed-form solution and no parameters required to tune. In addition, we develop a kernel version against non-linearity in person re-identification, and a fast version for more efficient solution. In experiments, we prove the validity and advantage of the proposed method for solving the singularity problem in person re-identification, and analyze the effectiveness of both kernel version and fast version. Extensively comparative experiments on VIPeR, PRID2011, CUHK01 and CUHK03 person re-identification benchmark datasets, show the state-of-the-art results of the proposed method.