Signature veriffcation is a biometric and document forensics technology useful for personal identiffcation in various security applications. Signature veriffcation in the writer-independent scenario remains a challenge, particularly in distinguishing between genuine signatures and skilled forgeries. In this paper, we propose a writer-independent signature veriffcation method based on deep metric learning with cross-writer attention. Our cross-writer attention module includes two parts: SimAM (a Simple, Parameter-Free Attention Module), as well as the cross-attention mechanism. SimAM is combined with each DenseBlock to interact information of two inputs, which makes the learned weights better account for the difference between two input signatures. Cross-attention aligns global and local information in learned feature representations of two input signatures. Further, we introduce a focal contrast loss function for deep metric learning to overcome the sample imbalance. Extensive experiments demonstrate the effectiveness of the proposed method, which achieves superior performance on several public datasets and also indicates the effectiveness of each module.