Distributed machine learning over peer-to-peer network has become popular in the past few years due to the growing demand for privacy protection. Recent peer-to-peer distributed K-means algorithm can achieve the same performance as centralized K-means, but they also has high sensitivity to initialization as centralized K-means, which worsens its performance for clustering. In this paper, we first proposes a distributed bisecting K-means algorithm over a peer-to-peer network to alleviate this drawback by combining bisecting K-means with Metropolis algorithm, since the previous works showed that bisecting K-means is much less sensitive to initialization than traditional K-means. It is shown by extensive simulations that our algorithm has the same performance with centralized bisecting K-means and outperforms the existing peer-to-peer distributed K-means.