A graph-based multiple view line matching method is proposed based on results of multiple view stereo (MVS) algorithms. With the 3D points and their visibility information provided by MVS, point-line correspondences are firstly established through 3D-to-2D re-projection. Each image line detected in different views is described using a 3D point set as well as a unit vector representing its coarse 3D direction. From such a description, pairwise similarity and consistency are evaluated. Then, a graph is constructed to contain all image lines as nodes. To get a unified node distance measure, a spectral graph analysis method is employed. Finally, a modified DBSCAN algorithm is introduced to obtain reliable line matches from the graph. Experiments show that our method is more robust and exhibits better accuracy than the existing methods.