The dominant approach for learning keypoint detectors relies on the covariance constraint. However, existing learned detectors sometimes extract unstable keypoints from edges.
To solve this problem, we propose a novel method that exploits local saliency knowledge to train a keypoint detector, and obtain a keypoint detector, called as SRK-Net, which can extract stable and repeatable keypoints.
Firstly, given an image, we propose a General Local Saliency Measure method (GLSM) to assess the local saliency value for each pixel and generate a local saliency map for this image.
Then we propose a Local Salient Structure Maintaining loss (LSSM) and a two-stage progressive training manner tailored for leveraging the supervision of the covariance constraint and the local saliency maps provided by our GLSM.
Experimental results show that the proposed SRK-Net performs better than all the existing keypoint detectors on HPatches dataset.