Automatic high precision assembly of millimeter sized objects is a challenging task. Traditional methods for precision assembly require explicit programming with real robot system, which require complex parameter-tuning work. In this paper we achieve reinforcement learning of precision insertion skill based on prioritized dueling deep Q-network (DQN). The Q-function is represented by the long short term memory (LSTM) neural network, whose input is 6D force-torque feedback. According to the Q values conditioned on the current state, the skill model select a 6 degree-of-freedom action from the predefined action set. To accelerate the learning process, the data from demonstrations is used to pre-train the model before the DQN starts. In order to improve the insertion efficiency and safety, insertion step length is modulated based on the instant reward. Our proposed method is validated with the peg-in-hole insertion experiments on a precision assembly robot. In addition, the reusability of the skill model in different types of insertion tasks is also investigated.