Abstract—Automatic high precision assembly of millimeter sized objects is a challenging task. Traditional methods for precision assembly rely on explicit programming with real robot system, and require complex parameter-tuning work. In this paper, we realize deep reinforcement learning of precision insertion skill learning, based on prioritized dueling deep Q-network (DQN). The Q-function is represented by the long short term memory (LSTM) neural network, whose input and output are the raw 6D force-torque feedback and the Q-value,respectively. According to the Q values conditioned on the current state, the skill model selects a 6 degree-of-freedom action from the predefined action set. To accelerate the learning process, the data from demonstrations is used to pre-train the model before the DQN starts. In order to improve the insertion efficiency and safety, insertion step length is modulated based on the instant reward. Our proposed method is validated with the peg-in-hole insertion experiments on a precision assembly robot. The reusability of the skill model is also investigated with different types of insertion tasks.