Automatic defect detection is an essential part of quality inspection in industrial production. It can improve the effectiveness and reduce the cost for labor. Most of the detection systems are designed for specific tasks and they usually could not adapt to multiple types of objects simultaneously. In this paper, a general and flexible defect detection system is established for multi-type surface defects, it can achieve defect detection tasks for complex 3-dementional surface with different types and material, which has a precision of 95.3% and recall of 96.2%. A database of surface defects is also built to help the system to manage the data, recognize unknown defects and analyze the possible problems in the production line.