In this paper, a 3D vision-based local obstacle avoidance system is designed and developed on a humanoid robot so that it can decide avoidance direction and walking motion effectively. We use a panorama environment map using speeded up robust feature (SURF) which is a robust image detector and descriptor to handle the obstacles which exist beyond the field of view. Moreover, we propose an avoidance direction decision method and a fuzzy logic based avoidance motion selection method. The robot decides the avoidance direction and avoidance walking motion for the obstacle by itself under information such as the size of objects and avoidance spaces. The proposed system is applied to the humanoid robot which we have built up with a Time of Flight camera. The results of the experiments show that the proposed method can be effectively applied to decide the avoidance direction and the walking motion.