This paper proposes a novel image data augmentation method based on the 4F optical diffuser system with two spatial light modulators. The method combines data augmentation methods based on geometric transformations with optical phase diffuser, resulting in a system which is effective and easy-to-use. This method can be used in an all-optical machine learning system or in a hybrid digital-optical system, and its digital implementation can also be used in learning-based computer vision applications. The method was tested in image classification of MNIST dataset. It was demonstrated that the augmented dataset significantly improves the classification accuracy. © 2021 Elsevier B.V.