Deep neural networks have shown their vulnerability to
adversarial attacks. In this paper, we focus on sparse ad-
versarial attack based on the ℓ 0 norm constraint, which can
succeed by only modifying a few pixels of an image. De-
spite a high attack success rate, prior sparse attack meth-
ods achieve a low transferability under the black-box proto-
col due to overfitting the target model. Therefore, we intro-
duce a generator architecture to alleviate the overfitting is-
sue and thus efficiently craft transferable sparse adversarial
examples. Specifically, the generator decouples the sparse
perturbation into amplitude and position components. We
carefullydesignarandomquantizationoperatortooptimize
these two components jointly in an end-to-end way. The
experiment shows that our method has improved the trans-
ferability by a large margin under a similar sparsity set-
ting compared with state-of-the-art methods. Moreover, our
method achieves superior inference speed, 700× faster than
other optimization-based methods. The code is available at
https://github.com/shaguopohuaizhe/TSAA.