Journal Article
Computer Vision

One Pixel Attack for Fooling Deep Neural Networks

Jiawei Su(Kyushu University), Danilo Vasconcellos Vargas(Kyushu University), Kouichi Sakurai(Advanced Telecommunications Research Institute International)
January 4, 2019IEEE Transactions on Evolutionary Computation2,585 citations
arXivPDFDOI

2.6k

Citations

125

Influential Citations

IEEE Transactions on Evolutionary Computation

Venue

2019

Year

Abstract

Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

Analysis

Why This Paper Matters

This paper challenges the prevailing assumption that adversarial perturbations need to be high-dimensional or perceptible. By showing that a single pixel change can fool state-of-the-art DNNs with high confidence, it reveals a fundamental vulnerability that is both surprising and practically concerning. The work bridges adversarial machine learning and evolutionary computation, offering a new tool for robustness evaluation.

Technical Contributions

The key innovation is the use of differential evolution (DE) to generate one-pixel perturbations in a black-box setting. Unlike gradient-based attacks, DE requires no network internals, only output probabilities. The method iteratively mutates and selects pixel modifications, efficiently exploring the sparse perturbation space. This approach is network-agnostic and can be applied to any classifier that outputs class probabilities.

Results

The attack achieved a 67.97% fooling rate on CIFAR-10 (Kaggle test set) with 74.03% average confidence, and 16.04% on ImageNet with 22.91% confidence. On the original CIFAR-10, similar vulnerability was observed. These results are notable given the extreme constraint of a single pixel modification, which is imperceptible to humans.

Significance

This work has broad implications for AI safety and robustness. It demonstrates that even minimal input changes can cause misclassification, urging the development of more resilient architectures. Additionally, it showcases evolutionary computation as a viable alternative to gradient-based attacks, particularly in black-box scenarios. The low cost and simplicity of the attack make it a practical tool for evaluating model robustness in real-world deployments.

adversarial-attacksdifferential-evolutionblack-box-attackcomputer-visiondeep-neural-networksrobustnessevolutionary-computation
Neural Networks

Related Papers