Adversarial Patch Attacks Explained and Defended Against

Adversarial patch attacks are a type of cyber threat that can compromise the security of AI-powered systems. These attacks involve adding a small, carefully crafted patch to an image or video that can deceive the AI system into making a false decision.

The patch can be designed to be extremely small, even smaller than a pixel, making it difficult to detect. This is because the patch is specifically crafted to exploit the weaknesses of the AI system's image recognition algorithms.

Imagine trying to sneak a small sticker onto a photo without anyone noticing - that's essentially what an adversarial patch does, but with the goal of deceiving the AI system.

Adversarial patches are a type of attack that can be applied to any background.

This is because the patch application operator, A, is universal in that it works for any background, as long as the transformations and location are properly applied.

The patch can be applied to an image at any location, thanks to the distribution over locations in the image, L.

This makes adversarial patches a versatile threat that can be applied to a wide range of images.

The transformations applied to the patch can also be varied, thanks to the distribution over transformations of the patch, T.

This allows the patch to be tailored to a specific image or scenario.

The training set of images, X, is used to train the patch application operator, A, to generate effective adversarial patches.

This training data is essential for the patch to work effectively.

Adversarial patches can be created using various attack methods. One of the first attack strategies proposed is the Fast Gradient Sign Method (FGSM), developed by Ian Goodfellow et al. in 2014.

This method creates an adversarial example by changing the input image in the direction of maximizing the loss of the network for classifying the input image as a label. The method uses the expression \( \tilde{x} = x + \epsilon \cdot \text{sign}(

abla_{x} J(\theta,x,y)) \) to create the adversarial example.

The intensity of the noise, \( \epsilon \), is a key factor in the FGSM method. A default value of \( \epsilon = 0.02 \) corresponds to changing a pixel value by about 1 in the range of 0 to 255.

FGSM can be adapted to increase the probability of a specific class instead of minimizing the probability of a label. However, for those, there are usually better attacks such as the adversarial patch.

Local Gradient Smoothing (LGS) is another attack method, proposed by Naseer et al. in 2019. It is based on the empirical observation that pixel values tend to change sharply within these adversarial patches.

Here are some key characteristics of adversarial patch-based attacks:

Adversarial patches are a type of attack that can be transferred to different models, but the effectiveness of the patch depends on the similarity between the original and target models.

The adversarial patch attack can be trained on multiple models, making it more versatile, but the patches may not work equally well on all models. For instance, a patch trained on ResNet34 may not have the same fool accuracy on DenseNet121.

The key factor that allows patch attacks to generalize well is if all the networks have been trained on the same data, such as ImageNet. This is because dataset biases can make networks recognize specific patterns in the underlying image data that humans wouldn't have seen, and/or only work for the given dataset.

Generating patches is a crucial step in creating adversarial attacks. This process involves applying transformations to a patch and then placing it in an image at a specific location.

The patch application operator, A, takes an input patch p, image x, location l, and transformations t. It first applies the transformations to the patch, and then applies the transformed patch to the image x at location l.

A key departure from prior work is that this perturbation is universal, working for any background. This means that the patch can be applied to different images and still be effective.

The transformations and locations are distributed over T and L respectively, indicating that the patch can be applied in various ways and at different positions within the image.

White-Box Attack Transferability is a crucial aspect of patch attacks, and it's interesting to see how well they can generalize across different models.

The adversarial patch attack, as proposed in Brown et al., was originally trained on multiple models and was able to work on many different network architectures.

However, the fool accuracy of patches trained on ResNet34 was significantly lower on a different network like DenseNet121.

But even with lower fool accuracy, the patches still had a considerable impact on DenseNet.

The reason patch attacks can generalize well is that all the networks have been trained on the same data, in this case, ImageNet.

This is why the knowledge of what data has been used to train a specific model is already worth a lot in the context of adversarial attacks.

Protecting Against Attacks can be a real challenge. The sad truth is that there isn't much we can do to protect a network against adversarial attacks.

White-box attacks require access to the model and its gradient calculation, which can be prevented by ensuring safe, private storage of the model and its weights. However, some attacks, called black-box attacks, also work without access to the model's parameters.

An intuitive approach to protect a model is to train/finetune it on adversarial images, leading to an adversarial training similar to a GAN. This usually just ends up in an oscillation of the defending network between weak spots.

Another common trick to increase robustness against adversarial attacks is defensive distillation, which trains a secondary model on the softmax predictions of the first one. This "smoothes" the loss surface, making it harder for attackers to find adversarial examples.

Neural networks, including CNNs, are vulnerable to adversarial attacks because they don't know what they don't know. A large dataset represents just a few sparse points in the extremely large space of possible images.