Adversarial Machine Learning: Attack Frameworks and Toolkits#

IBM Adversarial Robustness Toolkit (ART)#

ART is the most comprehensive framework for adversarial ML research and testing. Developed by IBM Research, it covers all four major attack categories with a unified API.

1
from art.attacks.evasion import FastGradientMethod
2
from art.estimators.classification import TensorFlowV2Classifier
3

4
# Create a classifier wrapper
5
classifier = TensorFlowV2Classifier(model=my_model, nb_classes=10, input_shape=(28, 28, 1))
6

7
# Generate adversarial examples
8
attack = FastGradientMethod(estimator=classifier, eps=0.1)
9
x_test_adv = attack.generate(x=x_test)

ART supports TensorFlow, PyTorch, Keras, scikit-learn, and MXNet, making it framework-agnostic. It includes implementations of over 40 attack algorithms and 20 defense mechanisms.

CleverHans#

cleverhans from the CleverHans Lab is one of the earliest and most cited adversarial example libraries. It provides reference implementations of major attack and defense methods, serving as both a research tool and educational resource.

The library focuses on constructing adversarial examples, building defenses, and benchmarking both against each other. Its clean API makes it straightforward to compare different attack strategies on the same model.

Foolbox#

foolbox distinguishes itself by supporting PyTorch, TensorFlow, and JAX with a single, consistent interface. If you need to test models across different deep learning frameworks, Foolbox provides the abstraction layer to do so efficiently.

1
import foolbox as fb
2

3
model = fb.PyTorchModel(my_model, bounds=(0, 1))
4
attack = fb.attacks.FGSM()
5
raw, clipped, is_adv = attack(model, images, labels, epsilons=0.03)

TextAttack#

TextAttack is purpose-built for adversarial attacks on NLP models. It provides a modular framework for composing attacks from four components: goal functions, transformations, constraints, and search methods.

Use cases include:

• Testing sentiment analysis models for robustness
• Evaluating text classification stability under perturbation
• Augmenting training data with adversarial examples
• Benchmarking NLP model defenses

DeepFool#

DeepFool implements a simple and accurate method for computing minimal perturbations needed to fool deep neural networks. It is particularly useful for measuring model robustness — the smaller the perturbation needed to cause a misclassification, the more vulnerable the model.

Adversarial ML Library (Ad-lib)#

Ad-lib takes a game-theoretic approach to adversarial ML. It provides modules for both learners and adversaries, enabling research into the strategic interaction between model defenses and attacks. This is particularly relevant for understanding how defenses hold up under adaptive adversaries.

Deep-pwning#

deep-pwning is a lightweight framework designed for experimenting with ML model robustness. Its minimal dependencies and straightforward design make it ideal for quick prototyping of adversarial attacks and defenses.

Microsoft Counterfit#

Counterfit from Microsoft Azure provides a generic automation layer for assessing ML system security at scale. It is designed for security teams who need to:

• Assess multiple models simultaneously
• Generate standardized reports on model vulnerabilities
• Integrate ML security testing into CI/CD pipelines
• Test models without deep ML expertise

Snaike-MLFlow#

Snaike-MLFlow is a red team toolsuite specifically targeting MLflow deployments. Since MLflow is widely used for experiment tracking and model serving, this tool addresses a significant attack surface in many ML pipelines.

secml-torch#

secml-torch extends the SecML library with PyTorch-specific functionality for robustness evaluation. It integrates seamlessly with existing PyTorch workflows, making it easy to add security testing to model development.

Malware Evasion#

The Malware Env for OpenAI Gym is a fascinating tool that frames malware evasion as a reinforcement learning problem. Agents learn to manipulate PE files (such as malware samples) to bypass antivirus detection, demonstrating how ML can be used offensively against ML-based security systems.

Diffusion Model Attacks#

BadDiffusion reproduces the research paper “How to Backdoor Diffusion Models?” published at CVPR 2023. It demonstrates how generative AI models can be compromised during training to produce malicious outputs, a growing concern as diffusion models are used for content generation.

ML Library Exploitation#

Charcuterie focuses on code execution techniques targeting ML or ML-adjacent libraries. It highlights how the supply chain of ML dependencies can be a vector for attacks.