Why Ensemble Defenses Seem Reasonable#

To understand why the paper’s conclusion is significant, we first need to appreciate why ensemble defenses seem like a good idea. Several factors contribute to this intuition:

Diversity of failure modes. Different defenses operate on different principles. Some detect adversarial inputs based on statistical properties (e.g., feature squeezing, detection-based methods), while others transform inputs to remove perturbations (e.g., input transformation, JPEG compression). It seems logical that an attack that evades a statistical detector might be caught by a transformation-based defense.

Complementary coverage. In traditional software security, different tools catch different classes of vulnerabilities. Static analysis finds buffer overflows; dynamic testing finds injection flaws; penetration testing finds misconfigurations. The hope is that adversarial defenses might similarly provide complementary coverage of the attack space.

Practical necessity. By 2017, it was clear that no single defense provided complete protection against adversarial examples. Rather than waiting for a “silver bullet,” combining existing defenses seemed like a pragmatic approach.

Theoretical analogies. In machine learning, ensemble methods (random forests, boosting) consistently outperform individual models. If ensembles work for prediction, why not for defense?

The Ensemble Defenses Studied#

The paper focuses on two specific ensemble defense strategies that were proposed in the literature:

Prediction Ensembles vs. Defense Ensembles#

In prediction ensembles (like random forests), the ensemble benefits from the statistical advantage of combining independent predictions. Even if each individual predictor is wrong some of the time, the majority vote across many predictors is more likely to be correct. This works because:

1. Each predictor has better-than-random accuracy.
2. Errors across predictors are somewhat independent.

In defense ensembles, the situation is different:

1. Each defense has weaker-than-required robustness. A defense that catches 90% of attacks is not “better than random” — it is catastrophically insecure for real-world deployment.
2. Failure modes are often correlated, not independent. Many defenses fail for the same underlying reasons (e.g., they don’t account for gradient masking or obfuscated gradients).

The Adaptive Adversary#

The most critical factor is the adaptive adversary. In real-world security, attackers adapt to known defenses. If a defender publishes their ensemble strategy, the attacker will craft adversarial examples specifically designed to evade all components simultaneously.

Consider an ensemble of three defenses: JPEG compression, bit depth reduction, and spatial smoothing. An adaptive attacker doesn’t need to defeat each defense separately. Instead, they can:

1. Formulate the attack as a single optimization problem that accounts for all three transformations.
2. Find perturbations that are robust to compression, survive bit depth reduction, and persist through spatial smoothing.
3. This is achievable because all three defenses are differentiable (or approximately so), allowing gradient-based optimization through the entire defense pipeline.

The key insight: the attacker faces the weakest link of the combined pipeline, not each defense independently.

The “Straw Man” Problem#

The paper demonstrates that in most ensemble defense proposals, the evaluation methodology is flawed. Specifically, researchers evaluate their ensembles against:

1. Non-adaptive attacks: Attacks that don’t account for the defense at all (e.g., standard FGSM or PGD without knowledge of the ensemble).
2. Single-defense adaptive attacks: Attacks that are designed to fool one defense at a time, rather than the ensemble as a whole.

When evaluated against a truly adaptive adversary who accounts for the entire ensemble, the combined defense performs only marginally better than its strongest individual component.

Attack Strategy#

The authors employ adaptive attacks that explicitly account for the ensemble. For each defense component, they compute (or approximate) the gradient of the attack objective through the entire defense pipeline.

For a defense ensemble $\mathcal{D} = \{d_1, d_2, \ldots, d_k\}$ applied sequentially to an input $\mathbf{x}$, the attacker optimizes:

$\min_\delta \quad \mathcal{L}(f(\mathcal{D}(\mathbf{x} + \delta)), y_{\text{target}}) + \lambda \|\delta\|_p$

where $f$ is the classifier, $\mathcal{D}$ is the combined defense transformation, and $\delta$ is the adversarial perturbation. The key is that the gradient $\nabla_\delta \mathcal{L}$ flows through all defense components simultaneously.

Handling Non-Differentiable Components#

Some defense components (like JPEG compression) are not smoothly differentiable. The authors handle this using:

• Straight-through estimators: Approximating the gradient of non-differentiable operations using the identity function (treating them as if they’re transparent to gradients).
• Proxy differentiable approximations: Replacing non-differentiable operations with smooth approximations that behave similarly locally.
• Expectation over Transformation (EOT): Sampling multiple random transformations (e.g., random JPEG quality levels) and averaging gradients over the samples.

Datasets and Models#

The experiments are conducted on standard benchmarks:

• MNIST: Handwritten digit classification (LeNet-5 architecture).
• CIFAR-10: Natural image classification (various architectures including ResNet).
• ImageNet: Large-scale image classification.

Using multiple datasets and architectures demonstrates that the findings are not artifacts of a specific model or data domain.

Result 1: Ensembles Provide Minimal Improvement#

The most striking finding is how little the ensemble improves over the best individual defense. In the authors’ experiments:

(These numbers are illustrative of the paper’s findings; exact values vary by attack strength and model.)

The ensemble reduces the attack success rate by only a few percentage points compared to the best individual defense — far from the robust protection one might hope for.

Result 2: Adaptive Attacks Neutralize the Ensemble#

When the authors apply their adaptive attacks (accounting for all defense components), the ensemble’s performance drops dramatically:

• Against non-adaptive attacks: The ensemble appears reasonably effective, reducing attack success rates significantly.
• Against adaptive attacks: The ensemble’s effectiveness collapses, and attack success rates approach those achieved against undefended models.

This gap between non-adaptive and adaptive evaluation is the hallmark of a weak defense and is one of the paper’s most important lessons.

Result 3: Component Order Matters Less Than Expected#

The authors test different orderings of the defense components in the ensemble pipeline. While there is some variation in effectiveness, no ordering provides substantially better protection. This is because the adaptive attacker accounts for the entire pipeline regardless of component order.

Result 4: Increased Computational Cost Without Proportional Benefit#

Ensemble defenses add significant computational overhead — each additional defense component requires additional processing per input. The paper shows that this overhead is not justified by the marginal improvement in robustness. In some cases, the computational cost of the ensemble is 5-10× that of a single defense, for only a 3-5% reduction in attack success rate.

1. Always Evaluate Against Adaptive Adversaries#

The single most important lesson: evaluate defenses against adversaries who know about the defense and actively try to defeat it. This is Kerckhoffs’s principle applied to adversarial ML — the security of a system should not depend on the secrecy of its defense mechanism.

2. Non-Adaptive Evaluation Is Meaningless#

If a defense only works against attackers who don’t know about it, it’s not a defense — it’s security through obscurity. The paper shows that many published defense improvements disappear under adaptive evaluation.

3. Report Against the Strongest Known Attack#

Defenses should be evaluated against the strongest available attacks at the time of publication. Showing that a defense works against FGSM but not PGD (a stronger attack) is not a meaningful contribution.

4. Be Skeptical of Marginal Improvements#

If an ensemble defense improves robust accuracy by only a few percentage points over its best individual component, it may not be worth the additional complexity and computational cost. Researchers should be honest about the practical significance (or insignificance) of their improvements.

5. Understand the Threat Model#

Every defense operates within a threat model that specifies what the attacker can and cannot do. The paper argues that threat models should be conservative (assuming strong adversaries) rather than optimistic (assuming weak, non-adaptive adversaries).

Related Work on Breaking Defenses#

Several other influential papers have similarly debunked defense strategies:

• Athalye, Carlini, and Wagner (2018): “Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples” systematically broke several ICLR 2018 defenses by identifying gradient obfuscation as a common failure mode.
• Carlini and Wagner (2017): Their CW attacks demonstrated that many proposed defenses could be defeated with appropriately optimized attacks.
• Tramer et al. (2020): “On Adaptive Attacks to Adversarial Example Defenses” showed that even defenses claiming robustness against adaptive attacks often fail under careful evaluation.

The He et al. paper is an early and important contribution to this line of work, specifically targeting the ensemble defense approach.

Don’t Rely on Defense Ensembles#

Combining multiple weak defenses is not a viable strategy for achieving robustness. If your threat model requires strong adversarial robustness, you need a fundamentally strong defense — not a collection of weak ones.

Invest in Adversarial Training#

The most consistently effective defense against adversarial examples is adversarial training — incorporating adversarial examples into the training data. While not perfect, adversarial training provides provable robustness within certain bounds and scales better than ensemble defenses.

Consider Certified Defenses#

Recent work on certified robustness provides mathematical guarantees that a model is robust to perturbations within a specified norm ball. While these guarantees come with trade-offs (often reduced clean accuracy), they provide the kind of assurance that ensemble defenses cannot.

Red Team Your Defenses#

Before deploying any defense, have an independent team attempt to break it using adaptive attacks. This is the adversarial ML equivalent of penetration testing and is essential for understanding real-world robustness.

Be Honest About Limitations#

No defense is perfect. Being transparent about what your defense can and cannot protect against is more valuable than claiming robustness that doesn’t hold under scrutiny.