---
title: 'Adversarial training'
description: 'Training on adversarially generated examples to improve robustness against attacks inside a defined threat model.'
canonical_url: 'https://darkfactory.dev/glossary/adversarial-training'
markdown_url: 'https://darkfactory.dev/glossary/adversarial-training.md'
collection: glossary
date_published: '2026-08-04T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Adversarial training


## Definition

**Adversarial training** trains a model on adversarially generated examples so it learns to perform better against attacks inside a defined threat model. A common formulation searches for a worst-case allowed perturbation around each training example, then updates the model to reduce loss on those hard examples.

The result is conditional robustness. It depends on the perturbation set, attacker knowledge, attack strength, optimization method, data distribution, and model capacity used during training. A model robust to bounded image perturbations is not thereby robust to data poisoning, prompt injection, unrestricted physical changes, or novel attacks.

## A strong defense with real tradeoffs

Madry and coauthors' 2017 robust-optimization account made projected-gradient attacks and the min-max training formulation a central reference for modern adversarial training. The method can be computationally expensive and may reduce clean-data accuracy or fail under a broader evaluation. Robustness claims should therefore include both ordinary performance and adaptive attacks designed against the actual defense.

## Distinguish it from nearby terms

Data augmentation adds transformed examples for generalization and may not involve an attacker. Red teaming searches for failures across a wider system. Adversarial training changes model optimization using selected attacks. Certified robustness attempts to prove behavior within a mathematical bound; empirical training alone does not provide that proof.

## Check your understanding

A classifier was adversarially trained against small pixel changes. Can it safely read road signs from a camera under stickers, weather, blur, and sensor faults? Not from that claim. Test the deployed threat model, including adaptive attacks and clean performance.

## Related terms

- [Adversarial example](https://darkfactory.dev/glossary/adversarial-example)
- [AI red teaming](https://darkfactory.dev/glossary/red-teaming)
- [Training](https://darkfactory.dev/glossary/training)

## Related factory areas

- [Security, privacy & software supply chain](https://darkfactory.dev/factory/security)

## Evidence and further reading

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://arxiv.org/abs/1706.06083)