Definition
Data poisoning is the deliberate introduction of malicious, misleading, or strategically biased material into an AI system's data pipeline. The target may be training data, fine-tuning examples, retrieval indexes, agent memory, feedback records, or evaluation sets.
An attacker may want to reduce overall quality, steer decisions about a particular subject, create a hidden backdoor, or make a compromised system appear safe during evaluation. The same basic problem can also arise without an attacker when bad labels, duplicated records, or contaminated benchmarks silently shape the system.
Defenses begin with the data supply chain: source provenance, controlled mutation rights, review of high-impact changes, anomaly detection, versioned datasets, and the ability to trace a suspicious output back to the records that influenced it.
Distinguish it from nearby terms
Training-data poisoning changes learned behavior through model updates. Retrieval or memory poisoning changes the information available at runtime without retraining. Prompt injection is narrower still: it uses content the model may interpret as instructions.
Check your understanding
A team carefully reviews its fine-tuning set but lets any tool write permanent agent memory. It has protected one data plane while leaving another open to poisoning. The control has to follow every path by which data can affect later behavior.