Definition
Self-supervised learning creates a training target from the structure of data rather than requiring a person to label each example. A text model can predict the next token or reconstruct a masked token. A vision or audio model can learn that two transformed views came from the same source, reconstruct missing regions, or predict one representation from another.
The training data may be called unlabeled because it lacks task-specific human annotations. It still contains the information used as the target. A next-token example uses earlier tokens as input and a later token as the answer. This turns large collections of raw text, images, audio, or video into many training examples.
The method often learns reusable representations during pretraining. A later stage can adapt them to a specific task through fine-tuning, prompting, or a supervised head. Much of modern language-model pretraining is self-supervised, though later post-training may use labeled demonstrations, preference data, or reinforcement learning.
Meta AI's 2021 article called self-supervised learning the "dark matter of intelligence" and helped popularize that metaphor. It did not claim to coin the technical term, so the phrase should be treated as a description, not an origin story.
Distinguish it from nearby terms
- Supervised learning uses externally supplied target labels for the task.
- Unsupervised learning is a broader category for learning structure without task labels. Self-supervised methods manufacture a prediction target from the data itself.
- Semi-supervised learning combines a smaller labeled set with a larger unlabeled set.
- Self-supervision does not mean the model checks its own safety, verifies its claims, or chooses its own objective.
Operational significance
Removing manual labels does not remove data risk. Source rights, duplication, contamination, representation gaps, and harmful content still matter. The pretraining objective also shapes what the representation preserves. Evaluate the downstream task instead of assuming that scale or a lower training loss guarantees useful knowledge.
Check your understanding
Masking 15 percent of a document and training a model to recover the missing tokens is self-supervised because the original document supplies the targets. Asking annotators to mark each document's topic is supervised learning.