Definition
Contrastive learning trains a representation by comparing examples rather than predicting only a class label. A positive pair is expected to represent the same item, meaning, or compatible views; a negative pair is expected to differ. The objective pulls positive representations together and, in methods that use explicit negatives, pushes negative representations apart or ranks the positive higher.
Pair construction supplies the supervision. Labels can define matches. Two augmentations of one image can form a self-supervised positive pair. Image and caption pairs can align modalities. The method will learn the assumptions embedded in those pairings, including shortcuts and mistakes.
Siamese networks and contrastive loss
Jane Bromley and colleagues described the "Siamese" neural network in 1993 for signature verification, with two identical subnetworks learning comparable feature vectors. Raia Hadsell, Sumit Chopra, and Yann LeCun's 2006 paper then named and formulated a contrastive loss for mapping neighbors close together while keeping non-neighbors separated by a margin. Later contrastive systems broadened the idea to larger batches, augmentations, and paired modalities.
Distinguish it from nearby terms
Contrastive learning is an objective or training strategy. An embedding is the representation it produces. Metric learning is the broader effort to learn a useful distance or similarity function; many contrastive methods are metric-learning methods, but the terms are not exact synonyms.
Check your understanding
You treat two crops of the same medical image as a positive pair. One crop removes the lesion that determines the diagnosis. What assumption did the augmentation violate, and why can the contrastive loss still improve?