Definition
Low-rank adaptation freezes selected pretrained weight matrices and learns each update as the product of two much smaller matrices. If the original weight has dimensions d by k, LoRA uses a chosen rank r that is much smaller than d or k. Training stores gradients and optimizer state for those low-rank factors instead of the full base matrix.
The adapter can remain separate from the base model or be merged into the weight for deployment. Separate adapters make it cheap to keep several task variants, but they create a versioned dependency: the adapter only has meaning with the compatible base model, target modules, rank, scaling, and tokenizer. LoRA reduces trainable parameters and training memory; it does not by itself shrink the frozen base model or guarantee lower inference cost.
Name and attribution
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen introduced the name "Low-Rank Adaptation," shortened to LoRA, in their 2021 paper. Their method injected trainable rank-decomposition matrices into Transformer layers while freezing pretrained weights.
Distinguish it from nearby terms
Full fine-tuning updates most or all model weights. LoRA learns a compact update for selected matrices. QLoRA backpropagates through a frozen quantized base into LoRA adapters, so quantization and low-rank adaptation solve different parts of the resource problem.
Check your understanding
An adapter file is only 50 MB, but it was trained against a different revision of the base model now in production. Which compatibility metadata must you check before treating the adapter as deployable?