---
title: 'Positional encoding'
description: 'A numerical signal that lets a sequence model distinguish token positions or distances.'
canonical_url: 'https://darkfactory.dev/glossary/positional-encoding'
markdown_url: 'https://darkfactory.dev/glossary/positional-encoding.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Positional encoding


## Definition

Positional encoding supplies a numerical signal about where tokens occur in a sequence. Content-only self-attention applies the same operation at every position and, without a mask or positional signal, cannot distinguish one permutation of the inputs from another except by permuting the outputs the same way. Language depends on order, so a transformer needs some mechanism that exposes absolute position, relative distance, or both.

A positional signal can be added to token embeddings, learned as a table, applied as a relative attention bias, or incorporated by rotating query and key vectors. These mechanisms behave differently near and beyond the sequence lengths used in training. The context-window limit, training distribution, and implementation still constrain usable length; a mathematical encoding defined for larger positions does not prove that the model can reason well there.

## The Transformer paper

Ashish Vaswani and colleagues' 2017 Transformer used sine and cosine functions at different frequencies and added the resulting vectors to input embeddings. The paper also tested learned positional embeddings and reported similar results for its translation tasks. It used the phrase "positional encoding," but position representations existed in earlier sequence models, so the paper should not be treated as the origin of the general idea.

## Distinguish it from nearby terms

Token embeddings represent token identity or content. Positional encoding represents location or distance. A causal attention mask controls which positions may communicate. The mask and encoding can both carry order-related information, but they have different jobs.

## Check your understanding

Two unmasked attention inputs contain the same token vectors in a different order and have no positional signal. What property of the computation makes the two orders indistinguishable except for the corresponding output permutation?

## Related terms

- [Transformer](https://darkfactory.dev/glossary/transformer)
- [Embedding](https://darkfactory.dev/glossary/embedding)
- [Attention](https://darkfactory.dev/glossary/attention)
- [Context window](https://darkfactory.dev/glossary/context-window)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)