---
title: Encoder
description: 'A model component that turns an input into an internal representation for classification, retrieval, reconstruction, or generation.'
canonical_url: 'https://darkfactory.dev/glossary/encoder'
markdown_url: 'https://darkfactory.dev/glossary/encoder.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Encoder


## Definition

An encoder maps an input into an internal representation that another component or task can use. The representation may be one vector, a sequence of vectors, or a hierarchy of feature maps. What it preserves depends on the architecture and training objective; calling something an encoding does not prove that it is compact, interpretable, or semantically complete.

In an autoencoder, the encoder produces a code from which a decoder reconstructs the input. In an encoder-decoder language model, it represents the source for a target-generating decoder. An encoder-only Transformer lets each input position attend in both directions and can feed classifiers, token labelers, or retrieval systems.

Jacob Devlin and colleagues' 2018 BERT paper is an influential encoder-only Transformer example. BERT pretrained bidirectional representations and adapted them to downstream language tasks with a small task-specific output layer. It popularized that architecture pattern, but it did not coin the general encoder role.

## Distinguish it from nearby terms

A decoder turns a representation or prior outputs into a target. An embedding lookup maps discrete IDs to learned vectors but may be only the first stage of an encoder. A tokenizer changes surface data into tokens; it does not learn the contextual representation produced by a neural encoder.

## Check your understanding

An encoder is trained only to classify document topics, then reused for exact quotation retrieval. Which information might its representation have discarded, and how would you test whether it is suitable for the new task?

## Related terms

- [Decoder](https://darkfactory.dev/glossary/decoder)
- [Encoder-decoder model](https://darkfactory.dev/glossary/encoder-decoder-model)
- [Transformer](https://darkfactory.dev/glossary/transformer)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Deep Learning](https://www.deeplearningbook.org/)
- [BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)
- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)