Models and training

Activation function

A function that transforms a neural network unit's pre-activation, usually to add nonlinearity or control how much signal passes forward.

Active learning

A training loop in which the learner chooses which unlabeled examples should be sent to an oracle for labels.

Autoencoder

A model trained to encode an input into an internal representation and decode that representation into a reconstruction of the input.

Backpropagation

A reverse pass through a differentiable computation graph that computes how the loss changes with respect to each parameter.

Causal language model

A language model trained to predict each token from the tokens that precede it, without access to future positions.

Claude

Anthropic's family of language models, including the Opus, Sonnet, Haiku, Fable, and Mythos lines, also used as the name of its assistant product.

Claude Fable

Anthropic's generally available Claude model line for demanding reasoning and long-horizon work, using safeguards that distinguish its deployment from Mythos.

Claude Haiku

Anthropic's Claude model line aimed at high-volume tasks where latency and cost strongly constrain the choice of model.

Claude Mythos

Anthropic's Claude model line deployed through trusted access programs with safeguards designed for advanced cybersecurity and life-sciences work.

Claude Opus

Anthropic's Claude model line aimed at complex coding, analysis, and knowledge work requiring sustained judgment.

Claude Sonnet

Anthropic's Claude model line positioned to balance capability, response speed, and cost for everyday coding and knowledge work.

Compute

The processing work or capacity used to train and run an AI system, measured with quantities such as operations, accelerator time, throughput, or cost.

Contrastive learning

A representation-learning approach that trains examples judged related to be closer and examples judged different to be farther apart in a learned space.

Convolutional neural network (CNN)

A neural network that applies shared learned filters across local regions of spatial or sequential data.

Cross-entropy

A loss that measures how poorly a predicted probability distribution represents the target distribution.

Data augmentation

Expanding or varying training examples through transformations or generation intended to preserve task-relevant meaning.

Decoder

A model component that turns an internal representation or prior outputs into a reconstruction, prediction, or target sequence.

Decoder-only model

A Transformer-style sequence model that uses causal self-attention to predict the next token without a separate encoder stack.

DeepSeek

The model family developed by DeepSeek-AI, including general, coding, and reasoning releases available through weights or hosted services.

Diffusion model

A generative model that learns to reverse a gradual noising process, producing samples by repeatedly denoising an initial random state.

Direct preference optimization (DPO)

A preference-training method that increases a model's relative likelihood of chosen responses over rejected responses without training a separate reward model.

Encoder

A model component that turns an input into an internal representation for classification, retrieval, reconstruction, or generation.

Encoder-decoder model

A model with separate paths for representing a source input and generating a target output conditioned on that representation.

Epoch

One nominal pass through a training dataset, usually divided into batches and followed by another shuffle or sampling cycle.

Federated learning

Training a shared model from updates computed by distributed participants while their raw training data remains local.

Fine-tuning

An additional training phase that adapts a pretrained model by updating all parameters or a selected subset on new data.

Frontier model

A general-purpose AI model at or near the leading edge of broadly evaluated capability at a particular time.

GPT

OpenAI's Generative Pre-trained Transformer model lineage, spanning numbered generations and named model tiers with different capabilities.

GPT Astra

OpenAI's GPT model tier introduced with GPT-6 for demanding reasoning, coding, research, and computer-use workloads.

GPT Luna

OpenAI's named GPT tier for efficient, focused workloads, including high-volume tasks and lower-cost steps within agent systems.

GPT Sol

OpenAI's named GPT model tier for substantial reasoning and agentic work, with its role and capabilities defined by the numbered release.

GPT Terra

OpenAI's named GPT tier introduced with GPT-5.6 to balance capability and cost between Sol and Luna.

Gemini

Google DeepMind's model family for language and multimodal tasks, whose name also identifies Google's assistant application.

Gemini Flash

The Flash tier within Google DeepMind's Gemini family, introduced for fast, efficient serving and released in versioned language and multimodal models.

Gemini Flash-Lite

The Flash-Lite tier within Google DeepMind's Gemini family, positioned for low-latency, low-cost serving and high-throughput tasks.

Gemini Pro

The Pro tier within Google DeepMind's Gemini family, with versioned models for broad language, multimodal, and complex reasoning workloads.

Gemma

Google's family of downloadable AI models for generation and specialized tasks, with capabilities and licenses that vary by release.

Generalization

The ability of a learned model or system to perform well on relevant data, tasks, or environments that were not used to fit it.

Generative adversarial network (GAN)

A generative model trained as a game between a generator that produces candidates and a discriminator that tries to detect generated samples.

Gradient descent

An iterative optimization method that moves parameters opposite the gradient of an objective being minimized.

Graphics processing unit (GPU)

A throughput-oriented parallel processor widely used for neural-network training and inference.

Grok

The language and multimodal model family introduced by xAI, also used as the name of its assistant and related products.

Hyperparameter

A configuration choice that governs a model or learning procedure but is not ordinarily learned as a model weight.

Instruction tuning

Fine-tuning a pretrained model on instruction-and-response examples so it learns to perform tasks described in natural language.

Jev

TypeSafe AI's model family for typed decisions and probabilities from supplied text, introduced as its first System One model.

Kimi

Moonshot AI's model family, including the Kimi K2 and K3 releases, whose architecture, capabilities, and licenses must be identified by version.

Latent space

A space of unobserved variables or learned representations used to model structure behind observed data.

Learning rate

A hyperparameter that sets the base step size for parameter updates during optimization.

Llama

Meta's family of foundation models, distributed as release-specific model artifacts under licenses whose openness is disputed.

Loss function

A function that converts a model's prediction and target into a training penalty for an example or batch.

Low-rank adaptation (LoRA)

A parameter-efficient fine-tuning method that freezes selected base weights and learns low-rank matrices representing their updates.

Masked language model

A language model trained to reconstruct selected hidden tokens from the visible context on both sides.

Mistral

Mistral AI's model family and broader model portfolio, with general and specialist releases whose capabilities and licenses differ.

Mixture of experts (MoE)

A model architecture in which a learned router selects one or more expert subnetworks for each input or token.

Model checkpoint

A saved snapshot of model parameters and, when needed, the surrounding state required to resume training.

Model distillation

Training a student model to reproduce useful behavior from a teacher model or ensemble, often to reduce inference cost.

Model weights

The learned numerical values in a model's connections or transformations, often used loosely for all learned parameters.

Muse Glimmer

Meta Superintelligence Labs' 30B dense multimodal model, distilled from Muse Spark and released with downloadable Apache 2.0 weights for local agent workflows.

Muse Spark

Meta Superintelligence Labs' multimodal reasoning model family for agentic and coding work, separate from the Muse agent and Muse Code applications.

Muse models

Meta Superintelligence Labs' Muse-branded models, including Spark, Glimmer, and specialized media and speech models, distinct from the Muse personal agent.

Objective function

The mathematical quantity an optimization or search process is configured to minimize or maximize.

Open-weight model

A model whose trained parameters are distributed under stated terms, without implying that its training process or complete system is open source.

Optimizer

An algorithm that turns gradients and accumulated training state into parameter updates.

Overfitting

A failure to generalize in which performance on development data is better than performance on representative unseen data.

Positional encoding

A numerical signal that lets a sequence model distinguish token positions or distances.

Pretraining

Training performed before a later adaptation stage, usually to learn reusable representations or capabilities from a broader dataset.

Proprietary model

A model whose owner materially controls access to its weights, development artifacts, operation, modification, or redistribution.

Quantization

Mapping model values to a lower-precision representation to reduce storage, memory traffic, or computation.

Quantized low-rank adaptation (QLoRA)

A fine-tuning method that backpropagates through a frozen quantized base model into trainable LoRA adapters.

Qwen

Alibaba's family of language and multimodal models, with downloadable and hosted releases that differ in size, specialization, and license.

Recurrent neural network (RNN)

A neural-network architecture that applies a recurrent update while carrying hidden state through a sequence.

Regularization

A change to the learning problem intended to improve performance on unseen data, often by limiting or perturbing what the model can fit.

Reinforcement learning from human feedback (RLHF)

A training approach that learns a reward signal from human judgments and uses reinforcement learning to optimize a model against that signal.

Reinforcement learning with verifiable rewards (RLVR)

Training a policy with rewards computed by a task verifier, such as an answer checker or executable test suite.

Scaling laws

Empirical relationships fitted to how model loss or performance changes with resources such as parameters, data, or compute.

Small language model (SLM)

A language model small enough for a target deployment's memory, latency, cost, or control constraints relative to contemporary large models.

Training batch

A group of training examples processed together to estimate a gradient before an optimizer update.

Training data

The examples and signals used to fit a model's learned parameters during pretraining, fine-tuning, or other learning procedures.

Transfer learning

Using representations or parameters learned in a source task or domain to help a target task or domain.

Variational autoencoder (VAE)

A latent-variable generative model trained with variational inference so it can reconstruct data and sample new examples.

Decision model

A model used to turn supplied context into bounded judgments that software can consume, such as an option, score, or estimated probability.

System One

TypeSafe AI's proposed class of models built for fast judgments and typed, machine-consumable outputs, with Jev as its first public model.

Open-source AI

An AI system released with the rights and preferred form needed to use, study, modify, and share the system.