Definition
A model architecture that routes each input or token through a selected subset of specialized parameter blocks rather than activating the entire model.
Distinguish it from nearby terms
MoE routes computation inside one model; model routing selects among separate deployed models or systems.
Check your understanding
Total parameter count and active parameters per token are different quantities in an MoE model.