Glossary

Mixture of experts — Glossary

An architecture that routes each input to a subset of specialized expert networks.

Also known as: MoE

Reviewed:

Full definition

MoE models can increase total parameters while activating only some experts per token, aiming for compute-efficient scaling. Routing quality and load balancing are central engineering challenges.

Related encyclopedia topic

Large Language Models — Large language models (LLMs) are neural networks trained on vast text corpora to predict tokens in context, enabling fluent generation, summarization…

← Back to glossary · Learning hub