What distinguishes MoE routers
Open Mixture-of-Experts models share the basic principle but diverge on almost every detail of routing. According to the comparison published by MarkTechPost and verified on November 7, 2026, against config.json, model cards and technical reports, all ten models examined use token-choice routing, in which each token independently selects its own experts. The differences center on five parameters: the number of routed experts, how many experts per token, the presence of always-active shared experts, the scoring function, and the load-balancing method.
The article places the range of routed experts between 8 and 384 and top-k between 1 and 8. The number of experts affects the weights resident in memory and all-to-all traffic between nodes; a lower top-k reduces computation and communication per token; shared experts add a dense path that every token pays for. According to the source, these are useful indications for sizing GPUs before an MoE deployment.
The configurations compared
Mixtral 8x7B, with 47 billion total parameters and 13 billion active, routes to 8 experts with top-2 and no shared experts, using softmax over the top-k and an auxiliary loss with a coefficient of 0.02 as indicated in the configuration. Qwen3-235B-A22B rises to 128 experts with top-8 and no shared experts and adopts a global-batch auxiliary loss with a coefficient of 0.001. Qwen3.6-35B-A3B changes approach: 256 experts, 8 routed plus one shared, with a gating method not disclosed in the configuration.
DeepSeek-V3 uses 256 experts with top-8, one shared, sigmoid scoring, and balancing without an auxiliary loss via a per-expert bias (noaux_tc); the experts are organized into 8 groups and the router first retains 4 of them, so each token reaches at most 4 nodes according to the technical report. DeepSeek-V4-Pro takes experts to 384 but lowers top-k to 6, switches to sqrtsoftplus, and uses hash routing in the first 3 MoE layers. Llama 4 Maverick alternates dense and MoE layers and selects a single routed expert out of 128 plus one shared, maintaining 17 billion active parameters out of 400 billion total. gpt-oss-120b uses a linear router over 128 experts with top-4, no shared experts, and MXFP4 weights, a format that allows the 117-billion model to fit on a single 80 GB GPU. Kimi K2 adopts the DeepseekV3ForCausalLM architecture with 384 experts and top-8, while GLM-4.5 prefers 160 experts and more layers.
Reported performance and information gaps
On the performance side, the source reports that routed sparsity varies by a factor of 4 in Mixtral (8 experts, top-2) up to a factor of 128 in Llama 4 Maverick (128 experts, top-1). The best figure cited concerns Kimi K2: at sparsity 48, the model would require 1.69 times fewer FLOPs than at sparsity 8 for the same validation loss, according to the model's report.
However, some gaps are openly acknowledged: Llama 4 Maverick and gpt-oss do not publish the gating or balancing method in their respective configurations, and the configurations and cards for Qwen3.6 and Llama 4 omit gating. Consequently, labels such as “17B active” can conceal routers that are very different from one another. Configuration transparency makes routing verifiable, but not for all models to the same degree; the missing details should be considered a limitation of the comparison, not an absence of functioning.
