Ledger on GitHub. Hub here. Branches elsewhere.
Three Frontier MoE Releases in Eight Days
Three Frontier MoE Releases in Eight Days
Two Chinese labs shipped open-weight mixture-of-expert models at frontier scale in the same week, followed by a third. DeepSeek V4 Pro 0813 reached general availability with 1.6 trillion total parameters and 49 billion active per token. The model scored 87.9 percent on Terminal-Bench 2.1 and 62.7 on DeepSWE, a coding benchmark that measures edits in real repositories.
DeepSeek shipped tooling alongside the weights. The release included an open-source agent harness, making it a model-plus-runtime rather than weights alone. This approach lets practitioners run agent loops without building scaffolding from scratch.
Three days later Qwen3.8-Max appeared as Alibaba’s new flagship. The architecture uses 2.4 trillion total parameters with 95 billion active and a native context window of 262,144 tokens. The model shipped with open weights shortly after announcement.
Alibaba reported that its open-model family crossed 3 billion downloads, a figure that would place it ahead of Meta’s Llama and Google’s Gemma in cumulative distribution. The claim comes from Bloomberg coverage citing company figures.
Z.ai followed with GLM-5.3, a long-context MoE at 753 billion total parameters and 40 billion active. The family supports a one-million-token context window and ships under an open-weight license.
All three releases use mixture-of-expert architectures that activate only a subset of parameters per forward pass. This pattern reduces inference cost while preserving model capacity. The cluster of releases in a single week suggests that MoE training and serving infrastructure has matured enough for multiple labs to ship at frontier scale simultaneously.
Infrastructure and Serving
Google announced TPU 8i and TPU 8t as separate chips optimized for inference and training. The split reflects the divergence between batch training and low-latency serving in production agent systems.
TPU 8i targets inference workloads. The design includes the highest on-chip SRAM in the TPU line, a CAE interconnect option, and Boardfly topology to reduce pod-level latency. Google described the chip as tuned for mixture-of-expert models and reinforcement-learning loops that require fast token generation.
TPU 8t remains the training variant. The two-chip strategy lets operators match hardware to workload without overprovisioning memory or interconnect for use cases that do not need both.
The inference/training split mirrors patterns in GPU cluster design, where H100 and H200 serve different roles despite shared architecture. Google’s approach makes the separation explicit at the silicon level.
Model Pricing and Consumer Deployment
DeepSeek V4-Flash-Vision-Exp appeared as a public multimodal MoE with 305 billion parameters. Pricing was listed at $0.44 input and $1.32 output per million tokens during peak hours. The model adds vision to the V4-Flash line and is available via API.
Meta released Muse Glimmer under Apache 2.0 as a 30-billion-parameter agentic model. The design targets consumer GPUs and scored 76.0 on SWE-Bench Verified. The model is intended for local inference without cloud dependencies.
xAI Grok 4.6 was mentioned in some frontier model trackers, but available evidence does not substantiate claims of capability improvement or half-price tier. The only verified data point is a tie on one benchmark. The model may exist, but the performance and pricing narratives lack public confirmation.
Table
| Release | Parameters | Active | Context | Benchmark | URL |
|---|---|---|---|---|---|
| DeepSeek V4 Pro 0813 | 1.6T | 49B | not stated | 87.9% Terminal-Bench 2.1; 62.7 DeepSWE | Link |
| Qwen3.8-Max | 2.4T | 95B | 262k native | not in sources | Link |
| GLM-5.3 | 753B | 40B | 1M | not in sources | Link |
| DeepSeek V4-Flash-Vision-Exp | 305B | not stated | not stated | not in sources | Link |
| Meta Muse Glimmer | 30B | not stated | not stated | 76.0 SWE-Bench Verified | Link |
| TPU 8i (inference) | not applicable | n/a | n/a | CAE; Boardfly; high SRAM | Link |
| TPU 8t (training) | not applicable | n/a | n/a | not in sources | Link |
For Practitioners
Act: Test DeepSeek V4 Pro and Qwen3.8-Max if your stack already handles MoE inference. Both models offer frontier capacity at open-weight terms, and the agent harness shipped with DeepSeek removes scaffolding work. Alibaba’s download count suggests a large user base that will surface integration patterns and failure modes quickly.
Watch: TPU 8i availability and per-pod pricing. If your inference load is MoE-heavy and latency-sensitive, a chip optimized for that profile may change cost structure. The CAE and Boardfly options suggest Google is targeting multi-agent orchestration and RL serving, not just batch completion.
Ignore: xAI Grok 4.6 claims until primary sources appear. Pricing and capability assertions circulating in tracker lists lack public confirmation. Wait for API documentation, a technical report, or reproducible benchmark runs before planning integration work.