Tencent Releases Hy4 preview: An Open-Weight MoE Model With a 1M Context Window
Tencent open-sourced Hy4 preview on 28 August 2026 under Apache 2.0: a 770B Mixture-of-Experts model that activates just 49B parameters per token and reads a one-million-token context window. This breakdown covers the full architecture, from 256 routed experts per layer to Gated DeepSeek Sparse Attention and the built-in speculative decoding layer; every benchmark figure Tencent published, including the 163-expert blind evaluation against GLM 5.3 and Kimi K3; the API price list against GPT-5.6 Sol; the eight-GPU serving recipes; and the four caveats worth naming before any of it reaches production.