Hy4 preview is Tencent’s new flagship Mixture-of-Experts model, and it arrived on 28 August 2026 with the weights already on Hugging Face and an Apache 2.0 licence attached. The headline specification is 770 billion total parameters, 49 billion of them activated per token, and a context window of one million tokens.
That combination is the point. A model this large would normally be a closed API you rent; Tencent has instead published the checkpoint, an FP8 quantised variant, a fine-tuning pipeline and the serving recipes to run it on eight GPUs. Anyone can download it, inspect it, and host it themselves.
The release also lands in the middle of a busy month for open weights from Chinese labs. Alibaba shipped a 125B multimodal Mixture-of-Experts preview days earlier, and Z.ai was unmasked as the lab behind the anonymous Ox Alpha model the same week. Tencent’s entry is the biggest of them by parameter count.
This breakdown covers exactly what was released, how the architecture works, what the one-million-token window is actually for, every benchmark figure Tencent has published, what the model costs to call or to self-host, and the caveats that matter before you put it anywhere near production. Every number below traces to a document in the References section.
Table of contents
- Hy4 preview: What Tencent Actually Released
- Inside the Hy4 preview Architecture
- The 1M Context Window, and What It Is Actually For
- Hy4 preview Benchmarks: Every Number Tencent Published
- What Hy4 preview Costs to Call or to Host
- Running Hy4 preview Yourself: The Serving Details
- How Hy4 preview Changes the Open-Weight Race
- The Caveats Worth Naming Before You Deploy
- What Hy4 preview Means for Your Business
- Hy4 preview: Frequently Asked Questions
- References and Further Reading
Hy4 preview: What Tencent Actually Released
The release is unusually complete for a preview. It is not a demo endpoint or a waitlist — it is weights, code, containers and a price list, published together.
The headline numbers
Hy4 preview comprises 770B total parameters with 49B activated per token. The backbone runs 78 layers with a hidden size of 6,144, a vocabulary of 120,832 tokens, and a context length of one million. On OpenRouter the window is listed precisely as 1,048,576 tokens, with completions capped at 64,000.
Where the weights live
Tencent published Hy4 preview and Hy4 preview-FP8 simultaneously on Hugging Face, ModelScope, GitCode and CNB. The licence is Apache 2.0, which permits commercial use, modification and redistribution without a separate agreement — a materially different position from the bespoke community licences several rival open-weight models ship under.
What “preview” means here
Tencent is explicit that this is an early version of the Hy4 generation, with headroom left in both pre-training and post-training. The team says it would rather ship early and hear what breaks, the same approach it took with Hy3 preview. Treat Hy4 preview as a capable checkpoint on a moving line, not a finished flagship.
Post-training is where most of the recent gains in this class of model come from, through supervised fine-tuning and reinforcement learning applied after the base run. Tencent credits a substantially larger post-training effort, alongside stronger pre-training, for what it calls the largest generation-over-generation gain it has measured.
Where you can use it without downloading it
The model is live inside Tencent’s own products — Yuanbao, ima, WorkBuddy and CodeBuddy — and reachable through Tencent Cloud TokenHub and OpenRouter. WorkBuddy and CodeBuddy are free for two weeks from launch.
| Property | Value |
|---|---|
| Architecture | Mixture-of-Experts |
| Total parameters | 770B |
| Activated per token | 49B |
| Layers | 78 (1 dense FFN, 77 MoE) |
| Experts per MoE layer | 256 routed, 1 shared |
| Experts used per token | Top-8 routed, plus the shared expert |
| Attention | Gated DeepSeek Sparse Attention, 64 heads |
| Context length | 1M tokens |
| Hidden size / vocabulary | 6,144 / 120,832 |
| Licence | Apache 2.0 |
| Variants | Hy4 preview, Hy4 preview-FP8 |
Inside the Hy4 preview Architecture
The interesting part of Hy4 preview is not its size. It is how little of that size runs on any given token, and the borrowed ideas that make a million-token window tractable at all.
A sparse backbone, layer by layer
Of the 78 layers, only the first uses a standard dense feed-forward network. The remaining 77 replace it with a Mixture-of-Experts block holding 256 routed experts and one shared expert. Every token activates the top-8 routed experts plus the shared one, so nine of 257 experts fire per layer.
The compute a single token actually touches
That routing is why a 770B large language model behaves, computationally, like a 49B one. Sparsity has become the standard way to buy capacity without buying the matching inference bill, and Hy4 preview takes it further than most open checkpoints released to date.
Gated DeepSeek Sparse Attention
The attention module uses Gated DSA with an IndexCache for cross-layer sparse index reuse — an approach Tencent credits directly to work from DeepSeek and GLM. The indexer runs 32 heads at 128 dimensions with a top-k of 2,048, and queries are compressed to 2,048 dimensions against key-value compression at 512.
Hyper-connections on the residual path
The residual pathway uses identity Hyper-Connections, giving four parallel residual streams instead of one. The stated goal is wider information flow between layers, which matters more as depth increases.
A built-in draft model
Alongside the backbone sits one native multi-token prediction layer of 10B parameters, 0.7B of them activated, dedicated to speculative decoding. Both official serving recipes use it: three speculative tokens under vLLM, and a three-step NEXTN configuration under SGLang.
Sparsity is the whole economic argument, and the ratio is stark when you put the three parameter counts side by side.
The 1M Context Window, and What It Is Actually For
A million tokens is the specification everyone quotes and the one most likely to be misread. It is a ceiling on what the model can read, not a promise about what it can use well.
A million tokens in practical terms
One million tokens is roughly 750,000 English words — a mid-sized codebase, several years of meeting notes, or a full set of contracts and their amendments. The point of the window is that none of it needs chunking, retrieval or summarisation first.
Sparse attention is what makes it affordable
A dense attention model at this length would be economically absurd, because attention cost grows quadratically with sequence length. Gated DSA plus IndexCache is precisely the mechanism that keeps the long tail of that window from dominating the bill, which is why the architecture and the context claim have to be read together.
The output ceiling is much smaller
Reading a million tokens does not mean writing them. OpenRouter caps completions at 64,000 tokens, so Hy4 preview is built to consume enormous context and emit a normal-sized answer. That shape suits analysis, review and refactoring far better than bulk generation.
Long context is not the same as long memory
Nothing in the release claims perfect recall across the full window, and no independent long-context benchmark result has been published for Hy4 preview yet. Treat the one-million figure as headroom to design against, and validate retrieval quality on your own documents before you rely on it.
Hy4 preview Benchmarks: Every Number Tencent Published
Tencent reports strong results across coding, science and agentic evaluations. All of them are vendor-reported, which is the single most important caveat in this article.
Coding and terminal work
Hy4 preview scores 82.9% on SWE-bench Multilingual and 65.7 on the public split of SWE-Bench Pro. On Terminal-Bench 2.1 it reaches 85.4, which Tencent says puts it ahead of DeepSeek V4 Pro. The largest generational jump reported is on DeepSWE, from 28.0 to 64.3.
Science and reasoning
On GPQA Diamond the model posts 92.3. On Humanity’s Last Exam with tools enabled it reports 55.4. SkillsBench V1 comes in at 62.9. These are the figures behind Tencent’s claim that Hy4 preview sits at the open-source frontier.
Agentic and tool use
MCP-Atlas returns 83.7 and Toolathlon-Verified 74.1, both agent-oriented evaluations that exercise tool calling rather than single-turn answers. APEX-Agents is markedly lower at 37.1, a useful reminder that agentic benchmarks are far from saturated.
The blind expert evaluation
Tencent also ran a human study: 163 internal experts rated model outputs on 203 engineering tasks. Hy4 preview averaged 2.99 out of 4.00, against 2.92 for GLM 5.3 and 2.94 for Kimi K3. Against GLM 5.3 it took 46.8% wins, 12.8% ties and 40.4% losses; against Kimi K3, 51.2% wins, 7.9% ties and 40.9% losses.
How to read that margin
A 0.05-point lead on a four-point scale with roughly 40% losses in both matchups is a narrow win, not a rout. Tencent’s own wording — “slightly ahead” — is the honest reading, and it is worth repeating because coverage tends to round these things up.
| Benchmark | Score | What it measures |
|---|---|---|
| GPQA Diamond | 92.3 | Graduate-level science questions |
| Terminal-Bench 2.1 | 85.4 | Command-line task completion |
| MCP-Atlas | 83.7 | Tool calling over connected servers |
| SWE-bench Multilingual | 82.9% | Real issue fixes across languages |
| Toolathlon-Verified | 74.1 | Multi-step tool workflows |
| SWE-Bench Pro (public) | 65.7 | Harder, contamination-resistant fixes |
| DeepSWE | 64.3 | Long-horizon software engineering |
| SkillsBench V1 | 62.9 | Broad professional skills |
| Humanity’s Last Exam (with tools) | 55.4 | Frontier general knowledge |
| APEX-Agents | 37.1 | Agentic planning and execution |
The human study is the most interesting result in the set, and also the closest.
What Hy4 preview Costs to Call or to Host
Price is where an open-weight release of this size does its real damage to the closed competition, and the published numbers are not close.
The API price list
Through Tencent Cloud, Hy4 preview costs $0.834 per million input tokens and $2.501 per million output tokens, with cache reads at $0.042 per million. The domestic Chinese price list is ¥6.00, ¥18.00 and ¥0.30 for the same three lines.
Against the closed frontier
GPT-5.6 Sol, the closed model it is most often compared against, lists $4.00 per million input and $20.00 per million output on its base tier, rising to $8.00 and $30.00 for large-context requests, with cache reads at $0.40. On output tokens — the expensive half of most agentic workloads — Hy4 preview is roughly an eighth of the price.
Self-hosting on a single node
Because only 49B parameters activate per token, the FP8 variant is designed to serve from one eight-GPU node. Tencent ships prebuilt containers for both vLLM and SGLang, so standing up an endpoint is a single docker run with a tensor-parallel size of eight rather than a bespoke inference project.
The throughput reality check
OpenRouter currently lists a single provider for Hy4 preview — Tencent Cloud — at 26 tokens per second with a P50 latency of 3.77 seconds. That is serviceable for batch and background work and slow for anything a person is watching a cursor blink for.
| Per million tokens | Hy4 preview | GPT-5.6 Sol (base) | GPT-5.6 Sol (large context) |
|---|---|---|---|
| Input | $0.834 | $4.00 | $8.00 |
| Output | $2.501 | $20.00 | $30.00 |
| Cache read | $0.042 | $0.40 | $0.40 |
| Weights available | Yes, Apache 2.0 | No | No |
| Self-hostable | Yes, eight GPUs | No | No |
The output-token gap is the one that changes budgets, so it is worth seeing to scale.
Running Hy4 preview Yourself: The Serving Details
Tencent published the serving configuration rather than leaving it to the community to reverse-engineer, which is the difference between a weights dump and a usable release.
Two supported runtimes
Prebuilt images exist for both vLLM (vllm/vllm-openai:hy4-preview) and SGLang (lmsysorg/sglang:hy4-preview, multi-architecture for x86 and Arm). Both run the FP8 variant at a tensor-parallel size of eight, and both expose an OpenAI-compatible endpoint.
Speculative decoding is on by default
The vLLM recipe enables three speculative tokens using the MTP method with the FLASHMLA_SPARSE attention backend. SGLang uses NEXTN with three steps, an EAGLE top-k of one and four draft tokens. Speculative decoding is not an optional extra here; it is part of how the published throughput is achieved.
Tool calling and reasoning modes
Both runtimes ship parsers for the model’s tool-call and reasoning formats, with automatic tool choice enabled. Reasoning defaults to a high-effort chain of thought, and a no_think option is available for direct answers. Tencent recommends a temperature of 0.9 and a top-p of 1.0.
Fine-tuning and compression
A complete fine-tuning pipeline ships with the model, and Tencent points quantisation work at AngelSlim, its own open compression toolkit covering low-bit quantisation and speculative sampling. If you are planning custom model development on top of an open checkpoint, those two pieces matter as much as the benchmark table.
| Setting | vLLM | SGLang |
|---|---|---|
| Container image | vllm/vllm-openai:hy4-preview | lmsysorg/sglang:hy4-preview |
| Tensor parallel size | 8 | 8 |
| Speculative decoding | MTP, 3 tokens | NEXTN, 3 steps |
| Attention backend | FLASHMLA_SPARSE | Runtime default |
| Tool-call parser | hy_v4 | auto |
| Architectures supported | x86 | x86 and Arm |
How Hy4 preview Changes the Open-Weight Race
The competitive question is no longer whether open weights can approach frontier quality. It is what a permissive licence does to the pricing power of the labs that keep their weights closed.
Apache 2.0 is the real story
A 770B model under Apache 2.0 can be deployed commercially, modified, distilled and redistributed with no negotiation. That is a genuinely different proposition from a “community” licence with revenue thresholds or use restrictions, and it is the part of the Hy4 preview release with the longest half-life.
The Chinese cadence
Tencent’s release follows a run of open-weight launches from Alibaba, Moonshot, Z.ai and DeepSeek — several of which Hy4 preview is explicitly benchmarked against. Tencent also credits DeepSeek and GLM for the attention and residual ideas it builds on, which is an unusually direct acknowledgement of how fast this particular field compounds.
What the closed labs still hold
On the numbers published so far, GPT-5.6 Sol still leads on the benchmarks where both report: 94.6 versus 92.3 on GPQA Diamond, and 88.8 versus 85.4 on Terminal-Bench 2.1. The closed model’s scores also carry independent leaderboard measurement, which none of Hy4 preview’s do yet.
Where that leaves buyers
For a business, the choice is now rarely “best model wins”. It is a portfolio question about which workloads justify frontier pricing and which are served perfectly well by an open checkpoint you can pin, audit and host — the same calculation behind running an AI agent entirely on local hardware.
The Caveats Worth Naming Before You Deploy
Enthusiasm about a release this size tends to outrun the evidence. Four things about Hy4 preview deserve stating plainly.
Every number is vendor-reported
There is no independent confirmation of any Hy4 preview benchmark score at the time of writing. The blind evaluation was run by Tencent’s own staff, on Tencent’s own task set, against models Tencent selected. That does not make the results wrong; it makes them unverified.
Tencent published its own limitations
The team lists two known issues: the model spends longer than necessary reasoning through complex tasks, and it has a tendency to over-verify its own work. Both are cost problems as much as quality problems, because both burn output tokens on an inference-priced endpoint.
One provider, early days
OpenRouter shows a single provider serving Hy4 preview. A single-provider model has no routing fallback, and its published uptime and availability figures cover only the first days of service. Redundancy for anything important means either a second host or your own deployment.
The self-optimisation claim needs care
Tencent says the model took part for the first time in the automated optimisation of its own training methods, data strategies, evaluation frameworks and low-level operators, contributing to a 31.8% increase in inference throughput over baseline. That is an engineering-productivity claim about the build process, not evidence of autonomy, and it should not be read as more than that.
What Hy4 preview Means for Your Business
Most organisations will not download a 770B checkpoint. They will still feel this release through pricing, through vendor negotiations, and through what becomes normal to expect from a long-context model.
Where a million-token window earns its keep
The window pays off when the context genuinely cannot be chunked: whole-repository code review, contract sets where a clause in appendix four changes the meaning of clause two, incident timelines assembled from many logs, or due-diligence packs. If your current pipeline spends more engineering effort on retrieval than on the answer, Hy4 preview is worth a trial.
Where it does not
Short, high-volume, latency-sensitive calls are the wrong workload. At 26 tokens per second on the public endpoint, an interactive assistant will feel slow, and a small dense model will beat it on cost per request for classification or extraction.
Questions to ask before switching
Ask where the endpoint runs and under whose jurisdiction, whether your data-processing terms permit it, what your fallback is when a single provider degrades, and how you would evaluate quality on your own tasks rather than on a leaderboard. Our AI models and tools hub tracks the releases these questions keep arriving with.
Run your own evaluation set
The cheapest useful thing any team can do this week is assemble fifty real tasks from its own backlog and score two or three candidate models on them. That set outlives every individual release, and it converts a benchmark table into a decision you can defend.
Hy4 preview: Frequently Asked Questions
Is Hy4 preview really open source?
The weights are published under Apache 2.0, which permits commercial use, modification and redistribution. Tencent has not published the training data or the full training code, so “open weights” is the more precise description.
What hardware do I need to run Hy4 preview?
Tencent’s own recipes target a single node with eight GPUs and a tensor-parallel size of eight, serving the FP8 quantised variant. That is the documented configuration; smaller footprints would need further compression work.
How does Hy4 preview compare with GPT-5.6 Sol?
On the two benchmarks where both publish, the closed model leads — 94.6 to 92.3 on GPQA Diamond and 88.8 to 85.4 on Terminal-Bench 2.1. Hy4 preview costs roughly an eighth as much per million output tokens and can be self-hosted.
Can Hy4 preview really use a million tokens of context?
The architecture supports a 1,048,576-token window, and sparse attention is what makes that economically viable. No independent long-context evaluation has been published, so validate recall on your own documents.
Is Hy4 preview available in the UK?
Yes. It is reachable globally through OpenRouter and Tencent Cloud TokenHub, and the weights can be downloaded and hosted anywhere. Check your data-residency obligations before sending regulated data to any hosted endpoint.
What does the “preview” label imply about stability?
Tencent describes it as an early version with real headroom left and says it will keep iterating quickly. Expect the model to be updated, and pin a specific revision if reproducibility matters to you.
References and Further Reading
Tencent: Tencent Releases and Open-Sources Tencent Hy4 preview
Hugging Face: tencent/Hy4-preview model card
Hugging Face: tencent/Hy4-preview-FP8
GitHub: Tencent-Hunyuan/Hy4-preview
OpenRouter: Hy4 preview API pricing and providers
ModelScope: Tencent-Hunyuan/Hy4-preview
vLLM Recipes: Hy4-preview deployment guide
arXiv: DeepSeek Sparse Attention
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.