PrismML wants to prove that a reasoning model does not have to be large to be good. On 17 September 2026 the Caltech spin-out released Ternary Bonsai 2 27B, a compressed version of Alibaba’s open-weight Qwen3.8 27B that fits in 5.9 GB, small enough for a well-specified PC and possibly a high-end smartphone. The company says it keeps 98.2% of the original model’s aggregate benchmark score while using more than nine times less memory.
TechCrunch framed the release as a bet that could “change how we all use AI”, and the framing is fair. If capable models run on the device you already own, the economics, privacy and latency of everyday AI all shift at once. That is a large claim for a startup that has raised only a seed round, so this article tests it against the published numbers.
We set out what PrismML actually shipped, where the compressed model keeps and loses capability, the memory arithmetic behind the headline, the Apple talks, the points where early coverage and the company’s own post disagree, and what it all means for organisations planning their AI strategy.
Table of contents
- What PrismML Released This Week
- Bonsai 2 27B by the Numbers
- Where PrismML Keeps and Loses Capability
- How Small Is 5.9 GB? The Memory Arithmetic
- Speed, Energy and the Target Hardware
- The Apple Question and PrismML’s Funding
- Where the Coverage and the Company Disagree
- How the Bonsai Line Has Evolved Since March
- How PrismML Compares With Other Ways to Shrink a Model
- What Analysts Say Still Needs Proving
- What PrismML Means for Businesses
- PrismML FAQ
- References
What PrismML Released This Week
The release is a single model with a long name. Ternary Bonsai 2 27B is a large language model in the second generation of the 27-billion-parameter Bonsai line, and it is built on a stronger base model than the first.
The model in one paragraph
PrismML took Qwen3.8 27B, a widely used open-weight model from Alibaba, and re-expressed its weights in a far more compact form. The result supports a 262,000-token context window, accepts text and image input, and is published under the permissive Apache 2.0 licence. Weights are on Hugging Face in GGUF and MLX formats, so the model runs through the usual local tooling on NVIDIA and Apple hardware.
What “ternary” means
Most models store each weight as a 16-bit number. PrismML’s approach restricts each weight to one of three values: minus one, zero or plus one. A small full-precision scaling factor is stored per group of weights so the model can recover the right magnitudes. PrismML puts the effective cost at 1.76 bits per weight once those scales are included, against 16 bits for the original.
Why the name says “2”
The first Bonsai 27B arrived about two months earlier and retained roughly 95% of its base model’s aggregate score. The new release moves to Qwen3.8 as the base and lifts retention above 98%, which PrismML describes as practically lossless. The company also publishes smaller Bonsai variants, and it says those have been downloaded a further 2.6 million times.
Bonsai 2 27B by the Numbers
The specification is easiest to judge side by side with the model it compresses. Every figure below comes from the PrismML announcement or is arithmetic on those figures.
| Specification | Qwen3.8 27B (original) | Ternary Bonsai 2 27B |
|---|---|---|
| Bits per weight | 16 | 1.76 effective |
| Weight values | Full 16-bit range | -1, 0 or +1, with FP16 group scales |
| Model footprint | About 54 GB | 5.9 GB |
| Aggregate benchmark score | 85.4 | 83.9 (98.2% retained) |
| Context window | Base model’s | 262K tokens |
| Input types | Text and image | Text and image |
| Licence | Open weights | Apache 2.0 |
The 54 GB figure is arithmetic, not a claim
Twenty-seven billion weights at 16 bits, or two bytes each, is 54 GB. CNBC reported the same figure in July, when PrismML said it had squeezed the earlier Qwen-based model “from roughly 54 GB to less than 4 GB”. Dividing 54 by 5.9 gives a ratio of about 9.2, which matches the “more than 9x” in the announcement.
Why 1.76 bits and not 1.58
A value that can take three states carries log2(3), or about 1.58 bits of information, which is why ternary models are often called 1.58-bit models. We explained that family in our earlier guide, What Is Ternary Bonsai?. The extra fraction of a bit in the new release pays for the FP16 scaling factors and for packing trits into bytes. Multiplying 27 billion by 1.76 bits and dividing by eight gives 5.94 GB, so the published footprint is internally consistent.
Where PrismML Keeps and Loses Capability
An aggregate score hides where the damage falls, and the company deserves credit for publishing the breakdown. The table compares the compressed model with its base and with the previous-generation Qwen3.6 27B.
| Capability group | Bonsai 2 27B | Qwen3.8 27B | Qwen3.6 27B |
|---|---|---|---|
| Agentic and tool calling | 77.57 | 79.74 | 80.05 |
| Coding | 81.58 | 82.17 | 82.57 |
| Instruction following | 82.66 | 81.25 | 74.53 |
| Knowledge and reasoning | 83.95 | 86.66 | 84.71 |
| Math | 96.57 | 97.06 | 94.64 |
| Vision | 78.59 | 81.64 | 79.82 |
| Overall | 83.9 | 85.4 | 83.6 |
Retention by capability group
Dividing each Bonsai 2 score by the matching Qwen3.8 27B score shows how much of the original survives compression in each area.
Knowledge is where compression bites
The weakest groups are vision and knowledge and reasoning, each losing three to four points of retention. That matches what chief executive Babak Hassibi told CNBC in July: factual recall tends to weaken before reasoning, math and coding. It makes sense. Facts are stored diffusely across many weights, and coarser weights blur them first.
Instruction following actually improved
Bonsai 2 scores higher than its own base on instruction following, 82.66 against 81.25. Compression noise alone would not do that, so the likely explanation is some post-training during the compression process. PrismML’s whitepaper describes the method; the announcement does not break it out.
The generational comparison is the striking one
The compressed model’s overall 83.9 edges past the full-precision Qwen3.6 27B at 83.6. In other words, this release at under 6 GB beats last generation’s uncompressed model at roughly 54 GB on the same suite. That is the strongest single argument in the release.
How Small Is 5.9 GB? The Memory Arithmetic
Memory, not raw compute, is what keeps large models off phones and laptops. The chart shows the weight footprint of a 27-billion-parameter model at common precisions, calculated as parameters multiplied by bits per weight.
What fits where
A 13.5 GB 4-bit model already needs a workstation GPU or a high-memory Mac. At 5.9 GB, the new file fits comfortably on a 12 GB or 16 GB consumer graphics card with room left for the context cache, and within the memory of the latest premium phones. “Possibly a high-end smartphone”, in TechCrunch’s words, is the honest phrasing: the long context window needs memory of its own, so the full 262K tokens will not be usable on a phone.
Why memory prices make this timely
CNBC noted that Morgan Stanley expects Apple’s cost per bit of DRAM to rise by roughly 190% year on year in fiscal 2027. When memory is getting dearer, a technique that cuts a model’s memory need by nine times is worth more than it would have been a year ago.
Speed, Energy and the Target Hardware
Smaller weights also move faster through a chip’s memory bus, which is usually the bottleneck when a model generates text. PrismML published three performance figures.
| Hardware | Published figure | What it means in practice |
|---|---|---|
| NVIDIA GeForce RTX 5090 | Up to 143 tokens per second | Faster than most people read; suits coding-agent loops |
| Apple M5 Max | 46.8 tokens per second | Comfortable interactive speed on a laptop |
| NVIDIA RTX 4090 | 0.714 mWh per token | 40% more energy-efficient than a full-precision 8B model |
The energy claim is the unusual one
Plenty of compression projects publish speed. Fewer publish energy per token, and this comparison is pointed: a 27B-class model using less energy per token than a full-precision 8B model. For battery-powered devices, and for datacentres paying for power, that is the number to watch once independent testers reproduce it.
Platform coverage
The model runs on NVIDIA GPUs through CUDA and on Mac, iPhone and iPad through Apple’s MLX framework, using custom low-bit kernels. Our AI models, tools and releases hub tracks how quickly such kernels reach the mainstream local runners, which is usually what decides whether a format gets used.
The Apple Question and PrismML's Funding
Two threads in the coverage have little to do with benchmarks and a lot to do with whether PrismML becomes important.
Apple is evaluating the technology
In July, Hassibi told CNBC that Apple and other companies were measuring the models’ speed, energy efficiency and on-device performance. “They’re really evaluating our technology right now,” he said, calling the talks very early. Asked again this week by TechCrunch, he declined to comment. Apple has not commented at any point. The fit is obvious: Apple wants more of Siri to run on the iPhone, and the most capable models do not fit.
The seed round figure depends on who you read
TechCrunch reports a $22.25 million seed round. CNBC reported in July that PrismML “raised a $16.25 million seed round” in March. The two figures differ by exactly $6 million, which would be consistent with a later extension, but neither outlet says so and PrismML’s own posts give no total. Backers named across the sources are Khosla Ventures, Cerberus Capital, Caltech and Google, with continuing support from Samsung.
The intellectual property sits with Caltech
The technique came out of Hassibi’s research group. Caltech owns the underlying patents and licenses them exclusively to PrismML, according to CNBC. Berkeley professor Ion Stoica, co-founder of Databricks, is an adviser.
Where the Coverage and the Company Disagree
Early reports compress a lot of detail, and three points are worth correcting against the primary source.
March or July?
TechCrunch says the first Bonsai was “released a couple of months ago in March” and matched 95%. PrismML’s post says the first Bonsai 27B came out “two months ago”, which places it in July, consistent with CNBC’s 14 July report. The March date belongs to the company’s seed round; its public launch of the original 1-bit family followed in early April.
“Three values” is not the whole story
Describing the weights as simply plus one, minus one or zero is accurate but incomplete. The FP16 group scales are what let a ternary model recover useful magnitudes, and they are why the effective cost is 1.76 bits rather than 1.58.
The download figures are the company’s
The “over 11 million” downloads for the original model and 2.6 million for smaller models are PrismML’s figures. When we checked Hugging Face on 18 September, the new GGUF repository alone showed about 406,000 downloads a day after release, which is a strong start but a different measure.
How the Bonsai Line Has Evolved Since March
The company has moved quickly for a seed-stage startup. Setting the public milestones in order makes the pace, and the gaps in what is known, easier to see.
| When | Milestone | Source |
|---|---|---|
| March 2026 | Seed round closes; $16.25 million per CNBC, $22.25 million per TechCrunch | CNBC, TechCrunch |
| Early April 2026 | Emerges from stealth with a 1-bit Bonsai model family | HPCwire |
| 14 July 2026 | Bonsai 27B, described as the first 27B-class model to run on a phone; about 95% retention | PrismML, CNBC |
| 17 September 2026 | Ternary Bonsai 2 27B on Qwen3.8; 5.9 GB; 98.2% retention | PrismML |
| Next few months | Models “in the several-hundred-billion-parameter range”; Gemma also planned | Hassibi to TechCrunch and CNBC |
Six months from seed to second generation
Two 27B-class releases within roughly nine weeks is a fast cadence. Part of that speed comes from the approach itself: the startup does not train base models from scratch, it compresses strong open-weight ones as they appear. Each new Qwen or Gemma release is therefore raw material rather than competition, which is a useful position for a small company to occupy.
The July model was smaller than this one
CNBC’s July report described a model “less than 4 GB”, while Bonsai 2 is 5.9 GB. Two compressed versions were released in July, and the sub-4 GB figure most likely refers to the pure 1-bit variant. The 95% to 98% improvement the company cites compares ternary with ternary, so it is a like-for-like gain rather than one bought with extra bits.
The open question is scale
Everything published so far sits at 27 billion parameters. The claim that bigger models compress more easily is plausible, since larger networks carry more redundancy, but it is untested in public. The first several-hundred-billion-parameter release will show whether the method holds where the stakes are highest: frontier-class models that today need a rack of datacentre GPUs.
How PrismML Compares With Other Ways to Shrink a Model
PrismML is not alone. TechCrunch points to Multiverse Computing, which has raised far more money for its own compressed models. The broader field offers several approaches with different trade-offs.
| Approach | How it shrinks the model | Typical trade-off |
|---|---|---|
| 4-bit post-training quantisation | Rounds existing weights to 16 levels | Cheap and widely supported; about 4x smaller |
| Ternary or 1-bit weights (PrismML) | Three weight values plus group scales | About 9x smaller; needs custom kernels |
| Tensor-network compression (Multiverse) | Factorises weight matrices into smaller pieces | Fewer parameters; quality varies by layer |
| Distillation into a smaller model | Trains a new small model on a large one’s outputs | Fast and small, but a different, weaker model |
Why ternary is attractive
With only three values, most multiplications become additions, subtractions or skips. That is why research such as Microsoft’s BitNet b1.58 work drew so much attention in 2024, and why hardware makers are interested. The catch is software: every runtime needs purpose-built kernels, and a format only matters once those kernels are everywhere.
What Analysts Say Still Needs Proving
The company’s claims come from its own benchmark suite. Independent analysts quoted by CNBC in July listed what would settle the question.
Scale, battery and reliability
Tarun Pathak of Counterpoint Research said long prompts, battery drain during multitasking and reliability over millions of requests would be critical: “The ultimate test will be millions of queries, thousands of device combinations and robust testing at scale.” Phil Solis of IDC flagged power as the biggest open question for a model running constantly in the background.
It will not end demand for chips
Gil Luria of D.A. Davidson argued that shrinking models moves chips rather than removing them: “You’re still going to need the GPU, and you’re still going to need the memory.” Cheaper inference also tends to increase use, a pattern familiar from every previous efficiency gain in computing.
The harness matters as much as the weights
TechCrunch makes a fair point that a 2% benchmark gap rarely decides real-world usefulness, because the software around a model shapes accuracy as much as the model does. For a local deployment, retrieval, tool design and prompt structure will matter more than the last two points of benchmark retention.
What PrismML Means for Businesses
For most organisations the practical question is not whether to adopt Bonsai 2 tomorrow, but which workloads could move off the cloud if models this capable run locally.
Privacy and data residency
“It’s also going to be private, because you’re not going to send it to the cloud,” Stoica told TechCrunch. A model on a laptop never transmits the document it is summarising, which simplifies data-protection reviews for contracts, HR files and client records. It does not remove the need for endpoint security: a model on a stolen device is only as protected as the disk it sits on, so standard cybersecurity controls still apply.
Hybrid routing
PrismML itself describes hybrid systems in which local models handle sensitive or frequent tasks and escalate to the cloud selectively. That design suits many firms: routine drafting and classification locally, hard reasoning in a hosted frontier model. The cost saving is real, but so is the complexity of running two tiers.
Where to start
Pilot on work where a small error is cheap and data sensitivity is high, such as internal search, summarisation and first-draft coding assistance. Measure on your own tasks, not on the vendor’s suite, and budget for the operational work of updating models across many devices.
What to watch next
Hassibi says the next models, “hopefully in the next couple of months”, will be “in the several-hundred-billion-parameter range”, and that larger models should be easier to compress without losing intelligence. Google’s Gemma is also in the pipeline, according to CNBC. If a several-hundred-billion-parameter model reaches workstation size with similar retention, the case for local AI in the enterprise changes substantially.
PrismML FAQ
What is PrismML?
PrismML is a Caltech spin-out led by professor Babak Hassibi that compresses open-weight LLMs using ternary and 1-bit weights so they can run on PCs and phones.
What is Ternary Bonsai 2 27B?
It is a compressed version of Alibaba’s Qwen3.8 27B that fits in 5.9 GB, supports a 262K-token context and image input, and keeps 98.2% of the original’s aggregate benchmark score.
Can it run on a phone?
PrismML supports iPhone and iPad through Apple’s MLX framework. On a phone, memory limits how much of the long context window can be used, so expect shorter working contexts than on a PC.
Is it free to use?
Yes. The weights are released under the Apache 2.0 licence, which allows commercial use. The company also offers paid work tailoring models to specific applications.
Is Apple working with PrismML?
Hassibi said in July that Apple was evaluating the technology. He declined to comment this week, and Apple has not commented. No deal has been announced.
References
PrismML hopes its tiny LLM will change how we all use AI (TechCrunch)
Introducing Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint (PrismML)
Bonsai 2 model collection (Hugging Face)
Apple in talks with PrismML on AI compression for iPhone (CNBC)
Multiverse Computing pushes its compressed AI models into the mainstream (TechCrunch)
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (arXiv)
What Is Ternary Bonsai? 7 Critical Facts About PrismML’s 1.58-Bit Model Family
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.