Bonsai 27B: A 27-Billion-Parameter Model That Fits on an iPhone

PrismML shipped a 27-billion-parameter language model on July 14, 2026, and the smaller variant occupies 3.9 gigabytes of storage on an iPhone.

The compression that put a 27-billion-parameter model in a pocket

For most of the past three years, running a 27-billion-parameter language model at home was not really a question. It required a datacenter GPU, or at minimum a workstation with 32 to 64 gigabytes of GPU memory. A full-precision FP16 copy of Alibaba's Qwen3.6 27B, the open-weight model that Bonsai is built from, occupies roughly 54 gigabytes on disk. That is more storage than most consumer laptops set aside for a single file, and it is well beyond anything a phone can hold in memory.

On July 14, 2026, a startup called PrismML released Bonsai 27B, a 27-billion-parameter language model that fits into 3.9 gigabytes and runs on an iPhone. The company shipped two variants: a 1-bit build at 3.9 gigabytes and a ternary build at 5.9 gigabytes. Both were open-sourced under the Apache 2.0 license, which lets developers and companies fine-tune the model and ship it inside commercial products at no cost. PrismML, founded by researchers from Caltech and backed by Khosla Ventures, Cerberus, Google, and continuing support from Samsung, distributed the weights on Hugging Face and GitHub, with an API preview available through Together AI.

The model is multimodal. It reads images as well as text, using a compact 4-bit vision tower that ships alongside the language weights. And it supports a 262,000-token context window, enough room for the text of a full book in a single prompt.

How PrismML trained Bonsai to survive at 1.125 bits per weight

The technique behind the compression is Quantization-Aware Training, or QAT. Instead of training a model at full precision and then squeezing it afterward, PrismML applied the low-bit constraint from the very first forward pass. The 1-bit variant stores each weight as one of two values, minus one or plus one, at an effective 1.125 bits per weight. The ternary variant stores each weight as minus one, zero, or plus one, at an effective 1.71 bits per weight. In both cases, the model learned during training to route information through a much narrower channel without collapsing.

QAT keeps full-precision gradients in the backward pass while enforcing the discrete constraint in the forward pass. One shared FP16 scale is applied per group of 128 weights, giving the compressed values enough dynamic range to survive gradient descent. The compression is end-to-end: embeddings, attention layers, MLP blocks, and the language-model head are all quantized, not just the transformer body.

The result is a model whose 1-bit variant is more than thirteen times smaller than the full-precision original. In PrismML's own phrasing, "The Bonsai 27B 1-bit variant is smaller than most full-precision 2B models." That is the pitch, and it is a direct comparison the company invites.

To reach the full 262,000-token context on-device, PrismML also compressed the KV cache. Moving from an FP16 cache to a 4-bit cache reduces memory for the full window from 17.2 gigabytes to 4.3 gigabytes. At a 100,000-token context with a full-precision cache, the 1-bit variant peaks at 11.6 gigabytes of memory, and the ternary variant peaks at 14.7 gigabytes.

What the benchmarks say

The important question is what the model gives up in exchange for that footprint. Across a 15-benchmark evaluation suite run through EvalScope, the FP16 baseline scored 85.07 on average. The ternary Bonsai scored 80.49, and the 1-bit Bonsai scored 76.11. That works out to 94.6 percent of full-precision capability for the ternary variant and 89.5 percent for the 1-bit variant.

Broken out by task, the FP16 baseline scored 95.33 on math, 88.74 on coding, 83.15 on knowledge, 80.00 on agentic tool-calling, and 72.61 on vision. The ternary variant scored 93.40, 85.96, 76.96, 74.01, and 65.19 on the same categories. The 1-bit variant landed at 91.66, 81.88, 73.39, 66.03, and 59.57. The gap widens most on agentic and vision workloads, where the compressed models give up the most ground, and it narrows sharply on math, where both quantized builds hold within a few points of the full-precision baseline.

Where the model runs, and how fast

The speed numbers are the ones that read like a category change. On an iPhone 17 Pro Max, Bonsai 27B generates about 11 tokens per second, roughly the pace of a fast human reader. On an Apple M5 Pro laptop, the ternary variant runs at about 26 tokens per second. On an M5 Max, the 1-bit variant hits 66 tokens per second. On an NVIDIA RTX 5090 desktop GPU, the 1-bit variant reaches 163 tokens per second, and the ternary variant runs at 134 tokens per second. On an H100 datacenter GPU with speculative decoding using PrismML's DSpark technique, the 1-bit variant reaches 143.8 tokens per second, a 1.37× speedup over the standard decode path.

The iPhone number is the one worth sitting with. Eleven tokens per second is close to conversational latency for chat-style workloads and comfortably above reading pace for summarization. It is happening on a phone, entirely on-device, from a model whose full-precision original would not fit on the same phone at all.

Why on-device inference changes the default

PrismML describes Bonsai 27B in its announcement post as "The first 27B-class model to fit a phone." The company's release brief calls it "A 27-billion-parameter multimodal reasoning model that fits in 3.9 gigabytes and runs on an iPhone." And its release note underlines the licensing choice: "Everything is open-sourced under Apache 2.0."

The technical claim is narrow and specific. It is not a claim that phones now replace datacenters, and it is not a claim that the compressed model matches its full-precision parent on every benchmark. It is a claim that a 27-billion-parameter model, quantized end-to-end and trained with the low-bit constraint in place, can retain roughly 89.5 percent of the parent model's capability across a 15-benchmark suite while being over thirteen times smaller on disk.

If that result holds up in production use, the default location of inference changes. Summarizing a long document, asking a coding question, or extracting a table from a photograph no longer needs to be a round-trip to a hosted API. Latency drops to whatever the phone can compute. Data stays on the device. The airplane cabin, the rural road with no signal bar, and the laptop on a kitchen counter all become viable places for a 27-billion-parameter model to do useful work.

That is a research and engineering result, released today, under a license that permits commercial use. What developers and companies do with it will be visible in the products that ship over the next several months.

Sources

• PrismML weights on Hugging Face: https://huggingface.co
• PrismML repository on GitHub: https://github.com
• Together AI API preview: https://together.ai