TypeSafe AI spent one week turning a strange launch into the fastest model adoption Vercel has ever recorded. Jev, the decision model from a founder who helped build ChatGPT, reached nearly 13% of Vercel’s paid teams within twenty-four hours of appearing on the AI Gateway — more than double any previous model launch on that platform, and roughly six times the first-day share of Claude Fable 5.1. The company briefly stopped being able to serve its own API because demand outran capacity.
That is a peculiar outcome for a model that cannot write a sentence. Jev does not generate text, cannot explain itself, and refuses to produce anything except typed values with probabilities attached. Developers did not adopt it in spite of that; they adopted it because of it.
This article is about the week after the launch rather than the launch itself. We covered what the model is, how it is built and what its benchmark claims measure in our Jev model launch analysis. What follows is the evidence that arrived afterwards: the adoption curve, what three named engineering teams measured when they swapped a language model out, the independent benchmark that shows a 32-point accuracy swing depending on how you ask, and the calibration numbers that decide whether any of it survives contact with your data. If your stack already routes artificial intelligence calls for classification, this is the week the economics changed.
Table of contents
- What TypeSafe AI Shipped, in One Paragraph
- The TypeSafe AI Adoption Curve That Made Developers Notice
- What Vercel Measured When It Swapped In TypeSafe AI
- Two More Teams, and the Critique That Came With Them
- The Accuracy Result Every TypeSafe AI Evaluation Should Reproduce
- Where TypeSafe AI’s Calibration Slips Out of Distribution
- The TypeSafe AI Integrations That Landed Within Three Days
- The Capacity Story Behind TypeSafe AI’s Sudden Demand
- A Shadow Evaluation Plan for TypeSafe AI
- Why a ChatGPT Inventor Went Looking for This
- What to Watch Over the Next Two Quarters
- References and Further Reading
What TypeSafe AI Shipped, in One Paragraph
TypeSafe AI’s Jev takes a piece of state and a map of typed questions, answers all of them in a single parallel pass, and returns values rather than prose. There is no chat interface, no chain of thought, no image input and no generated string to parse. The current model is jev-1.13.0, with jev-latest and jev-preview both pointing at it.
The three TypeSafe AI question types and nothing else
A Choice picks one option from a set you defined and returns probabilities across all of them. A Score rates the state against an ordered rubric. A Noul answers a yes-or-no question but deliberately returns a probability between 0 and 1 rather than a boolean, so a 0.51 announces itself as a coin flip instead of hiding as true.
The commercial shape is the story
TypeSafe AI charges input at $0.042 per million tokens, which the documentation also states as $42 per billion — a unit change that tells you who the pricing is aimed at. Output tokens are not metered at all. That combination is what makes the deployment patterns below affordable rather than merely interesting.
What the TypeSafe AI documentation now says about limits
| Property | Stated value |
|---|---|
| Model ID | jev-1.13.0 (jev-latest, jev-preview) |
| Input price | $0.042 per million / $42 per billion |
| Output price | Not metered |
| Context budget | 64k per request; 32k for state plus longest question |
| Throughput limit | 250,000 tokens per second |
| Request limit | 1,200 requests per minute |
| Input types | Text only: string, JSON object, array of text |
| Fine-tuning | None; customisation via state, instructions, criteria |
| Training on your data | No customer data retained |
The TypeSafe AI Adoption Curve That Made Developers Notice
Vercel published its own gateway telemetry on 18 September, and it is the most concrete evidence anyone has released about how fast TypeSafe AI’s model spread. The numbers describe paid teams that sent traffic to the model, measured against every other model launch the gateway has hosted.
Past every comparison model by hour twelve
TypeSafe AI overtook every comparison model within its first twelve hours. By hour eighteen it was at roughly 10% of paid teams, and by hour twenty-four it reached nearly 13%. Vercel’s framing is that it “reached more than twice as many paid teams as any previous model launch” — and that every other recent launch stayed below 7% after a full day.
Two times GPT-5.6, six times Fable
The relative figures are more striking than the absolute one. At the same point in the curve, TypeSafe AI held twice the share of the GPT-5.6 family and more than six times the share of Claude Fable 5.1. Working backwards from the stated 13%, that puts the GPT-5.6 family near 6.5% and Fable 5.1 at roughly 2.2% of paid teams on their equivalent first days.
What thirteen percent does not tell you
A team counts once it sends a request. Nothing in the published figures separates a production workload from an engineer pasting a snippet into a scratch project on launch day, and that distinction is the whole difference between a curiosity and a migration. Treat the curve as evidence of unusually broad evaluation, not of unusually fast production commitment. The retention number nobody has published yet is the one that matters.
What Vercel Measured When It Swapped In TypeSafe AI
The single most-quoted data point of the week came from Pranit Sharma, a software engineer at Vercel, speaking to TechCrunch. His team ran a classifier that reviews shell commands for safety before execution — a small, high-volume, entirely decision-shaped workload.
Five to eighteen times faster, and more accurate
That classifier had been running on OpenAI’s ChatGPT Luna 5.6. After swapping it for the TypeSafe AI model, Vercel reported results five to eighteen times more quickly, with greater accuracy as well. Both halves matter: latency gains alone would be a trade, and an accuracy gain on the same task removes the usual objection that a cheap model is a worse model.
Why a command-safety check is the ideal shape
The task has a fixed answer space, needs no explanation, runs on every command, and sits directly in the user’s latency path. A frontier model asked to do it produces a paragraph of reasoning nobody reads and bills for every token of it. This is the same logic teams already apply when they push repetitive steps out of a language model and into workflow automation pipelines.
The four jobs Vercel’s own post names
| Job | What the model decides | Why it fits |
|---|---|---|
| Tool or subagent selection | Which handler takes this step | Fixed option set, runs every turn |
| Workflow continuation | Carry on, stop, or escalate | Boolean with a usable probability |
| Urgency or risk scoring | Where this sits on a rubric | Ordered levels, no prose needed |
| Output verification | Whether a generation breaks a rule | Cheap enough to run on every response |
Two More Teams, and the Critique That Came With Them
Vercel was not the only named TypeSafe AI deployment. The other two are more interesting, because one of them lost on accuracy and adopted the model anyway, and the other came with a warning attached.
Bryo AI: slower on accuracy, decisive on economics
Nikhil Mudholkar, CTO of Bryo AI, tested the TypeSafe AI model against Gemini for classifying business emails. Gemini was slightly more accurate. It was also ten to twenty times more expensive. What decided it for him was neither figure: “it is the only one that hands back a real probability which makes it ideal for automating workflows!!” The probability, not the verdict, was the product.
Earendil: the hallucination problem moves, it does not vanish
Armin Ronacher, CTO of Earendil — which builds the open-source agent harness Pi — supplied the week’s sharpest caveat. “At the end of the day, it delegates the hallucination problem a little bit to the user,” he said. His elaboration is the operating manual: “The user has to say, okay, if this only comes back with 50% probability, maybe this is a coin toss, and I disregard it. But if it’s 95%, sure, then I can do something with it.”
Model routing as the next obvious TypeSafe AI use
Ronacher also named routing as the use case he expects to matter. Predicting whether a workload needs an expensive model is valuable, but doing that prediction with an expensive model defeats the purpose. A decision that costs a rounding error and returns in milliseconds makes real-time sorting viable for the first time. He expects imitators: the field, he suggested, has not had to be creative while language model inference stayed cheap and subsidised.
| Team | Task | Replaced | Reported result |
|---|---|---|---|
| Vercel | Shell command safety review | ChatGPT Luna 5.6 | 5–18x faster, greater accuracy |
| Bryo AI | Business email classification | Gemini | 10–20x cheaper, slightly less accurate |
| Earendil | Harness design commentary | — | Routing use case; confidence is the caller’s job |
The Accuracy Result Every TypeSafe AI Evaluation Should Reproduce
The most useful independent TypeSafe AI work published this week is a phishing benchmark that ran 2,000 synthetic emails with labels drawn from URL reputation feeds. Its finding is not that TypeSafe AI is good or bad. It is that the same model on the same data scores 62.6% or 95.0% depending entirely on how the question is framed.
62.6% when you ask the whole question at once
Asked the single question “is this phishing?”, TypeSafe AI scored 62.6% against 81.3% for Haiku, a gap that clears statistical significance comfortably (McNemar p < 0.0001). The breakdown is worse than the headline: it caught only 43.2% of the phishing emails while incorrectly flagging 18.0% of the legitimate ones. Used that way, it is not a usable filter.
95.0% when you split it five ways
The same benchmark then decomposed the task into five atomic signals — shortened URLs, free hosting, free email addresses claiming organisational affiliation, and two more of the same shape. A logistic regression fitted on 1,000 labelled emails and evaluated on the other 1,000 scored 95.0% for Jev against 93.2% for Haiku, a difference with no statistical significance at all (p = 0.063).
Why decomposition is the whole TypeSafe AI technique
This is the documented usage pattern working exactly as specified, and it is where most first evaluations of TypeSafe AI’s model will go wrong. A System One model is built to answer one narrow, well-scoped question at a time. “Is this phishing?” is a composite judgment that bundles five sub-judgments and an inference rule. Ask it as one question and you get a guess; ask the five parts and combine them in your own code, and the combination outperforms the frontier model it was losing to.
The TypeSafe AI cost and latency that came with it
At list pricing the same benchmark put 1,000 emails at $0.038 through Jev, against $0.462 for Haiku on the single verdict and $1.02 for the five signals — roughly 12x and 27x respectively. Median latency measured from France was 239ms against 687ms. The decomposed run is the more expensive configuration on both models, and it is still the cheap one.
Where TypeSafe AI's Calibration Slips Out of Distribution
Calibration is the load-bearing claim of the entire TypeSafe AI product, because a probability you cannot trust is worse than no probability at all. The same independent testing put numbers on it, and they are not flattering once you leave the training distribution.
An ECE of 0.107 against a 0.024 noise floor
On 900 synthetic support tickets, expected calibration error came in at 0.107 — about 4.4 times the 0.024 noise floor of the measurement itself. That is not a rounding artefact. It means the stated probabilities drift materially from observed frequencies on data the model was not tuned against.
The drift is not uniform across answer types
Yes-or-no answers came back underconfident, at a fitted temperature of 0.66. Choice and Score answers came back overconfident, at 3.29 and 3.40. A single global threshold applied across all three question types will therefore be wrong in both directions at once, which is the failure mode most likely to ship quietly.
The unknowable-task result is the one to remember
Asked questions that could not be answered from the state provided, the TypeSafe AI model was right 44.7% of the time while assigning its answers an average probability of 0.74. That is the precise shape of the problem Ronacher described: the confidence score is handed to you, and it is your job to discover when it is decorative.
Criteria wording carries more weight than anything else
A pre-registered study on 400 items found TypeSafe AI scoring 95.9% zero-shot, against 77.2% for hand-written keyword rules and 66.0% for TF-IDF. The same study found that wrong criteria descriptions are catastrophic: 16.7%, below the 25% random floor. Badly worded instructions do not degrade the model gracefully; they invert it.
| Finding | Measured | What it means for your code |
|---|---|---|
| Expected calibration error | 0.107 vs 0.024 floor | Recalibrate on your own labels |
| Yes/no answers | Underconfident (T = 0.66) | Thresholds sit too high |
| Choice answers | Overconfident (T = 3.29) | Escalate more than the score suggests |
| Score answers | Overconfident (T = 3.40) | Same, per rubric |
| Unanswerable questions | 44.7% right at 0.74 stated | Always offer a “none of these” option |
| Wrong criteria wording | 16.7%, below random | Review question text like production code |
The TypeSafe AI Integrations That Landed Within Three Days
Adoption numbers are easy to dismiss. Four platform teams shipping first-class TypeSafe AI support inside seventy-two hours of a launch is harder to wave away, because that work is planned and reviewed rather than impulsive.
LangChain exposes TypeSafe AI as an ordinary runnable
LangChain shipped TypeSafeClassifier in a langchain_typesafe package, exposing decisions as a standard Runnable so they can be invoked, batched or composed alongside anything else in a chain. It accepts plain strings, structured JSON and LangChain message objects, and records traces and token usage into LangSmith. Its documented AutoMode middleware uses the model to classify risky tool calls before execution.
Cloudflare, Langfuse and the gateways
Cloudflare’s AI documentation now lists TypeSafe AI as typesafe/jev. Langfuse published a guide to using it as an evaluation judge, which is a neat inversion — the cheap model grading the expensive one. Vercel’s AI Gateway carried it from day one. Together those cover most of the places a team building autonomous AI agents would reach for a decision endpoint.
What a listing proves and what it does not
A gateway listing proves that routing exists, not that the routing is good. None of these integrations constitutes an accuracy endorsement, and none of them has been in production long enough to have a failure story yet. Treat them as distribution, which is genuinely what was missing, and not as validation.
| Platform | What shipped | Developer-visible surface |
|---|---|---|
| Vercel | AI Gateway routing | Model available at launch, telemetry published |
| LangChain | TypeSafeClassifier runnable | langchain_typesafe, LangSmith traces, AutoMode |
| Cloudflare | Model listing | typesafe/jev in the AI docs |
| Langfuse | Evaluation guide | Jev as an eval judge over traces |
The Capacity Story Behind TypeSafe AI's Sudden Demand
Enthusiasm has a bill attached. The most telling operational detail of the week is that TypeSafe AI briefly lost the ability to serve requests from its own API because demand exceeded what it could supply.
TypeSafe AI rate limits that adjust dynamically
The published account limits are 250,000 tokens per second and 1,200 requests per minute, with the documentation noting that limits are being adjusted dynamically because of demand, and that higher ceilings are available on custom plans. A documented limit that moves is a limit you cannot design against. Read those two numbers as account quotas, not as a latency or throughput promise for any single request.
What to assume about headroom
Assume less than you are told, and build the fallback first. Any decision path that routes through TypeSafe AI with a two-week-old production footprint needs a defined behaviour for a 429 or a timeout — usually the previous model, occasionally a static rule, never an exception that surfaces to a user. This is basic intelligent automation hygiene, and the capacity incident is the reason to apply it now rather than later.
Version pinning is part of the same discipline
Both jev-latest and jev-preview currently resolve to jev-1.13.0. They will not always. Any evaluation you run against an alias measures a moving target, and any threshold you calibrate against one silently decays when the alias moves. Pin the explicit version in production and re-validate deliberately.
A Shadow Evaluation Plan for TypeSafe AI
The independent testing came with a methodology worth copying wholesale, because it is designed to answer the only question that matters: does this decide your cases correctly, at a cost worth having?
Pick one decision that already has ground truth
Choose a single high-volume decision where a human already records a final disposition — a ticket that got closed a certain way, an email someone actioned, a flag a reviewer upheld. That disposition is your label, and it exists already. Run 1,000 to 2,000 records through in shadow mode, deciding nothing, logging everything.
Decompose before you conclude anything
Run both framings. Ask the composite question once, then break it into three to six atomic signals and combine them in your own code. The phishing benchmark’s 62.6%-to-95.0% swing came entirely from that difference, and a team that only tests the composite version will reject the model on evidence that says nothing about its ceiling.
Always offer an escape option
Add a “none of these” option to every Choice. The unknowable-task result — right 44.7% of the time while reporting 0.74 confidence — is what happens when the model is forced to pick from a set that does not contain the answer. Giving it somewhere honest to go converts a confident error into a routable one.
Fit calibration on half, score on the other half
Do not consume the published probabilities raw. Fit a calibration per question type on half your labelled set and evaluate on the other half, exactly as the benchmark did. The per-type temperatures show why: one global threshold cannot serve answer types that err in opposite directions.
Compare cost per correct decision, not per call
Price per million tokens is the wrong denominator. Divide your total spend by the number of decisions that turned out right, for the incumbent model and the challenger, and include the cost of whatever you do with the low-confidence cases. That single number is the one a finance conversation survives.
| Step | Action | Failure it prevents |
|---|---|---|
| 1 | Pick one decision with human dispositions | Evaluating against synthetic labels |
| 2 | Shadow 1,000–2,000 records | Conclusions from a 50-row sample |
| 3 | Pin the version, never the alias | Silent decay when latest moves |
| 4 | Test composite and decomposed framings | Rejecting on the 62.6% framing |
| 5 | Add “none of these” everywhere | Confident answers to unanswerable questions |
| 6 | Calibrate per question type, split half/half | One threshold wrong in both directions |
| 7 | Score cost per correct decision | Buying a cheaper way to be wrong |
Why a ChatGPT Inventor Went Looking for This
Diogo Almeida helped build ChatGPT at OpenAI and worked on reinforcement learning from human feedback, the technique most responsible for the assistant era. He left two years ago because he thought the field had optimised for the wrong thing.
The complaint that produced TypeSafe AI
“We have lightning in a bottle, and yet it is not useful,” he told TechCrunch. His diagnosis is narrower than it sounds: “The problem is we are optimizing for human language … We have been super good at human language for four years, but it’s not useful for automation because computers speak a different language.” TypeSafe AI is the attempt to serve the second audience.
Synthetic data as the actual bet
Almeida trains Jev exclusively on synthetic data, through a method he calls reinforcement learning from calibrated decisions. He rates that decision above the launch itself and above his own earlier work: making all of their own data has been, in his words, one of the best bets he has ever made — better, he says, than RLHF. He stays deliberately vague about the underlying architecture, which outside observers suspect is built on an open-weight language model.
The name is a wager on volume
The TypeSafe AI model is named for William Stanley Jevons, whose paradox holds that improving the efficiency of a resource tends to increase total consumption of it rather than reduce it. Almeida expects “smart software all over the place in a way that’s emergent and distributed”, closer to the early internet than to a handful of mega-apps. The pricing is designed to cause that, not merely to survive it.
What to Watch Over the Next Two Quarters
Whether the thirteen percent becomes traffic
Day-one breadth is a curiosity; week-eight retention is a business. The figure to look for is what share of those paid teams are still routing production volume after the novelty passes, and Vercel is uniquely positioned to publish it.
Whether an accuracy benchmark with real ground truth appears
Every published comparison so far uses either agreement with other models or synthetic corpora. The missing artefact is a reliability diagram on a public dataset with human labels. Its continued absence, given how cheap the model is to run, would itself be informative.
Whether anyone ships a rival
Ronacher expects TypeSafe AI competitors now that the utility is visible, and the incumbents already have most of the pieces — constrained decoding and typed outputs are existing features on every major API. The defensible part is the calibration training, not the type safety.
Whether TypeSafe AI capacity catches up with the curve
A vendor that ran out of API capacity in week one, publishing rate limits that adjust dynamically, is a vendor whose supply story is unfinished. Watch for stable published limits and a named enterprise customer; either would say more than another benchmark.
References and Further Reading
A new kind of AI model from a ChatGPT inventor is thrilling developers — TechCrunch
Jev is the fastest-adopted model in AI Gateway history — Vercel
Models — TypeSafe AI documentation
Introducing System One Models and Jev — TypeSafe AI
Jev scores 62.6% asked once and 95% split five ways — The Daily Brief
What is Jev? A guide to TypeSafe AI’s System One model — LangChain
TypeSafe integrations — LangChain documentation
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.