Jev's 379-Request Batch: What the One-Cent Estimate Includes
AGENTS & WORKFLOWS
Jev's 379-Request Batch: What the One-Cent Estimate Includes
Correction, October 17, 2026: 379 requests is the exploration-batch count, including repeated comment passes. It excludes separate claim-check and draft-review tasks. The cost is now explicitly a reconstructed estimate, with its recorded and estimated components separated.
A one-cent batch sounds persuasive until its boundaries become unclear. The Jev exploration here includes repeated comment passes, while separate claim checks sit outside the 379 requests. The reconstructed bill is useful only with those recorded and estimated components left visible.
In this article
Jev answers typed questions about text or JSON and returns values that code can use. It is not served through the chat-completions endpoint: that request returns the documented 400 error shown below. The experiments here use its decision interface for classification, ranking and checking.
The useful candidates are repeated questions with a defined answer space. Feed filtering, comment labels, archive ranking and citation triage each give the code a small decision to act on. They also provide different ways to inspect mistakes.
| What I pointed it at | Items | Questions each | Wall clock | Cost |
|---|---|---|---|---|
| A Hacker News front page, gated for relevance | 30 stories | 3 | 11.1 s | $0.00074 |
| The Jev launch thread, read and tallied | 129 comments | 5 | 9.8 s (8 at a time) | $0.00442 |
| My own 68-post archive, searched | 68 titles | 3, in one call | 0.6 s | $0.00028 |
| This post's own claims, checked against sources | 16 citations | 2 | 1.7 s | $0.00064 |
Typed answers and their fields
Three question types carry everything, and they are simpler than the marketing language suggests. Noul is a revived old word for yes/no and comes back as one probability. Choice picks one option out of a list you define (up to 255 of them, which matters later). Score puts the state somewhere along an ordered rubric you write. You can mix all three in one request and they all see the same state.
That confidence number is the most misread part of the API. It is a statistic about the shape of the probability distribution, not a promise the answer is right, and the docs say so plainly: calibration holds across groups of predictions, never per-answer. The pattern the docs recommend, and the one I ended up using, is to keep the thresholds in my own code — high confidence acts, middling confidence gets flagged, low confidence goes to me.
answer.confidence on every answer will crash on the yes/no questions. (screenshot: docs.typesafe.ai)Filtering a front page
I keep a long list of feeds, and the expensive part is not reading them — it is deciding what deserves reading. So I pulled the 30 current front-page stories and asked three questions per story: which topic is this, would a reader of this blog want it, and how much substance is behind the headline.
Thirty requests, 11 seconds, $0.00074. Four stories cleared my 0.7 relevance line: self-hosted inference orchestrators compared, a Jev build running offline on Apple silicon at 45 decisions per second, Qwen-Image-2.1, and the unfortunately named "I turned Jev into a (lousy) chatbot". One item scored 0.01 — a GameCube decompilation — which is the kind of thing I would normally have opened out of curiosity. Nothing about that run needed a language model's prose; it needed a sorting decision, and the answer cost less than the electricity of my monitor being on.
A batch of 129 comments
The launch thread on Hacker News is now 506 comments deep, and I wanted the shape of it rather than the vibe of it. I grabbed every top-level comment and its first replies with some actual text in it — 129 of them — and asked five questions per comment: is the author speaking from their own use, what is their stance, what is the single main topic, how promotional is the writing, and is there anything here worth quoting.
python jev.py --batch comments.jsonl --questions-file q.json --out answers.jsonl
# 129 rows, 8 workers, 9.8s wall, 105,110 input tokens, $0.00442, zero errorsWhat came back is the chart below: the biggest single topic is what people could use it for (39), closely followed by what it cannot do (29), with benchmark skepticism third (12). Stance skews to "sees real value, names real problems" (49) over straight positive (38), and the promotional-writing score sat at a median 0.69 out of 2, which is roughly "opinion with reasoning".
Two of those comments are interesting enough to quote, and they point the same direction. A builder: "We've already started using it for some pretty powerful decision tree stuff... The one downside is that the context window is very small (32k.)" And a team that got access early: "Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models... works incredibly well in concert with LLMs, not as a replacement." (Both from the Hacker News thread.)
The number I did not expect: 2 of 129 substantive comments described first-hand use. That is different from the launch-week tweet census I wrote about before, where 23% of relevant tweets came from people who had run it themselves — and the difference is probably the room, not the product. X filled up with builders showing screenshots; a Hacker News launch thread fills up with people arguing about the claims. It is a good reminder that "what is the community saying" has a different answer on every platform, and that reading a thread is not the same as measuring one.
The manual label check
This is the part where a triage tool usually stops being useful, so I checked it by hand. I took ten comments from the batch — every thirteenth — read each one myself, wrote my own stance and topic for it, and compared without letting Jev see my answers.
Stance agreed on 9 of 10. Main topic agreed on 7 of 10, and all three misses were the same boundary: comments attacking the launch's speed and cost comparisons, which I filed under "doubts the benchmarks" and Jev filed under "what it can't do". That is a category design problem on my side, and the docs' Choice page already has the fix — when two options keep getting confused, stop describing them as one-line strings and give each one fields for what it covers and not_for, plus example inputs. I wrote the option list in five minutes over coffee. It shows.
The confidence number did real work here. Three of my four disagreements came back with a confidence under 0.7 on that question, so a threshold of "auto-accept above 0.7, send the rest to a human" would have flagged them for me. One did not: a commenter's topic came back wrong at confidence 0.97. Confident and wrong is the failure mode to plan for, which is why the last step of this pipeline is still me reading a sample.
Ranking the 68-post archive
Every time I write a post I want to link something older, and I usually end up scrolling my own archive. So I numbered the 68 published post titles, sent them as one state, and asked two separate Choice questions — which post to link from a hands-on piece about cheap decision models, and which post is about running a model on local hardware — plus a Noul question asking whether a local-model post exists at all.
One request, 0.6 seconds, 6,750 tokens, $0.00028. The first question came back at confidence 0.99 with my own Jev launch post at probability 1.000 — which is exactly the link this post needed, and it is the link I used. The second was a different story: the top candidate (a Mac mini RAM post) only reached 0.37, second place 0.22, and confidence dropped to 0.35 — while the Noul check said 0.99 that such a post exists. That pairing is the whole trick, and it comes straight out of the docs' line-by-line search recipe: ranking always returns a best candidate even when nothing fits, so you ask a second question about whether an answer exists at all. Rank where to look, check whether there is anything to find.
The docs push this pattern to 218 lines of a terms-of-service document in one request, which is a practical way to get semantic search without a vector store, a chunking strategy or an index to keep warm. Past 255 candidates you split it into two passes — one Choice picking a window, a second ranking inside it.
Checking claims against supplied sources
This is the use I have been waiting for someone to build for years: an automatic check that every claim in a draft is actually supported by the thing it cites. So I ran it on this post. I built 16 citations — 13 real claims from the sections above, each with the quote and the source I was leaning on, plus three deliberate failures: one where the quote is not in the source at all, one where the source says the opposite, and one where the source never addresses the claim.
My first attempt asked Jev to do everything: does the source contain the quote word for word, and how does it relate to the claim. It caught all three planted failures — and then it flagged four of my thirteen good citations as fabricated. The quotes were in the sources. I went back to the docs and found the answer in the cookbook this recipe comes from: the fabricated-quote check there is a string match in code, not a model question, precisely because it is the one part of the job with an exact answer that code cannot get wrong. So I rebuilt it that way — normalize whitespace, look for the quote, and only hand the quote's context to Jev to judge whether it supports the claim.
The second run verified all 13 real citations, caught all three failures (contradiction at confidence 0.99, unsupported at 1.00, and the missing quote stopped at the string match before any Jev call was spent), and flagged one claim at confidence 0.66 — the one about TypeSafe's own Hermes skill cookbook. I re-read the source line myself: it is supported, and it still went into the human pile, which is what the threshold is for.
One question versus six
Everyone repeats the same advice about this API — ask many narrow questions per call, because they run in parallel — so I measured it. Same state, same data, once with one question and once with six, five repeats each. The one-question calls had a median of 0.359 s and 553 billed tokens. The six-question calls: 0.375 s and 808 tokens. Six times the answers for 4% more wall clock.
The cost model follows from that: you pay for the state once per request, no matter how many questions you attach. Which means the expensive mistake is not asking too many questions — it is calling Jev once per question, or sending it a state it does not need. My comment run averaged 815 input tokens per comment for five answers; the same five answers from a chat model would have meant five prompts with the whole comment pasted into each.
Four rules carried over from a day of this, and they are all in the docs rather than invented here. Keep one judgment per question and combine the answers in code, because a broad question hides several decisions behind one number. Split anything you would describe with "and". Keep the policy in your code — thresholds, escalation, retries — and let the model only produce the reading. And use it for selection rather than generation: find the candidate values in code, let Jev pick one, then copy the chosen value verbatim, which is how you get extraction that cannot invent a value that was never there.
Failures and prompt sensitivity
The vendor's own jaggedness page is refreshingly blunt, and my day matched it. It reads instructions literally and answers the question you wrote rather than the one you meant. It cannot count or do arithmetic — keep the math in your code. It treats dates as text. Its accuracy drops as you add irrelevant detail to the state, which is a real cost when your instinct is to paste in everything. And my personal favourite line in the whole documentation set: "State is data, and jev-1.13 does not treat it as hostile by default." Prompt injection inside the state can move an answer. If a decision gates something expensive, that is your problem to solve in code.
It also cannot explain itself. You get a number, never a reason, so when a label is wrong you debug your question, not its reasoning — which is a different skill from prompt engineering and takes a few rounds to get used to. Non-English input is handled but, in the docs' own words, not equally well; test before you rely on it.
And then there is the part I got wrong, which is worth more than the part I got right. My first version of the "first-hand user" question produced 1 hit in 129 comments. I rewrote it to be explicit about what counts — ran it, built with it, got access — and got 2. Same data, same model, one question rewritten, and the number moved by a factor of two. Neither of those numbers is "the truth"; both are my question being read literally against a sample I then had to check by hand. Thresholds and labels are triage, not verdicts, and the only reason I can state 2 of 129 with a straight face is that I read ten of them myself.
What the request total includes
The saved batch contains 30 front-page calls, one archive-ranking call, an initial 80-comment pass, two 129-comment passes with different questions, and ten fan-out repetitions. That is 379 requests with repeated inputs. The per-call token subtotal is reproducible from the saved usage fields; the ten fan-out token counts are an estimate from two representative records. The ten-comment manual check uses a single reader’s interpretation, including a borderline stance label, so it is a diagnostic sample rather than independent accuracy ground truth.
I set myself a rule while running these jobs, and I have kept to it: if the same judgment repeats over eight or so items — classify, route, rank, score, filter, verify — it goes to Jev instead of my main model, and if it is one question about one thing, it does not. That rule has already earned its place in three spots: gating feeds before they reach my context, reading threads I would otherwise skim, and the draft check below.
The draft check is the one that surprised me. Before publishing this, I ran all 32 paragraphs of this post through the same sort of request — four questions per paragraph, looking for unsourced numbers, hype, jargon and anything that needs verifying. It came to $0.00074, and it flagged 17 of the 32 paragraphs for something, which is exactly the failure mode I described further up: the criterion asks whether the source is named in that same paragraph, and half of this post cites everything in the footer instead. I read the list, acted on the two flags that were right (one paragraph named numbers without its source nearby, one was boosterish about the docs), and ignored the rest. A triage list is for deciding what to look at, not for deciding what is true.
The 379-request figure covers the exploration batch, including repeat passes over overlapping comments. It is not 379 distinct examples and is not the total for every later draft-review task. The archived per-call records cover 369 requests and 240,254 input tokens. Ten fan-out timings are also saved; applying each configuration’s saved representative token count to its five repeats adds an estimated 6,805 tokens. At the historical $0.042 per million input tokens, the combined estimate is $0.010376478, rounded to $0.0104. Separate synthetic citation-check runs and later paragraph reviews are outside that total.
These runs support using a decision model for narrow, repeated judgments. They do not establish reliable unsupervised operation of a scheduled feed or comment service. That would need a separate log of inputs, thresholds, interventions and failures over time.
Sources and method: all measurements are my own runs on 2026-09-21 through OpenRouter (typesafe/jev-1.13-20260917) at the published rate of $0.042 per million input tokens with output tokens free — the API's per-call usage fields match that arithmetic. Response times are wall clock from this machine and include network. Comment data is Hacker News thread 49717558, collected the same day; quotes are attributed by platform, kept verbatim, and no individual is named. The front-page batch used the Hacker News public API at the same timestamp. The three planted citation failures are mine, not the vendor's. TypeSafe documentation referenced: primitives · Choice · Noul · state · models and pricing · double-checking citations · line-by-line search · jaggedness. Earlier post on the launch numbers and the field data: Jev: 444x Cheaper on the Homepage, 32x in Real Runs.
Related on this blog: Cloudflare Clef vs Jev: What the 13.5x Speed Claim Means — Cloudflare’s new decision models, with a closer look at the vendor’s speed, accuracy and input pricing figures.