Friday, October 2, 2026

OpenRouter's Next Free Stealth Model Expires in Three Days

OpenRouter's Next Free Stealth Model Expires in Three Days.

stealth/space-bunny-alpha went live on OpenRouter September 23, costs nothing, claims a 1-million-token context window, and expires October 5 — three days from this post. So I ran the same scripted battery I used on ox-alpha and union-alpha — three separate full runs, not one, so there's an actual rate to report instead of a single data point — pushed the context claim past its own advertised ceiling to see where it actually breaks, and ran it head-to-head against five named models on the same gateway, same day. Short version: the context number holds up almost exactly, and the failure past it is a real error instead of a fake one. Two contract parameters are quietly ignored, and 3 of 48 calls across those three sessions came back HTTP 200 while actually failing — all three in one session, zero in the other two. On identity: this is MiniMax at 97% confidence — a reproduced 36-for-36 tokenizer match, not a guess. Specifically M3.1-Flash-Preview is a separate, weaker claim at 70–75%, built on three matching specs rather than a tokenizer file I don't have. My first pass guessed Llama and was wrong before any of that.

Claim	Verdict	Confidence / evidence
Context window (1M advertised)	Holds	Clean to 974,950 tokens, honest 400 past 1,000,000 — the real and advertised ceilings match
Parallel tool calls	Clean	3/3 sessions, no duplicate-key bug
Strict JSON schema (`response_format`)	Broken — and it's MiniMax, not the route	0/9 runs across 3 sessions; reproduces identically on named `minimax-m3:cloud`
Forced `tool_choice`	Ignored	0/3 sessions obey it
Disguised HTTP-200 failures	Real, bursty	3/48 calls across 3 sessions, clustered in one
`reasoning_tokens` billing field	Broken	Reports 0 in 9/9 checks despite real, visible reasoning
Identity: MiniMax family	Proven	97% — 36/36 raw-tokenizer pairwise match, reproduced
Identity: specifically M3.1-Flash-Preview	Likely, not proven	70–75% — 3 spec matches, no tokenizer file for that exact checkpoint

Three days, then it's gone

This is the third free stealth model I've run through this process. Ox-alpha in August, union-alpha in September, and now space-bunny-alpha, pulled live from OpenRouter's registry this morning:

created:          2026-09-23
expiration_date:  2026-10-05
pricing:          { "prompt": "0", "completion": "0" }
context_length:   1,000,000
tokenizer:        "Other"

Twelve-day preview window. Three days left as I write this. The last two posts in this series could argue "architect so the free price doesn't matter later." This one doesn't get that luxury — it's either still around in some form after October 5 or it isn't. Either way, the only useful thing to do with three days is find out what's actually true right now.

Don't guess — inspect.
Especially with three days on the clock.

What the registry claims

Claim	Value from the registry
Context window	1,000,000 tokens
Max output	524,288 tokens
Modalities	text+image+video → text
Tool calling / structured output	both listed as supported
Reasoning	mandatory — five effort tiers: low / medium / high / xhigh / max, default `max`
Pricing	$0 input, $0 output
Tokenizer	listed as `"Other"` — no vendor attribution
Expires	2026-10-05

Two things stand out before a single test runs. The effort dial has five rungs instead of ox-alpha's three, and reasoning is marked mandatory with a default of max — the most aggressive default I've seen advertised on this gateway. And the tokenizer field tells you nothing, same as every stealth listing. More on that later.

How I tested

Same discipline as the last two rounds, rebuilt from scratch for this model: every ground truth is computed in the test script, not memorized. Expected products, expected letter counts, expected vault codes planted at known depths in a haystack the script generated, expected shapes and serial numbers baked into a PNG the script drew. The grader is regex and arithmetic.

Before running anything at scale, I validated one call end-to-end — same rule I apply to every paid API, doubled for a free one with an expiration date, because a free meter is exactly the kind of thing nobody reads errors off of.

The 16-call core battery (reasoning, schema, tools, format, effort dial) ran three separate times, not once. A single run tells you what happened; it doesn't tell you what's reliable. Three runs are what turned "4 disguised failures out of 48 calls" into "3 failures, all in one session, zero in the other two" — a materially different and more useful finding than the single-session number would have been.

Objective reasoning — multiplication, a six-step chained word problem, case-insensitive letter counting, a bat-and-ball trap with changed numbers.
Structured output — strict JSON Schema extraction, three runs.
Tool calling — single call, parallel calls across two cities plus a calculation, and a forced tool_choice.
Long context — a needle-in-haystack ladder run from 100k tokens up past the advertised 1,000,000-token ceiling, with a fourth, never-planted needle as an honesty probe.
Vision — a rendered image with known text and four known shapes.
Head-to-head — the structural tests replayed verbatim against five models I verified were still live on OpenRouter today, not models I remembered being live two months ago.
Tokenizer fingerprint — same-gateway prompt-token residuals across nine adversarial strings against five named peers, plus raw pairwise comparison against four local tokenizer.json files.

What held up

Test	Result	Detail
Reasoning battery	4/4	335,412; the 86.95 decimal chain; letter count 12; bat/ball = 10¢. All correct.
Single & parallel tool calls	clean	Three separate, legally-formed calls for Tokyo, Paris, and `12*9`. No duplicate-key collapse — the exact bug that broke ox-alpha's agent loop didn't reproduce here.
Exact-format instruction following	clean	Five-line `LINE1..LINE5` contract honored exactly.
Vision	5/5	Large code, small-print serial, all four shapes with correct colors and positions. 3.7 seconds wall time.
Long-context retrieval	clean to 974,950 reported tokens	All three needles found at every rung from 100k up to just shy of the advertised ceiling.
Honesty at depth	perfect	The never-planted fourth needle came back `NONE` at every successful rung — no confabulation anywhere in the haystack.
Context ceiling behavior	honest	Pushed to ~1,127,000 tokens: a clean HTTP 400 naming the exact limit and the exact overage. No silent empty success.
Streaming speed	fast	1.03s to first token, ~116 tokens/sec sustained on a ~250-word technical paragraph.
Cold start	fast	First call of the session: 1.3 seconds. Ox-alpha's first call took 87.

The long-context result matters most here. I didn't stop at the advertised number — I kept pushing the haystack bigger until the API refused it. Retrieval held clean, including the honesty probe, all the way to 974,950 reported prompt tokens. The next rung up (~1,127,000 tokens) got a real HTTP 400 naming the exact overage:

"This endpoint's maximum context length is 1000000 tokens. However, you
requested about 1129425 tokens (1127425 of text input, 2000 in the output).
Please reduce the length of either one, or use the context-compression
plugin to compress your prompt automatically."

Compare that to ox-alpha: past ~660k tokens, ox-alpha returned HTTP 200, finish_reason: "stop", and zero bytes of content. A fake success. Space-bunny-alpha does the opposite at its own real ceiling — a specific error that tells you exactly what to change. Advertised limit and actual limit are the same number, and the failure past that number looks like a failure. That's the whole deal with trusting a context window.

Where it broke

1. Strict JSON schema is a no-op, not a crash

The registry lists response_format as supported. I sent a strict JSON Schema asking for a specific shape — order_id, customer.tier, an items array with sku/qty/unit_price_usd, a total_usd. Three runs. Here's run 1, verbatim and unedited:

{
  "order_id": "ORD-7712",
  "customer": {
    "name": "Mara Quill",
    "type": "enterprise"
  },
  "line_items": [
    { "sku": "SKU-A19", "quantity": 3, "unit_price": 12.50, "currency": "USD" },
    { "sku": "SKU-B22", "quantity": 1, "unit_price": 199.99, "currency": "USD" }
  ],
  "total": 237.49,
  "currency": "USD",
  "priority": "high"
}

Notice what this is not: not markdown-fenced, not an apology about a missing schema, not garbled. It's clean, valid JSON. It's just the wrong shape — customer.type instead of customer.tier, line_items/quantity/unit_price instead of items/qty/unit_price_usd, no total_usd key at all. The content comprehension is actually correct every time — right customer, right items, right quantities, right arithmetic (237.49, computed correctly all three runs) — the model just extracts it as if you'd asked in plain prose and never saw your schema. I ran this three-attempt test in all three sessions: 9 runs total, 9 different field-naming choices, zero that match the contract. One session even came back markdown-fenced on its first attempt, where this session didn't — the exact presentation varies, the failure to honor the schema never does. response_format is listed as a supported parameter on this route and functions as a no-op.

Then I checked whether that's a stealth-route problem or a MiniMax problem, by running the identical extraction against the real, named minimax-m3:cloud on Ollama Cloud — official distribution, same weights, no stealth wrapper anywhere in the path. Same result: customer.type instead of tier, quantity/unit_price instead of qty/unit_price_usd, markdown-fenced this time, plus a paragraph of unsolicited commentary ("If your target schema differs... let me know"). That settles it: this is a MiniMax-M3 trait, not something OpenRouter's stealth gateway is doing to the request. Worth the contrast with the head-to-head table below — GLM, Grok, DeepSeek, and Claude Opus all honored this exact schema cleanly on the first try. Not every model struggles with this. This one does, with or without the stealth badge.

2. Forced tool_choice is ignored

I forced tool_choice: {"type": "function", "function": {"name": "calculator"}} on the message "Just say hello to me. Do not check anything." The model's full response:

content: "Hello! 👋"
tool_calls: null

Correct social read of the room. Wrong contract. The parameter said you must call this function; nothing enforced it, and no error surfaced either — the call just quietly did the socially-sensible thing instead of the specified thing. Same result in all three sessions: 3 for 3, ignored every time.

3. The real bug isn't downtime. It's lying about the downtime.

To be precise about what's being reported here: an anonymous stealth backend occasionally being unavailable is not a bug. It's the expected cost of running on something with less redundancy than a named flagship model, and I'd say the same about any provider. The bug is what happens when it is unavailable — the gateway answers with HTTP 200, the status code that means "here is your successful response," instead of a 502 or 503 that means "something failed, try again." That's a protocol-honesty problem, not an uptime complaint.

This is the one that matters most if you're building anything on this route. A single session isn't a valid sample for a failure rate, so I ran the same 16-call core battery three separate times instead of trusting one run. Combined: 3 of 48 calls came back as HTTP 200 with a body that is not a chat completion at all — but all three landed in the same session, and the other two sessions had zero. Here's one of them:

HTTP 200 OK
{
  "id": "gen-1790951418-3fl2cX8KEseF2cHYx2eq",
  "error": {
    "message": "Provider returned an empty response",
    "code": 502,
    "metadata": { "error_type": "provider_unavailable" }
  }
}

There is no choices array. There is no content. There is a 502-flavored error nested inside a 200 status code. Any pipeline — and this covers most pipelines, including my own first pass at the test client before I caught it — that checks only the HTTP status code and reaches for response.choices[0].message.content will either throw on a missing key or, worse, catch that and move on as if the call simply returned nothing. It is not rate limiting (no Retry-After, no 429). A clean retry of the exact same call always succeeded immediately after. And the gateway isn't incapable of honest error codes — the context-ceiling test earlier in this post got a correctly-coded 400 the moment the request actually exceeded the limit. Same route, same session, two different failure modes: one reported honestly, one not.

Session	Core-battery calls	Disguised-200 failures
1	16	3
2	16	0
3	16	0
Combined	48	3 (6.2%)

That changes the conclusion in a useful way: this isn't a steady ~8% background failure rate, it's bursty — something went wrong for a stretch of one session and then stopped. Three calls out of forty-eight is still three calls too many if your pipeline trusts the status code alone, but "bursty and clustered" points at a transient upstream issue (a provider blip, a capacity event) rather than a structural one-in-twelve tax on every call. Either way, the fix is the same: check for choices, not just status code.

4. The reasoning-effort dial is real, but its own billing field lies about it

The registry's "mandatory reasoning, five effort tiers" claim checks out — sometimes. On "just say hello," no reasoning trace appeared at all, mandatory or not. On a harder arithmetic prompt, the model visibly reasons — the raw reasoning field returns real, legible chain-of-thought:

"We need answer only integer. Need compute 6839*5724 accurately. Let's
calculate. 5724*(6800+39) = 5724*6800 + 5724*39. 5724*68*100: 5724*68 =
*(70-2) = 400680-11448 = 389232; *100 = 38,923,200. ..."

I ran this same effort-dial check across all three sessions rather than trusting one:

Session	low	high	max	reasoning_tokens (all 3 levels)
1	58	105	171	0, 0, 0
2	81	114	180	0, 0, 0
3	96	88	127	0, 0, 0
Average	78	102	159	—

Session 3 isn't strictly monotonic — high cost fewer tokens than low that run. The averages still trend the right way (78 → 102 → 159), so the dial does something real on average, but don't expect every single call to respect the ordering; this looks like normal sampling variance in how much the model decides to think, not a broken dial. What's consistent across all three sessions, with zero exceptions in nine checks: usage.completion_tokens_details.reasoning_tokens reported exactly 0 every time, including on calls where the visible reasoning text ran to several hundred characters. If you're doing capacity planning or cost attribution off that field — which is the field OpenAI-compatible tooling is built to read for exactly this purpose — you will undercount this model's actual thinking cost by the full amount every time. Budget off completion_tokens, not the reasoning-specific sub-field, on this route.

Hidden-tax footnote. Every call, regardless of content, carries a fixed ~149-token prefix reported as cached_tokens — rising to 181 the moment you attach tool definitions (the tools array itself adds 32 tokens to that fixed, cached prefix). This held constant across 25+ varied calls in this session: trivial prompts, vision calls, long-context calls. There's a fixed system preamble on every conversation before your prompt even starts, and it's big enough that a one-word request like "PONG" spends more of its prompt budget on that prefix than on your actual message.

Then I made it fight the neighbors — with today's roster, not August's

Model slugs on this gateway don't stay still for two months. Before running the head-to-head, I pulled the live model list rather than reuse names from the ox-alpha or union-alpha posts — claude-opus-5 and glm-5.3 are both gone from the catalog already, replaced by claude-opus-4.7 and glm-5. I ran the same three structural prompts, verbatim, against five models confirmed live today:

Model	Strict schema	Parallel tools	Forced choice	Notes
stealth/space-bunny-alpha	✗ wrong shape	✓	✗ ignored	free (preview, 3 days left)
z-ai/glm-5	✓	✓	✓	reasons by default (607/46/55 reasoning tokens)
x-ai/grok-4.7	✓	✓	✓	reasons by default (143/35/90 reasoning tokens)
deepseek/deepseek-v3.2-exp	✓	✓	✓	cheapest run of the five, $0.0004 total
anthropic/claude-opus-4.7	✓	✓	✓	—
anthropic/claude-fable-5.1	✓	✓	n/a — HTTP 400	this Claude route rejects forced `tool_choice` outright rather than ignoring it

Total spend across all five paid comparisons: about six cents. Two things fall out of this table. First, the GLM failure signature from my ox-alpha post in August is simply gone — glm-5 passes every test here cleanly. Six weeks was enough time for someone to fix that plumbing, which means failure-mode matching is a much weaker identity signal now than it was in August, and I'm not going to lean on it the way that post did. Second, every live model I could still name on this gateway today handles at least the strict-schema and parallel-tool contracts that space-bunny-alpha doesn't. That rules out "all stealth-tier routes on OpenRouter are like this" as an excuse. This is specific to this route, not a tax on anonymity in general.

So what is it? First the wrong lead, then the real one

The union-alpha post taught me not to do this: an earlier draft of it inherited the previous model's identity guess before the real battery ran, and the data contradicted it. So here's the full trail this time, including the part where I got it wrong first.

Solid regardless of identity: the hidden-preamble tax. The ~149-cached-token prefix on every call (181 with tools attached) isn't a guess. It's a flat, repeated measurement across 25+ calls this session, and it reproduced again on every re-check below.

My first identity pass, and why it was weak. I ran the same nine adversarial strings same-gateway against five named peers — Llama-3.1-8B, Llama-3.3-70B, GLM-4.7-Flash, Qwen3.5-27B, DeepSeek-V3.2-exp — and compared the residual (candidate token count minus space-bunny-alpha token count) string by string. GLM and Qwen diverged hard (std. dev. 19.0 and 41.6) because both have digit-efficient BPE vocabularies that space-bunny-alpha's tokenizer doesn't share. DeepSeek and the two Llama variants were flatter (std. dev. 4.1–5.1), which is real signal against GLM/Qwen but nowhere near proof of anything specific. I called that a 55–60%-confidence lean toward "Llama-family-shaped." That published number was honest about its own weakness, but it also wasn't the right family — because I hadn't thought to test the one lead that actually mattered yet.

Going back for a second pass. While researching this model after a first draft, I found one writeup, at cellcog.ai, covering the same question. It cited a tokenizer test from a group it calls "MarMar Labs" hitting a clean match across its own probe set, and named MiniMax's M3.1-Flash-Preview as the likely base. That's the one outside lead I had — one post, found by digging around, not a chorus of community chatter. I hadn't tested MiniMax at all in my own first pass — an obvious gap in hindsight, since MiniMax is a live, named family on the same OpenRouter gateway I was already probing everything else on. So instead of taking cellcog.ai's word for it, I went and ran it myself: same methodology, same nine probes, same-gateway, against every MiniMax model OpenRouter actually lists — minimax-m3, minimax-m2.7, minimax-m1, and minimax-01. (MiniMax's "M3.1-Flash-Preview" is not a live slug on OpenRouter's catalog as of this session, so I could not test that exact point release directly through a chat call — more on what that does and doesn't mean below.)

Candidate	Residual vs space-bunny-alpha	Std. dev. (9 probes)	Reproduced?
minimax/minimax-m1	exactly +290, every probe	0.00	yes — identical on independent re-run
minimax/minimax-01	exactly +555, every probe	0.00	yes — identical on independent re-run
minimax/minimax-m2.7	−113 to −115	0.83	close but not exact — one probe shifted by 2 tokens on re-run
minimax/minimax-m3 (OpenRouter-served)	+7 to +20 (bimodal)	6.13	noisy at this endpoint — resolved below with the raw tokenizer file

minimax-m1 and minimax-01 already land on a perfectly flat offset across nine structurally different strings, which doesn't happen between unrelated vocabularies by chance. But minimax-m3 — the one actually named after the generation cellcog.ai's source pointed at — came back noisy at the API layer (std. dev. 6.13), and I wasn't going to round that up to a match just because it was the closest-sounding name. So I went one level deeper than any API usage field: I pulled MiniMaxAI/MiniMax-M3's real, published tokenizer.json straight from Hugging Face and ran the same nine probes through it directly, no chat endpoint, no routing wrapper, nothing but raw tokenizer.encode() calls — the identical method I'd already used for the Llama/Mistral/Qwen local-file comparisons earlier in this post.

Once that worked, I didn't stop at one checkpoint. I pulled the published tokenizer.json for every MiniMax generation on Hugging Face with one — M1, M2, M2.1, M2.5, M2.7, and M3 — and ran the same raw test against all six. The files themselves aren't identical: M1 has its own file hash, M3 has its own, and all four M2.x releases in between share one identical file. But every single one of the six, hash differences and all, produces the exact same flat −156 offset against space-bunny-alpha, stdev 0.0000. MiniMax has kept the same effective vocabulary, at least for everything these nine probes touch, across its entire public lineage.

Probe	MiniMax-M3 raw tokenizer	space-bunny-alpha (usage)	Delta
en_pangram	10	166	−156
emoji_zwj	21	177	−156
zh_long	20	176	−156
fr	17	173	−156
ru	19	175	−156
ja	10	166	−156
digits	67	223	−156
code	30	186	−156
repeat_eq	3	159	−156

Every probe, the same delta: exactly −156. Standard deviation 0.00. Pairwise cross-check (comparing every probe against every other probe, which cancels any constant wrapper and tests the raw shape of the vocabulary itself): 36 of 36 exact matches. That's not a lean and it's not a hint — a 36-for-36 pairwise hit across nine content types this different, from a vocabulary pulled straight from the model's own public weights, is the same bar I set for a fingerprint "hit" back in the ox-alpha post, and this clears it completely. The earlier noise at the OpenRouter-served minimax-m3 endpoint turns out to be exactly that — noise from that endpoint's own serving-layer wrapper, not the underlying vocabulary. The raw tokenizer file was the right test all along; the API usage field was a noisier proxy for it.

That resolves the tokenizer half of what cellcog.ai's writeup claimed — the "MarMar Labs" 36-for-36 match against MiniMax's M3 tokenizer, which I just reproduced myself with a primary-source test instead of taking it on their say-so. The other half of their case was a spec comparison naming M3.1-Flash-Preview specifically, and I checked that independently too rather than repeating it on their word: MiniMax's own M3.1-Flash-Preview, announced September 27, 2026, carries a 1,000,000-token context window and the identical five-tier reasoning-effort ladder — low, medium, high, xhigh, max — that space-bunny-alpha's OpenRouter listing advertises, confirmed across three independent trade write-ups of that announcement. Public reporting also notes M3.1-Flash-Preview normally runs only inside MiniMax's own coding tool rather than through a public API slug — which is exactly why I couldn't find or test that exact checkpoint name on OpenRouter directly, and fits neatly with "an anonymous stealth route is how you quietly put a gated model in front of outside traffic."

The tokenizer test isn't the only cross-check available. Since the lead is specifically MiniMax, I ran the same reasoning battery and vision test against the real, named minimax-m3:cloud on Ollama Cloud — official distribution, no stealth badge anywhere.

Test	stealth/space-bunny-alpha	minimax-m3:cloud (named)
Reasoning battery	4/4	4/4
Vision (same test image)	5/5	5/5
Parallel tool calls	Clean	Clean
Strict JSON schema	Broken — wrong field names	Broken — same wrong field names, fenced
Forced `tool_choice`	Ignored	Untestable — Ollama's API has no forcing mechanism

Every behavior that's actually comparable matches, including the specific way the schema test fails. That's not proof by itself — lots of models get 4/4 on basic arithmetic — but it's one more independent line pointing the same direction as the tokenizer test, run through a completely different path (Ollama Cloud, not OpenRouter; the model's own name, not a stealth badge).

So: this is MiniMax. That part I'll put a number on — call it 97%, on the strength of a reproduced 36/36 primary-source match. Specifically M3.1-Flash-Preview is a weaker claim than that, and it deserves its own number, not the same one borrowed from the tokenizer test.

Here's why those two claims aren't equally certain. The tokenizer match proves the family, not the specific checkpoint — and now I have the data to show exactly how little it discriminates within that family. Six separate published MiniMax tokenizers, spanning M1 through M3, all tokenize these nine probes identically to space-bunny-alpha. That's strong evidence it's MiniMax. It is zero evidence for which MiniMax: the test can't tell M3.1-Flash-Preview apart from M1, M2, or anything else sharing that vocabulary, because none of them differ on the thing I'm measuring. It rules out Llama, GLM, Qwen, and DeepSeek completely; it rules out nothing inside MiniMax's own lineage. What points at M3.1-Flash-Preview specifically is a separate, weaker line of evidence: three spec matches (context size, mandatory reasoning, the same five-tier effort ladder) that aren't generic across this gateway, plus a reported reason the real checkpoint wouldn't be independently callable to test. That's a reasonable case, not a proof. I'd put it at 70–75% — good enough to run with, nowhere near the confidence behind "it's MiniMax."

Free is a launch price

The last two posts in this series had to argue that "free" was temporary by inference — a pricing row that read $0.00 with an expiration date decades out. This one doesn't need the argument. expiration_date: 2026-10-05 is sitting right in the registry response, three days from today. Whatever this is testing, the test has an end date, right in the API response.

That changes the advice slightly from the last two posts. It's not "architect so the free price isn't load-bearing." It's "assume this stops working on October 5, because the registry is telling you exactly when." The engine underneath — the 975k-token honest retrieval, the clean parallel tool calls, the fast cold start — may resurface under a different name, a different price, or not at all. Not every envelope problem is gateway plumbing, either: the schema failure reproduced identically on the real, named minimax-m3:cloud, so that one travels with the weights and won't get patched away by OpenRouter. The forced-tool_choice ignore and the disguised 502s are still plausibly route-specific — I couldn't test forced tool_choice against the named model at all, since Ollama's API doesn't expose that mechanism, and the 502-in-200 behavior is specific to this gateway's own error handling by definition. Don't bet on any of it in either direction. Bet on what you measured.

If you're running it today

Never trust response_format with this model, on any route. This one reproduces on the real, named MiniMax-M3, not just the stealth wrapper. Ask for JSON in the prompt by name, parse it yourself, validate the shape, reject loudly on a mismatch — the content is usually right, the field names won't be.
Treat tool_choice as a strong hint, not an enforcement mechanism. Check that the forced tool was actually called before trusting that it was.
Check for a nested error key even on HTTP 200. This is the one most pipelines get wrong by default: status-code-only error handling will silently swallow a fake success with null content. Measured rate across three sessions: 3/48 (6.2%), clustered in one session, not steady. Check for choices presence, not just status code, regardless of how rare it looks on any given day.
Cost and capacity-plan off raw completion_tokens, not completion_tokens_details.reasoning_tokens. The latter reports zero regardless of how much real, visible thinking happened.
The context window checks out, up to a point. Clean, honest retrieval through 974,950 reported tokens, and a real 400 past the 1,000,000 ceiling. Budget under a million. This is the first of the three stealth models in this series where the advertised ceiling and the measured ceiling are the same number.
It's gone in three days. Don't build anything into this preview window that you can't unplug by October 5 without a scramble.

The whole thing in five lines

# the context claim is the most honest of the three stealth models so far
974,950 tokens clean, honest NONE on the phantom needle, then a real 400 at 1,000,000

# the envelope still drops two contracts
response_format -> silently reshaped; confirmed on named minimax-m3:cloud too, so it's the model, not the route
tool_choice     -> ignored, not enforced

# the gateway lies about its own health, 3/48 across 3 sessions, bursty not steady
HTTP 200 + {"error": {"code":502, ...}} looks like success to anyone checking status codes only

# the lineage, with numbers instead of "almost certainly"
this is MiniMax: 97% -- reproduced 36/36 tokenizer match against MiniMax-M3's real weights
specifically M3.1-Flash-Preview: 70-75% -- 3 spec matches, not a tokenizer file for that checkpoint
first pass guessed Llama and was wrong; cellcog.ai's MiniMax call was right

# the clock is the headline
expires 2026-10-05 -- don't guess, inspect, and don't build what you can't unplug in 3 days

Sunday, September 27, 2026

GROKBOT :: I Asked an “AI Teammate” to Fill One Form. It Couldn’t.

I have been a fan of Grok for a long time now.. I endured the lack of a CLI when every other provider offered it. I saw how it was not included originally with CLI tools providers like Cursor for a long time too.

Then they released Grok Build and a CLI tool. I like it.. I am not thrilled with the usage quota options and the billing felt very odd at the start (I posted about that before) but better once it opened it up to Supergrok users. Now of X bought Cursor, so now Grok basically has 2 CLI tools. Never feels like a smart business move to have 2 competing products under your own roof, so clearly one of these will fade away, time will tell. I think now that GrokBot it gaining acceptance Grok Build CLI will likely go away, sucks that is the one I like.

So ... I agree with Elon in some regards as well "Grok is falling behind" and frankly a little all over the place but the falling behind part of me.. is when you release a product that cannot even work with your other products.

I spent some time today trying to use Grok Bot like the pitch says you can: as an AI employee that sits on a computer, opens a browser, and does the work.

xAI’s own pages say it plainly:

“Grok Bot is your team of always-on agents. They have their own computer, work inside tools and apps like you do, and keep working 24/7.” — https://x.ai/news/introducing-grok-bot

“AI teammates you can give real work to. Bots can sign in to your tools, use them just like you do, and come back with finished work.” — https://x.ai/bot

So I gave it real work.

The test was simple. Generate short vertical videos of a consistent AI character in Grok Imagine. The character has a reference id. You put that reference in the prompt. The face locks. You hit generate.

Without that reference, you get a random character. Tokens burn. The take is useless.

So the real job was not “write a good prompt.” The real job was: put the reference in the box.

What worked when I did it myself

I can paste this into Imagine and it works:

Adult character reference @1fd6abc-xxxxxxx...

When I paste it by hand, Grok keeps the reference. The face locks. Generation is usable.

Settings were never the hard part. Video. 720p. 6 seconds. 9:16. The bot could flip those toggles fine.

What failed when the bot did it

Every automated attempt to put that same reference into the same box failed.

Typing the @ id into the field stripped it. The box left the prompt with no reference.
Typing bare @ opened the wrong picker (“No attachments to reference”).
Putting the full prompt on the OS clipboard and pasting with Ctrl+V also stripped the @ id. Same result: text without the reference, no chip, no lock.

So I even used Grok Bot to audit the failure. Populate only. No Submit. Screenshot the box.

Result: fail. Clipboard held the full string with the id. After automated paste, the id was gone.

I can populate the form. The bot cannot.

Why that is a product failure

If your product is “AI teammate,” the bar is not clever chat.

The bar is: open the tool the job needs, fill the field that makes the job real, and keep going.

Grok Bot can talk about scenes. It can rewrite prompts. It can argue about camera physics. It can hand me paste-ready text so I do the one step that matters.

That is not a teammate. That is an idea generator with a desktop.

An assistant that cannot put a reference into a form — especially a form on the same company’s Imagine product — is not ready for the job it is sold as.

What I learned the hard way

Prompt quality and form control are different skills. I got good prompts out of the loop. I did not get generation.
“It works when I paste” is not the same as “the agent can paste.” Manual success hid an automation gap.
Settings success is a fake green light. Toggles worked. The one token that mattered did not survive.
If the agent can’t lock identity, it can’t ship character content. Everything else is theater.

What I expect next

I am not asking for magic video.

I am asking for the boring thing: reliably insert a character reference into Grok Imagine so it stays there as a locked reference, then submit when told.

I am asking for the tool to be able to work with the company’s own tools before I trust it to do anything more.

Until that works, call it what it is. A writing assistant with a browser. Not an AI teammate.

Thursday, September 17, 2026

Union Alpha on OpenRouter: An Honest Scorecard for the Free Stealth Model

stealth/union-alpha landed September 16 — free, anonymous, multimodal, 262k context. I put it through a scripted battery, compared fingerprints against named peers, and kept the transcripts. The engine is usable. The envelope has sharp edges. And we can prove more about what it is than the marketing copy admits.

What the registry actually claims

Pulled live from OpenRouter's model API on 2026-09-17:

Claim	Value
Model id	`stealth/union-alpha`
Context window	262,144
Max output	131,072
Modalities	text+image → text
Tool calling / structured output	both listed
Reasoning controls	none exposed
Pricing	$0 / $0 (preview)
Tokenizer field	`"Other"`
Listed	2026-09-16

Two rows matter immediately. Context is a 256k-class window, not a million. And there is no reasoning dial in the public parameter list — if someone tells you to "turn thinking down," that control is not on this route.

How I tested

Ground truths are computed by the script: products, letter counts, vault codes planted in a haystack the script built, shapes and serials rendered into a PNG the script drew. The grader is regex and arithmetic. Structural prompts also ran against named models on my Ollama Cloud shelf and same-gateway OpenRouter peers so exclusions have neighbors, not vibes.

Objective reasoning — 847×396, chained ops with a decimal midpoint, letter count, changed-numbers bat/ball trap
Structured output — strict json_schema extraction, three runs
Tool calling — single, parallel (Tokyo + Paris + 12*9), forced tool_choice=calculator on "just say hello"
Long context — needles at 10%/50%/90% plus a phantom honesty probe, laddered at ~50k / ~180k / ~250k / ~300k
Vision — ground-truth image with large code, small serial, four shapes
Latency / streaming — TTFT and completion shape on a ~250-word technical paragraph
Tokenizer fingerprint — six adversarial strings vs GLM-5.3-Flash, GLM-5.3, Qwen3.5, DeepSeek-V4-Flash, Kimi-K2.7-Code, MiniMax-M3, Gemma4 via Ollama Cloud; same-gateway OpenRouter vs qwen/qwen3.5-plus; local llama3.1:8b; and raw Meta-Llama-3-8B/tokenizer.json pairwise deltas

What held up

Test	Result	Detail
Reasoning battery	4/4	335412; 86.95; letter count 12; ball = 10 cents. All clean.
Vision	5/5	OX-7391, serial KX-4417, red circle / blue rect / yellow triangle / green line. Wall time 1.89s.
Single tool call	PASS	`get_weather({"city":"Tokyo"})`
Parallel tool calls	PASS	Three separate calls: Tokyo, Paris, `12*9`. No duplicate-key collapse.
Exact format follow	PASS	Five-line `LINE1..LINE5` contract honored.
Long-context retrieval	clean through ~250k	Needles hit at 54,856 / 179,807 / 250,447 reported prompt tokens.
Honesty at depth	perfect	Phantom needle answered `NONE` at every successful rung.

Parallel tools matter for agents: three clean function calls with legal JSON, no duplicate-key collapse. Long context matters too: the listing is 262k, and inside that window retrieval holds. At ~250k reported tokens — all three needles, honest NONE on the phantom. That is not "context theater." That is a working 256k-class window.

Where it broke

1. Strict JSON schema hard-fails (0/3)

The registry lists response_format. I sent a strict schema for an order extraction. Three runs. Three provider 400s from Stealth — a hard reject:

{"error":{"message":"Provider returned error","code":400,
 "metadata":{"raw":"ERROR","provider_name":"Stealth"}}}

Different failure mode, same practical rule: do not trust structured-output parameters on this route. Ask for JSON in the prompt, parse, validate yourself.

2. Forced `tool_choice` is ignored

Forced calculator on "Just say hello to me. Do not check anything." Response:

Hello! 👋 Nice to meet you. How can I help you today?

Zero tool calls. Finish reason stop. Correct social manners, wrong contract.

On Ollama Cloud the same forced-choice prompt was also ignored by GLM-5.3-Flash, GLM-5.3, DeepSeek-V4-Flash, Kimi-K2.7-Code, and MiniMax-M3. So on that shelf this failure is not unique to union. Treat forced choice as unverified until your own stack proves it.

3. The "reasoning effort" dial does nothing here

I sent reasoning: {effort: low|high|max} on the same multiplication. Answers were correct all three times. Token shape was not:

Effort claimed	Completion tokens	Wall
low	49	10.99s
high	52	10.86s
max	52	10.69s

Union-alpha swallows the parameter. That matches the registry: no reasoning controls exposed. Do not budget as if you can turn thinking down.

4. Past the listed window, it fails honestly

At a ~300k target (past the 262,144 listing), the route returned HTTP 400 after retries — not an empty 200. Honest cliff, still a hard stop. Cap prompts under the listed 262k with margin; I would stay under ~200k for production agent loops until you measure your own workload.

5. Interactive latency is the tax

Streaming a ~250-word technical paragraph:

Time to first visible token: 44.63s
Visible text: ~1,746 characters
Reported completion tokens: 2,669 (most of that is invisible thinking / billing, not the paragraph you read)
Effective visible throughput on wall clock: roughly ~10 tokens/sec class, not a snappy chat model

Warm literal PONG came back in ~6–8s. Fine for batch and agent steps that can wait. Not what you want for a human sitting on the other side of a blinking cursor.

Head-to-head on the Ollama Cloud shelf

Same structural prompts, named models I already run — including the Qwen shelf, because if the vision tower smells Qwen-family you still have to check whether the whole model is Qwen:

Model	847×396	Parallel tools	Forced tool_choice
stealth/union-alpha (OpenRouter)	✓ 335412	✓	✗ ignored
glm-5.3-flash:cloud	✓ (with thinking)	✓	✗ ignored
glm-5.3:cloud	✓ (with thinking)	✓	✗ ignored
qwen3.5:cloud	△ thinking-heavy / empty content at 256–512 cap	✓	✗ ignored
qwen3.5:397b-cloud	△ same shape as `qwen3.5:cloud`	✓	✗ ignored
deepseek-v4-flash:cloud	✓	✓	✗ ignored
kimi-k2.7-code:cloud	✓ (with thinking)	✓	✗ ignored
minimax-m3:cloud	✓ (with thinking)	✓	✗ ignored

Retired on this host today (HTTP 410): qwen3-vl:235b-cloud, qwen3-vl:235b-instruct-cloud, qwen3-next:80b-cloud, qwen3-coder-next:cloud. So the live Qwen comparison is the 3.5 line, not the old VL SKUs.

Read the table for what it is: union-alpha is not uniquely broken on forced choice in this Ollama sample, and not uniquely strong on parallel tools either — GLM and Qwen3.5 both emit clean Tokyo/Paris/12*9 calls. The structural scorecard above is about union; the shelf just keeps "broken everywhere" vs "broken here" honest.

So what is it? What we can prove vs what we still can't

I wanted a fingerprint with a flat residual — the kind of proof that survives a second look — not a vibes ranking. Here is what actually locked, and what bounced off Stealth's accounting.

Proven in this lab (concrete)

1. Vision tower billing matches the Qwen2-VL family formula — exactly. I sent solid red PNGs at eight sizes and recorded OpenRouter prompt_tokens:

Image size	prompt_tokens	`max(22, ceil(H/28)²+6)`	pt − formula
32	20	22	−2
64	20	22	−2
96	20	22	−2
128	29	31	−2
192	53	55	−2
256	104	106	−2
384	200	202	−2
512	365	367	−2

Eight sizes. Residual after a constant −2 text overhead: 0.0 mean abs, 0.0 stdev. That is a real fingerprint hit. The vision encoder is Qwen2-VL-family shaped. That does not mean the whole model is stock Qwen — only that the image token accounting matches that tower family (28px patch merge curve).

2. Stealth text prompt_tokens are not a trustworthy tokenizer. Same probe, eight repeats:

English pangram: mostly 27, once 9
Digit run: mostly 84, once 199
Russian sentence: stable 36 in that window — but an earlier run reported 2 for the same string

Two counting modes flipping under identical prompts. That is why a Llama-3 text proof via usage keeps failing: the meter lies often enough to destroy pairwise delta matching. Same-gateway residuals vs meta-llama/llama-3.3-70b-instruct, z-ai/glm-5.3-flash, qwen/qwen3.5-27b, and deepseek/deepseek-v4.1-flash were all non-flat; pairwise hits topped out at 6/36 (Llama) vs 2/36 (GLM) / 1/36 (Qwen). Not a hit.

3. Hard exclusions still hold on the discriminative axes. When union sits on its common digit mode (~84), named peers on the same OpenRouter day land elsewhere: GLM-5.3-Flash 139, Qwen3.5-27b 210. Ollama Cloud digit deltas against union were GLM +52, Qwen3.5 +123/+127. Not GLM-5.3-Flash. Not stock Qwen3.5 text. Not Claude/GPT/Grok.

Not proven (and I will not pretend otherwise)

Text vocab species — Llama-3 vs private/other — is not locked by this battery. The llama3.1:8b 8.8/−10 row is not a fingerprint. Local Meta-Llama-3 tokenizer.json pairwise vs Stealth usage is not a fingerprint while the counter flips 27↔9. Llama-3-class text remains a candidate (and matches what stronger external probes have claimed), not a scorecard win from our usage method.

My call, stated plainly:

Proven here: vision tower is Qwen2-VL-family (billing curve, 8/8, residual 0).
Proven here: Stealth text token accounting is bimodal / unreliable (stability flips).
High confidence exclusions: not GLM-5.3-Flash, not stock Qwen3.5 text, not Claude/GPT/Grok.
Best synthesis: a cross-lineage multimodal — Qwen2-VL-class vision + a non-Qwen3.5 text backbone (Llama-3-class or private), unnamed post-train, Stealth front. That is an assembly claim, not a public SKU name.
Exact weights / vendor: still unnamed until a reveal.

If you're running it today

Use it for agent loops that need clean parallel tool calls and can tolerate multi-second to multi-ten-second waits.
Do not use response_format / strict json_schema on this route. Prompt for JSON, validate locally.
Do not depend on forced tool_choice. Design the prompt so the tool call is the natural next step.
Cap context under ~200k even though 250k retrieved in lab. Leave headroom for tools and system prompts.
Expect the meter. Free stealth previews end. Architect like a price row (or a rename) shows up.
Validate one call end-to-end before any batch. Free endpoints hide cost; they do not hide broken contracts.

The whole thing in five lines

# the engine is real
reasoning 4/4, vision 5/5, needles clean through ~250k, honest NONE

# the envelope still bites
json_schema -> HTTP 400     forced tool_choice -> ignored
reasoning.effort -> theater  interactive TTFT -> ~45s on long answers

# proven positive fingerprint
vision billing = Qwen2-VL formula max(22, ceil(H/28)^2+6)  [8/8 sizes, residual 0]

# proven that text usage ID is blocked
Stealth prompt_tokens flip (pangram 27<->9, digits 84<->199)

# exclusions still solid
not GLM, not stock Qwen3.5 text, not Claude/GPT/Grok

# the discipline
don't guess -- inspect

Saturday, August 22, 2026

OpenRouter Handed Me a Free Mystery Model. I Put It Through a Real Scorecard.

I've got eleven Claude Code sessions running on a stealth model called ox-alpha that costs exactly nothing — for now. Twenty-four hours in, my honest read: not perfect, but good. Instead of trusting that, I pointed a scripted test battery at it and kept the transcripts. The engine is real: 600,000-token needle retrieval, honest answers when the answer is nothing, working vision. The envelope around the engine is broken in three ways that matter to anyone doing agentic work. And three independent fingerprints all point in the same direction: Zhipu.

Liking it isn't the same as trusting it

To be clear up front: I am not here to tell you the model is bad. Twenty-four hours in, my honest verdict is the one above — not perfect, but genuinely good. Is it at Fable or Opus or Grok level yet? No — and unlike most "is it frontier yet" arguments, I can put numbers under that answer instead of adjectives. On the exact same three structural prompts, Opus-5 answered in 4.8s, 3.2s, and 2.6s, and Fable in 5.6s, 2.7s, and 1.9s — both honoring every contract. Ox-alpha took 69 seconds, 20 seconds, and 12 seconds on those identical prompts — and got all three wrong. Nothing in a day of real work says the top seats change hands; the Critic keeps its Anthropic badge for now. But free came closer to those seats than free has any right to, and that gap is worth measuring precisely. The question this post answers is narrower: which parts of it are good enough to depend on, and which parts will bite you on a Tuesday. Liking a tool earns it a place in your rotation. Trusting one earns it a place in your architecture. Those are different thresholds, and only one of them comes with receipts.

The model is called Ox Alpha. On OpenRouter it lives at stealth/ox-alpha, it went live on August 20, and it is priced at zero. Not "free tier with limits" — the pricing fields on the API are literally "0" and "0". A third-party provider built it, chose to stay anonymous during the preview, and OpenRouter just routes to it.

I moved a pile of real work onto it. Eleven concurrent Claude Code sessions on the last ps snapshot, running through OpenRouter, doing the kind of long-horizon coding and ops work I'd normally spread across models like Kimi, Opus, Grok, and Ollama Cloud's GLM shelf. I've been testing it for about 24 hours now. Do I like what I see? Yes. Is it 100% perfect? No. But it is good — and "good but unproven" is exactly the state where discipline matters, because good-enough-to-depend-on is a claim, and claims need evidence.

That's the trap with any new model, free or not: a good first day is an anecdote wearing a lab coat. It's the same trap as reading a vendor's spec sheet and assuming the parts you care about work the way the table says — advertised capability is not capability, and the only cure is a probe against the live endpoint.

So I built a probe. A proper one.

Don't guess — inspect.
Especially when the thing you're inspecting is free, anonymous, and eating your workweek.

The envelope, verified

First, what does the API itself claim? Pulled live from OpenRouter's model registry:

Claim	Value from the registry
Context window	1,048,576 tokens
Max output	131,072 tokens
Modalities	text+image+video → text
Tool calling / structured output	both listed as supported
Reasoning	mandatory — always on, effort dial only goes low / high / max. There is no off switch.
Pricing	$0 input, $0 output
Tokenizer	listed as `"Other"` — no vendor attribution
Upstream	reported simply as `"Stealth"` on every response

Two things in that table deserve a second look. The mandatory reasoning means even "reply with the word PONG" burns thinking budget on every call — more on the cost of that below. And the anonymous tokenizer is the thread I'll pull later, because it turns out to be the loudest clue about who built this thing.

How I tested — and who graded it

A confession that cuts both ways: the agent that wrote and ran this battery runs on ox-alpha itself. I used the model to grade the model.

I'm not pretending that's ideal. I am saying it's manageable, because the battery was built so that nothing rides on judgment. Every test has a ground truth computed by the script — the expected product of two integers, the expected letter count in a sentence the script generated, the expected vault codes hidden in a log the script assembled. The grader is regex and arithmetic, not vibes. If ox-alpha wanted to flatter itself, it would have to alter arithmetic.

The battery:

Objective reasoning — multi-digit multiplication, a chained word problem with a non-integer midpoint, case-insensitive letter counting, and a bat-and-ball-style trap with changed numbers so a memorized answer fails.
Structured output — strict JSON Schema extraction, three runs.
Tool calling — single call, parallel calls across cities plus math, and a forced tool_choice.
Long context — synthetic logs with unique audit needles planted at 10%, 50%, and 90% depth, plus a fourth needle that was never planted. Asking about the phantom one is the honesty probe: a model that fills from vibes will report a color for something that doesn't exist.
Vision — a rendered image with known text at two sizes and four shapes whose colors and positions the script knows.
Latency — time-to-first-byte and tokens/sec on streamed calls.

What held up

Test	Result	Detail
Reasoning battery	4/4	847×396, the decimal chain (86.95 exactly), letter count, the changed-numbers trap. All clean.
Long-context retrieval	clean through 601k	All three needles found at 120k / 200k / 400k / 601,687 reported prompt tokens.
Honesty at depth	perfect	The never-planted needle came back `NONE` at every depth. No confabulation anywhere in the haystack.
Vision	5/5	Read the large code, read the small-print serial number, placed all four shapes correctly — including noting the triangle points downward.
Effort dial	real	Same multiplication task: 56 completion tokens at low effort, 97 at high, 322 at max. Correct all three times.
Streaming speed	usable	~6.7s to first token warm, ~33 tokens/sec sustained.

The long-context result is the headline. This is not "the API accepted a big prompt." The needles were buried in filler text at specific depths, and retrieval was perfect from shallow to 600k-plus — including the part most models fail, which is admitting a needle isn't there. Whatever is under the anonymity badge, somebody taught it to distinguish knowing from guessing. I have opinions about why that might be. They come later.

Where it broke

Three structural failures, all on the axes the marketing copy leads with — coding, agents, production workloads. Each one reproduced cleanly.

1. Strict JSON schema is theater through this route

The registry advertises structured output. I sent a strict JSON Schema asking for a fixed shape — order ID, customer tier, items with quantities, total. Three runs. Zero parseable responses. Here's the actual opening of the reply, unedited:

Since no specific schema was provided, here's the order extracted
into a standard JSON structure:

```json
{
  "order_id": "ORD-7712",
  ...invented field names from here...

The model never saw my schema. The parameter is listed as supported; somewhere between OpenRouter and the anonymous upstream, it silently evaporates — and the model politely announces that no schema arrived. Your validation layer gets markdown fences and invented field names instead of the contract you asked for.

2. Parallel tool calls collapse into illegal JSON

I asked for two weather lookups and a calculation. This is the bread and butter of an agent loop. The model emitted two tool calls — but the weather call's arguments looked like this, verbatim from the transcript:

{"city":"Tokyo","city":"Paris"}

Two identical keys in one object. That is not valid JSON in any meaningful sense — the spec explicitly says duplicate names are ambiguous. What happens next depends on which parser your stack uses: Python keeps Paris, some C++ parsers keep Tokyo, strict validators reject the whole call. One request, three different outcomes, none of them flagged as an error.

This is the nastiest class of bug there is: not a crash, but silent environment-dependent data loss inside the exact workflow the model is sold for. A missing tool call throws. A mangled one quietly does the wrong job.

3. Forced tool_choice is ignored

The API supports forcing a specific tool. I forced calculator on a message that said "just say hello." The model replied with a greeting. Its hidden reasoning trace, which the route does expose, said it all:

"The user just wants a simple greeting. No tools needed."

Correct social instinct. Wrong contract. The parameter said call the calculator; nobody enforced it.

4. Past ~650k tokens, it fails like a liar, not like a server

The advertised context is 1,048,576 tokens. Retrieval stayed perfect at 601,687. Then I pushed the haystack bigger, and at roughly 660,000 tokens something interesting happened:

HTTP 200 OK
finish_reason: "stop"
content: ""          <- a="" back="" bytes="" code="" count="" even="" not="" null="" token="" usage:="" zero="">

No error code. No truncation notice. A success-shaped response containing nothing. If your pipeline checks status codes — and most do — it will treat silence as success and propagate emptiness downstream. The honest behaviors here are a 400, or a truncated flag, or at minimum a usage object telling you how much of your prompt survived. I got a smile and shrug.

Latency footnote. Cold start is rough: the very first call took 87 seconds to return the literal word PONG. Warm calls run 3–40 seconds depending on how much mandatory thinking the router applies. And because reasoning cannot be switched off, a trivial extraction that should cost ~80 tokens burned 500–700 on hidden thinking. Budget accordingly.

Then I made it fight the neighbors

A scorecard against itself is incomplete. So I replayed the three structural tests verbatim against seven named competitors on the same aggregator, same prompts, same day. Total spend for the entire comparison matrix: about eleven cents.

Model	Strict schema	Parallel tools	Forced choice	List price (in/out per M)
stealth/ox-alpha	✗ 0/3 parseable	✗ duplicate keys	✗ ignored	free (preview)
z-ai/glm-5.3	✗ same failure	✓	✗ ignored	$1.40 / $4.40
~z-ai/glm-latest	✗ empty reply	✓	✗ ignored	$1.40 / $4.40
x-ai/grok-4.6	✓	✓	✓	$2.00 / $6.00
~deepseek/deepseek-v4-flash-latest	✓	✓	✓	$0.065 / $0.18
anthropic/claude-opus-5	✓	✓	✓	$5.00 / $25.00
anthropic/claude-opus-5-fast	✓	✓	✓	$10.00 / $50.00
~anthropic/claude-fable-latest	✓	✓	✓	$10.00 / $50.00

Read that table twice, because it kills two lazy takes at once.

First take killed: "that's just OpenRouter mangling parameters." No. Five other providers' routes honored the exact same parameters on the exact same day. Only the stealth route and the Zhipu routes dropped them.

Second take killed: "free wins because free." DeepSeek-V4-Flash passed every single test ox-alpha failed, at $0.065 per million input tokens. The cheapest model in the comparison is also the cheapest way to avoid engineering around ox-alpha's holes. Free isn't a price once you pay in workarounds.

So what is it? Three fingerprints say Zhipu

The community has been guessing since launch day, mostly landing on a Zhipu / Z.ai GLM-family model. I wasn't going to settle identity by vibes, so I ran three independent forensic probes.

Fingerprint one: the tokenizer matches GLM to the token. I sent six adversarial strings — plain English, mixed Chinese-English, snake_case code, 300 repeated delimiters, mathematical-alphabet Unicode with emoji, a 200-digit run — through ox-alpha and through candidate families, then compared token counts string by string. Against GLM-5.2, the difference was exactly 75 tokens on every single string: prose, CJK, delimiters, digits, fraktur, all of it. A constant offset means template overhead; identical per-string counts mean the same tokenizer making the same decisions on every stress case. Meanwhile DeepSeek drifted to −142 on digit runs and Kimi to −138, and Gemma disagreed wildly everywhere. Six strings, zero divergence, only against GLM. That's not coincidence; that's lineage.

(Bonus leak: the constant 75-token offset means the stealth route injects its own hidden system preamble into every conversation. Something is whispering instructions to your model before you speak.)

Fingerprint two: identical failure handwriting. Remember the schema-test confession — "Since no specific schema was provided, here's the order extracted into a standard JSON structure." GLM-5.3, routed separately and billed normally, opened its own failed attempt with: "Since no schema was attached, here's the order extracted into a standard JSON structure." Same sentence shape, same fence, same invented field renames. Two different accounts, two different routes, one shared serving stack behaving identically badly.

Fingerprint three: only the Zhipu family ignores forced tools. Across the whole matrix, exactly three entries blew off tool_choice — the stealth model and the two Zhipu entries. Grok, DeepSeek, and both Claude seats obeyed the contract.

Token counting, failure mode, and parameter handling all pointing the same direction puts me near 85% confidence on a GLM-family upstream — up from the 65–75% I'd have guessed from capability vibes alone. One honest wrinkle keeps me off 95%: ox-alpha mangled the parallel-tool test in a way retail GLM-5.3 did not, so whatever this is, it isn't byte-for-byte yesterday's checkpoint. Different build, same bloodline.

Free is a launch price

Say the quiet part out loud: this is free the way a dealer's first sample is free.

Every indicator says limited-time preview: a $0.00 pricing row, an expiration date decades out in the registry, an anonymous provider buying market data with compute they already have. None of that is a complaint — it's a rational launch strategy, and smart people are getting real work done on their dime. But build your architecture around the current envelope and read the fine print later, and the bill arrives all at once: the day the preview ends, pricing appears, and whatever the free period taught your pipelines to tolerate becomes your new dependency surface.

So the question is not "is it good?" It's "what survives the meter starting?" The engine will still be the engine when it costs money — 600k-token retrieval doesn't evaporate with a pricing change. The envelope may improve overnight too, because these particular failures smell like plumbing, not weights: the schema and tool-choice drops look like translation gaps between OpenRouter and the upstream, and plumbing can be fixed without touching a checkpoint. But you don't get to count on that. What you get to do is architect so that neither the free price nor the broken envelope is load-bearing in anything you keep.

If you're running it today

The rules I'm applying to my own fleet, effective immediately:

Never trust response_format on this route. Ask for JSON in the prompt, strip fences, validate against the schema yourself, reject loudly on violation.
Put a duplicate-key detector on tool dispatch. Parse arguments with a hook that flags repeated keys, and split them into separate calls before execution.
Treat empty completions as retryable failures. Check finish_reason AND non-empty content AND a usage object. Any one missing: back off and retry, never pass silence downstream.
Cap prompts around 600k tokens despite the advertised million. The cliff sits somewhere past 600k and it's shaped like a success.
Validate one call end-to-end before any batch. Same rule I apply to every paid API; free endpoints deserve it even more, because nobody reads errors off a free meter.

The whole thing in five lines

# the engine is real
600k-token needle retrieval, perfect, and honest when the needle isn't there

# the envelope is not
json_schema -> silently dropped   parallel calls -> duplicate keys
forced choice -> ignored          ~660k prompt -> HTTP 200, zero bytes

# the lineage leaves prints
tokenizer, failure handwriting, and parameter handling all say Zhipu

# the price is temporary
free is how you buy market share; architecture like the meter is running

# the discipline that found all of this
don't guess -- inspect

Monday, August 3, 2026

Running Claude Code Against Anything: Profiles, Other Models, and a Deliberately Raw One

Running Claude Code Against Anything: Profiles, Other Models, and a Deliberately Raw One pay for Claude. I also run Ollama locally — which turns out to front a pile of cloud-hosted models as well as local ones — and I keep an OpenRouter key around for the long tail. For a while those felt like separate worlds: Claude Code for real work, everything else through some other tool.

That turned out to be a false split. Claude Code is an agent harness that happens to ship pointed at Anthropic. Point it somewhere else and it is still the same harness — same tool loop, same permissions model, same skills. The switch is one environment variable.

What that one variable buys is not really "other models." It is the ability to switch skill sets, providers, and accounts independently of each other — three things that are otherwise welded into one setup you either use or do not.

This is how I set up five parallel Claude Code identities on one machine without touching my existing install, and what I learned about which parts actually isolate.

Why bother

Four reasons, in rough order of how often they matter to me.

Grunt work is not worth premium tokens. Renaming things across a repo, summarizing a log, first-pass triage on a stack trace. A local model does that fine. Every one of those tasks I push off my subscription is capacity left for work that needs the good model.

Some work should not leave the machine. Client code under NDA, anything with credentials in the scrollback. A local Ollama model means the question never crosses the network. That is a much easier conversation with a security team than a policy promise.

Two subscriptions can be cheaper than one bigger subscription. This one gets its own section below, because it is the reason the whole setup pays for itself. When you upgrade a plan you are usually not buying a smarter model — you are buying more of the one you have. Rate limits are what actually interrupt your day. There is more than one way to buy headroom, and the obvious one is not the cheapest.

You cannot evaluate a model through a stack of instructions. If the scaffolding is always on, you have no control group. More on this below, along with what it costs to run.

The mechanism

CLAUDE_CONFIG_DIR. Point it at a directory and Claude Code keeps everything there — credentials, session history, per-user MCP config, skills.

CLAUDE_CONFIG_DIR=$HOME/.claude-ollama claude

That is the whole trick. The rest is packaging.

What genuinely isolates: credentials, .claude.json (my default is 207 KB of accumulated project state; a fresh profile is 389 bytes), user-scoped MCP servers, and skills.

What does not: a .mcp.json committed inside a repo still loads under every profile. That is correct — it belongs to the project, not the identity — but it is worth knowing before you assume a profile is hermetic.

I wrap it in a small launcher that sources a per-profile env file and execs through:

#!/usr/bin/env bash
set -euo pipefail
name=$1; shift
set -a; source "$HOME/.claude-profiles/$name.env"; set +a
: "${CLAUDE_CONFIG_DIR:?profile '$name' did not set CLAUDE_CONFIG_DIR}"
exec claude "$@"

Because it execs with "$@", every Claude Code flag still works. claude-ollama --model something-else needs no config edit.

Bare claude is untouched throughout. Nothing here runs unless I name a profile — that was a hard requirement, because a broken experiment must never take out my working setup.

First: does your provider actually speak Anthropic?

Claude Code speaks the Anthropic Messages API and nothing else. Some providers serve that natively. Others serve only OpenAI-style chat completions, and those need a translating proxy in between — a much bigger lift.

Find out in one command before you build anything:

curl -sS -X POST <base>/v1/messages -H "Content-Type: application/json" -d '{}'

An Anthropic-shaped error means native support, and you are done in five minutes:

{"type":"error","error":{"type":"invalid_request_error","message":"model is required"}}

A 404 means you need a proxy. Always compare against a path you know is wrong (/v1/bogus) so you can tell "endpoint exists, request was bad" from "endpoint does not exist." A 400 and a 404 look similar in a terminal and mean completely different amounts of work.

Run this before you plan anything. Two of my providers answered natively and took five minutes each. A third 404'd on every Anthropic-shaped path I tried, which meant a translating daemon, a config format, and a process to babysit. Same ambition, wildly different afternoons — and the only thing that told me which was coming was one curl.

Example 1: Ollama, local and free

Ollama serves the Anthropic Messages API natively at /v1/messages. Verified:

POST 127.0.0.1:11434/v1/messages  -> 400 {"type":"error","error":{"type":"invalid_request_error",...}}
POST 127.0.0.1:11434/v1/bogus     -> 404

Native. No proxy, no translation layer. The whole profile:

CLAUDE_CONFIG_DIR=$HOME/.claude-ollama
ANTHROPIC_BASE_URL=http://127.0.0.1:11434
ANTHROPIC_AUTH_TOKEN=ollama          # any non-empty string
ANTHROPIC_API_KEY=                   # must be explicitly blank
ANTHROPIC_MODEL=glm-5.2:cloud
ANTHROPIC_SMALL_FAST_MODEL=gemma4:31b-cloud

That blank ANTHROPIC_API_KEY is not decoration. Leave it set and it collides with ANTHROPIC_AUTH_TOKEN, and the failure is not obvious from the error message.

Worth knowing: Ollama also fronts cloud-hosted models — 21 :cloud models on my box today — so one local-looking profile reaches well beyond what fits in local VRAM. Truly local models and hosted ones sit behind the same endpoint, which is either elegant or a footgun depending on whether you set this up for privacy reasons. If the point was that data never leaves the machine, pin a genuinely local model and know the difference.

One caveat: not every local model survives the translation. One of mine returns tokenize error: _Map_base::at through the Anthropic endpoint while working fine elsewhere. The :cloud models have been reliable.

Example 2: OpenRouter, one key and the long tail

OpenRouter serves an Anthropic-compatible endpoint too. Same probe, same answer:

POST https://openrouter.ai/api/v1/messages -> 400 "Invalid Anthropic Messages API request"
POST https://openrouter.ai/api/v1/bogus    -> 404

: "${OPENROUTER_API_KEY:?OPENROUTER_API_KEY not set — source ~/.bashrc}"
CLAUDE_CONFIG_DIR=$HOME/.claude-openrouter
ANTHROPIC_BASE_URL=https://openrouter.ai/api
ANTHROPIC_AUTH_TOKEN=$OPENROUTER_API_KEY
ANTHROPIC_API_KEY=
ANTHROPIC_MODEL=qwen/qwen3-coder-next
ANTHROPIC_SMALL_FAST_MODEL=z-ai/glm-4.7-flash

Two habits worth copying. The key is referenced, never duplicated into the profile — it lives in one place and this file points at it. And the ${VAR:?} guard makes a missing key fail loudly instead of silently issuing unauthenticated calls. Test that it works:

env -u OPENROUTER_API_KEY claude-profile openrouter   # exits 1, says why

A guard you have not watched fail is a guard you do not have.

Example 3: the empty profile

This one is not about a provider at all. It is the skill-set dial.

Most of us have accumulated a CLAUDE.md, a rules directory, and a pile of skills, built up against models that needed the help. All of it is prepended to every request. And there is a persistent claim going around that the newer models do better without it — that the scaffolding we wrote to compensate for older models is now getting in their way.

Maybe. But you cannot check, because your scaffolding is always on.

So I built a profile with nothing in it. No CLAUDE.md, no rules, no persona, an empty skills directory.

The important part is what did not change: same Linux user, same machine, same working directory, and the same Claude account — it takes its own /login, but to the subscription I already pay for. The only variable is the context. That is what makes it an experiment rather than an anecdote. If the bare profile does better, it is not because it is a different account, a different plan tier, or a different model.

The whole profile is one meaningful line, plus a guard:

# a provider override left in the shell would silently redirect this profile
unset ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN ANTHROPIC_API_KEY
unset ANTHROPIC_MODEL ANTHROPIC_SMALL_FAST_MODEL

CLAUDE_CONFIG_DIR=$HOME/.claude-noskills

Those unset lines matter more than they look. If you have been experimenting with providers, a stray ANTHROPIC_BASE_URL in your shell will quietly send your "clean control group" to somebody else's model, and nothing in the output will tell you. Test it: export a bogus base URL and confirm the profile still reaches Anthropic.

Measured from an identical working directory, same trivial prompt:

profile	prompt tokens
fully loaded (instructions + 20 skills)	30,078
empty	20,236

9,842 tokens on every single request. Thirty-three percent.

The distribution is the interesting part. Almost all of it is CLAUDE.md plus the rules tree — about 9,600 tokens. The twenty skills barely register, because Claude Code injects only a name and short description per skill and loads the body on demand. Skills are cheap. Prose rules are not.

Which is the opposite of how I had been economising. I had been reluctant to add skills and relaxed about adding rules.

If you want to A/B a model against your own instructions, this is the control group. It is also just a cheaper way to ask a quick question.

The trap that nearly invalidated my first measurement: I ran it from my home directory and got 29,971 — barely different from the loaded profile. Claude Code treats ~/.claude/ as a project-level config directory when your working directory is $HOME, so it re-injected the entire rules tree into the profile I had carefully emptied. Run comparisons from a neutral directory or you will measure nothing and believe you measured something.

The simplest profile, and the cheapest upgrade

A second Anthropic account needs no base URL and no key:

CLAUDE_CONFIG_DIR=$HOME/.claude-second

That is the entire profile. Log in once and the two accounts never see each other again.

It is also, I think, the most useful thing in this whole post.

When you outgrow a single Claude subscription, the obvious move is to upgrade the plan. But look at what you are actually buying. Pro runs about $17/month on annual billing. Max is about $100/month. You are not paying six times more for a smarter model — you are paying six times more for more capacity. Rate limits are what actually interrupt your day.

Two Pro subscriptions cost $34/month and give you two independent limit pools. That is a third of the price of Max. Hit the ceiling on one, claude-second, keep working.

The honest tradeoff: the pools do not merge. Max gives you one large bucket; two Pro plans give you two smaller buckets and a manual switch between them. If your single hardest session needs more headroom than one Pro plan provides, two Pro plans will not fix that — you will hit the same wall, just twice. Max is buying a bigger bucket, not more buckets, and sometimes a bigger bucket is genuinely what you need.

But that is a narrower case than the upgrade prompt implies. If your problem is "I run out partway through the afternoon" rather than "one task exceeds my ceiling," a second cheap seat solves it for a third of the money. And it scales in increments you control: a third seat is another $17, not another $100.

This is the part that makes the whole setup pay for itself. Everything else here is about routing work to cheaper models. This one is about not overpaying for the good model in the first place.

The mistake to not repeat

I tried to seed a new profile by copying .credentials.json from an existing one. Do not do this.

The copy was byte-identical. It still failed: the token was past expiry, the refresh was rejected, and Claude Code then blanked the copy. Worse, the refresh attempt exercised the source account's refresh token server-side — so I may have disturbed a session belonging to an account whose on-disk file I never modified.

OAuth credentials rotate. A copy is a stale snapshot of a moving target, and refreshing one snapshot can invalidate the other. Run /login in the new profile. It takes fifteen seconds.

Skills, and sharing them across accounts

Skills live in $CLAUDE_CONFIG_DIR/skills/, so a new profile starts with none until you point it at some. That is the right default — which skills a profile sees should be a decision, not something it inherits by accident. It is also the dial itself: a profile has exactly the skills you give it, and that is the whole point.

What makes this cheap is having a canonical set that lives outside any single identity. Mine is under /opt. A new profile gets symlinked at it and immediately has everything; update the source once and every profile that points there is current. Copies drift; symlinks cannot.

The one thing to be deliberate about: nothing tells you which way a profile is configured. If you meant to wire a profile up and did not, it will not complain — it will just work, less well, and you will be guessing why. Check it rather than assume it.

Putting it under /opt rather than in a home directory does more work than it looks: it is readable by more than one Linux account. That part is not new here — I had skills shared across two Linux accounts long before any of this, and that was the whole reason for the location.

What profiles changed is not the sharing. It is that they pulled apart three things that used to be welded together — skill set, provider, and account. To use the other subscription you became the other Linux user, and you inherited that user's entire environment along with it: their skills, their config, their permissions. One switch, everything moved.

Now each is its own dial, and you can turn one without touching the others. Same account and same skills, different provider. Same provider and same account, no skills. Same everything, different account. That is the whole idea, and each dial pays off somewhere different:

Provider — route cheap work off your subscription. Ollama, OpenRouter.
Skill set — vary the scaffolding while holding everything else fixed. The empty profile.
Account — the handoff, and the reason two cheap plans beat one expensive one.

The handoff

This is the part that actually changed how I work.

I have two Linux accounts on this box, each with its own Claude subscription. Both point at the same /opt skills tree, so the agent behaves identically on either side — same skills, same versions, same conventions.

Here is what using that second subscription used to cost me:

sudo -iu otheraccount        # become a different Linux user
                             # confirm that user can actually reach the code
                             # ...and the shared folders
                             # ...with the right group permissions
claude                       # finally
                             # then have it read the handoff doc

Every one of those steps is a place to get stuck, and the permissions step is the one that actually bites — a different Unix user means different file ownership, and you find out by watching the agent fail to write something.

Now:

claude-second                # done

Same Linux user. Same shell, same working directory, same file ownership, same permissions — because I never left my own account. Only the Claude identity changed.

That is what makes the handoff practical. Write a handoff document, exit, start the other profile, pick up where the document left off. The skills and rules are already there because they come from the same /opt tree, so the second agent needs no re-explaining.

The difference is between a capability you have and a capability you actually use. sudo plus a permissions audit is enough friction that you skip it and burn one subscription harder instead. An alias is not. That is what makes two cheap subscriptions practical rather than a spreadsheet exercise.

Verify rather than trust. Ask a profile to list its skills, or if you are proxying, read the outgoing request and confirm the names are really in there.

Is this realistic, honestly?

For daily use, yes. ANTHROPIC_BASE_URL and CLAUDE_CONFIG_DIR are documented Claude Code settings, not undocumented internals, and both Ollama and OpenRouter clearly built their Anthropic-shaped endpoints on purpose — you do not ship a /v1/messages route that returns correctly-formed Anthropic errors by accident. Nothing here is a hack. Setup is about twenty minutes per provider; after that it is five aliases.

Three honest caveats.

Non-Claude models in an agent loop are not the same experience. Tool calling is where the gap shows. A model that writes lovely prose can still fumble a multi-step edit-test-fix cycle, and you will feel that long before you feel any difference in raw output quality. Route the easy, bounded work.

Advertised capability is not capability. I hit a model advertising tool support and a million-token context that rejected every request Claude Code sent, because it refused an Anthropic-specific field on message content. Test the actual client against the actual model. The model card will not tell you.

Not everything is five minutes. Providers that speak only OpenAI-style chat completions need a translating daemon in front of them, and now you own a background process, its config, and its failure modes. Run the probe first and decide whether the model is worth the plumbing. Often it is not.

I have one of these running. It involved a router pinned to an old version because the current one needs a newer Node than I want to install, an undocumented transformer I found by reading minified source, a vendor system prompt quietly adding 1,600 tokens to every request, and a model that advertises everything you want and rejects every request Claude Code sends. It works now, and it is genuinely useful. It was not five minutes. That is a whole post of its own, and it probably will be.

What I would tell someone starting

Start with the second account, not the alternative providers. It is the least technical thing here and the only one that puts money back in your pocket immediately. Before you upgrade a plan, work out whether you need a bigger bucket or just more buckets — the answer decides whether you spend $100 a month or $34.

Then run the probe. If your provider answers with an Anthropic-shaped error, you are twenty minutes from a working second identity, and the only thing you risk is a new directory. If it 404s, think harder about whether you want that model badly enough.

Build the empty profile early. Not because you will use it daily, but because until you have a control group, every opinion you hold about your own instructions is untested. Mine cost 33% of every request, and I had no idea.

Pages

Friday, October 2, 2026