Sunday, September 27, 2026

GROKBOT :: I Asked an “AI Teammate” to Fill One Form. It Couldn’t.

I have been a fan of Grok for a long time now..  I endured the lack of a CLI when every other provider offered it. I saw how it was not included originally with CLI tools providers like Cursor for a long time too. 

Then they released Grok Build and a CLI tool. I like it.. I am not thrilled with the usage quota options and the billing felt very odd at the start (I posted about that before) but better once it opened it up to Supergrok users. Now of X bought Cursor, so now Grok basically has 2 CLI tools. Never feels like a smart business move to have 2 competing products under your own roof, so clearly one of these will fade away, time will tell.  I think now that GrokBot it gaining acceptance Grok Build CLI will likely go away, sucks that is the one I like. 

So ... I agree with Elon in some regards as well  "Grok is falling behind" and frankly a little all over the place but the falling behind part of me.. is when you release a product that cannot even work with your other products. 

I spent some time today trying to use Grok Bot like the pitch says you can: as an AI employee that sits on a computer, opens a browser, and does the work.

xAI’s own pages say it plainly:

“Grok Bot is your team of always-on agents. They have their own computer, work inside tools and apps like you do, and keep working 24/7.” — https://x.ai/news/introducing-grok-bot

“AI teammates you can give real work to. Bots can sign in to your tools, use them just like you do, and come back with finished work.” — https://x.ai/bot

So I gave it real work.

The test was simple. Generate short vertical videos of a consistent AI character in Grok Imagine. The character has a reference id. You put that reference in the prompt. The face locks. You hit generate.

Without that reference, you get a random character. Tokens burn. The take is useless.

So the real job was not “write a good prompt.” The real job was: put the reference in the box.


What worked when I did it myself

I can paste this into Imagine and it works:

Adult character reference @1fd6abc-xxxxxxx...

When I paste it by hand, Grok keeps the reference. The face locks. Generation is usable.

Settings were never the hard part. Video. 720p. 6 seconds. 9:16. The bot could flip those toggles fine.

What failed when the bot did it

Every automated attempt to put that same reference into the same box failed.

  • Typing the @ id into the field stripped it. The box left the prompt with no reference.
  • Typing bare @ opened the wrong picker (“No attachments to reference”).
  • Putting the full prompt on the OS clipboard and pasting with Ctrl+V also stripped the @ id. Same result: text without the reference, no chip, no lock.

So I even used Grok Bot to audit the failure. Populate only. No Submit. Screenshot the box.

Result: fail. Clipboard held the full string with the id. After automated paste, the id was gone.

I can populate the form. The bot cannot.

Why that is a product failure

If your product is “AI teammate,” the bar is not clever chat.

The bar is: open the tool the job needs, fill the field that makes the job real, and keep going.

Grok Bot can talk about scenes. It can rewrite prompts. It can argue about camera physics. It can hand me paste-ready text so I do the one step that matters.

That is not a teammate. That is an idea generator with a desktop.

An assistant that cannot put a reference into a form — especially a form on the same company’s Imagine product — is not ready for the job it is sold as.

What I learned the hard way

  1. Prompt quality and form control are different skills. I got good prompts out of the loop. I did not get generation.
  2. “It works when I paste” is not the same as “the agent can paste.” Manual success hid an automation gap.
  3. Settings success is a fake green light. Toggles worked. The one token that mattered did not survive.
  4. If the agent can’t lock identity, it can’t ship character content. Everything else is theater.


What I expect next

I am not asking for magic video.

I am asking for the boring thing: reliably insert a character reference into Grok Imagine so it stays there as a locked reference, then submit when told.

I am asking for the tool to be able to work with the company’s own tools before I trust it to do anything more.

Until that works, call it what it is. A writing assistant with a browser. Not an AI teammate.

 

Thursday, September 17, 2026

Union Alpha on OpenRouter: An Honest Scorecard for the Free Stealth Model

Union Alpha on OpenRouter: An Honest Scorecard for the Free Stealth Model

stealth/union-alpha landed September 16 — free, anonymous, multimodal, 262k context. I put it through a scripted battery, compared fingerprints against named peers, and kept the transcripts. The engine is usable. The envelope has sharp edges. And we can prove more about what it is than the marketing copy admits.


What the registry actually claims

Pulled live from OpenRouter's model API on 2026-09-17:

ClaimValue
Model idstealth/union-alpha
Context window262,144
Max output131,072
Modalitiestext+image → text
Tool calling / structured outputboth listed
Reasoning controlsnone exposed
Pricing$0 / $0 (preview)
Tokenizer field"Other"
Listed2026-09-16

Two rows matter immediately. Context is a 256k-class window, not a million. And there is no reasoning dial in the public parameter list — if someone tells you to "turn thinking down," that control is not on this route.


How I tested

Ground truths are computed by the script: products, letter counts, vault codes planted in a haystack the script built, shapes and serials rendered into a PNG the script drew. The grader is regex and arithmetic. Structural prompts also ran against named models on my Ollama Cloud shelf and same-gateway OpenRouter peers so exclusions have neighbors, not vibes.

  • Objective reasoning — 847×396, chained ops with a decimal midpoint, letter count, changed-numbers bat/ball trap
  • Structured output — strict json_schema extraction, three runs
  • Tool calling — single, parallel (Tokyo + Paris + 12*9), forced tool_choice=calculator on "just say hello"
  • Long context — needles at 10%/50%/90% plus a phantom honesty probe, laddered at ~50k / ~180k / ~250k / ~300k
  • Vision — ground-truth image with large code, small serial, four shapes
  • Latency / streaming — TTFT and completion shape on a ~250-word technical paragraph
  • Tokenizer fingerprint — six adversarial strings vs GLM-5.3-Flash, GLM-5.3, Qwen3.5, DeepSeek-V4-Flash, Kimi-K2.7-Code, MiniMax-M3, Gemma4 via Ollama Cloud; same-gateway OpenRouter vs qwen/qwen3.5-plus; local llama3.1:8b; and raw Meta-Llama-3-8B/tokenizer.json pairwise deltas

What held up

TestResultDetail
Reasoning battery4/4335412; 86.95; letter count 12; ball = 10 cents. All clean.
Vision5/5OX-7391, serial KX-4417, red circle / blue rect / yellow triangle / green line. Wall time 1.89s.
Single tool callPASSget_weather({"city":"Tokyo"})
Parallel tool callsPASSThree separate calls: Tokyo, Paris, 12*9. No duplicate-key collapse.
Exact format followPASSFive-line LINE1..LINE5 contract honored.
Long-context retrievalclean through ~250kNeedles hit at 54,856 / 179,807 / 250,447 reported prompt tokens.
Honesty at depthperfectPhantom needle answered NONE at every successful rung.

Parallel tools matter for agents: three clean function calls with legal JSON, no duplicate-key collapse. Long context matters too: the listing is 262k, and inside that window retrieval holds. At ~250k reported tokens — all three needles, honest NONE on the phantom. That is not "context theater." That is a working 256k-class window.


Where it broke

1. Strict JSON schema hard-fails (0/3)

The registry lists response_format. I sent a strict schema for an order extraction. Three runs. Three provider 400s from Stealth — a hard reject:

{"error":{"message":"Provider returned error","code":400,
 "metadata":{"raw":"ERROR","provider_name":"Stealth"}}}

Different failure mode, same practical rule: do not trust structured-output parameters on this route. Ask for JSON in the prompt, parse, validate yourself.

2. Forced tool_choice is ignored

Forced calculator on "Just say hello to me. Do not check anything." Response:

Hello! 👋 Nice to meet you. How can I help you today?

Zero tool calls. Finish reason stop. Correct social manners, wrong contract.

On Ollama Cloud the same forced-choice prompt was also ignored by GLM-5.3-Flash, GLM-5.3, DeepSeek-V4-Flash, Kimi-K2.7-Code, and MiniMax-M3. So on that shelf this failure is not unique to union. Treat forced choice as unverified until your own stack proves it.

3. The "reasoning effort" dial does nothing here

I sent reasoning: {effort: low|high|max} on the same multiplication. Answers were correct all three times. Token shape was not:

Effort claimedCompletion tokensReasoning tokens reportedWall
low49010.99s
high52010.86s
max52010.69s

Union-alpha swallows the parameter. That matches the registry: no reasoning controls exposed. Do not budget as if you can turn thinking down.

4. Past the listed window, it fails honestly

At a ~300k target (past the 262,144 listing), the route returned HTTP 400 after retries — not an empty 200. Honest cliff, still a hard stop. Cap prompts under the listed 262k with margin; I would stay under ~200k for production agent loops until you measure your own workload.

5. Interactive latency is the tax

Streaming a ~250-word technical paragraph:

  • Time to first visible token: 44.63s
  • Visible text: ~1,746 characters
  • Reported completion tokens: 2,669 (most of that is invisible thinking / billing, not the paragraph you read)
  • Effective visible throughput on wall clock: roughly ~10 tokens/sec class, not a snappy chat model

Warm literal PONG came back in ~6–8s. Fine for batch and agent steps that can wait. Not what you want for a human sitting on the other side of a blinking cursor.


Head-to-head on the Ollama Cloud shelf

Same structural prompts, named models I already run — including the Qwen shelf, because if the vision tower smells Qwen-family you still have to check whether the whole model is Qwen:

Model847×396Parallel toolsForced tool_choice
stealth/union-alpha (OpenRouter)✓ 335412✓✗ ignored
glm-5.3-flash:cloud✓ (with thinking)✓✗ ignored
glm-5.3:cloud✓ (with thinking)✓✗ ignored
qwen3.5:cloud△ thinking-heavy / empty content at 256–512 cap✓✗ ignored
qwen3.5:397b-cloud△ same shape as qwen3.5:cloud✓✗ ignored
deepseek-v4-flash:cloud✓✓✗ ignored
kimi-k2.7-code:cloud✓ (with thinking)✓✗ ignored
minimax-m3:cloud✓ (with thinking)✓✗ ignored

Retired on this host today (HTTP 410): qwen3-vl:235b-cloud, qwen3-vl:235b-instruct-cloud, qwen3-next:80b-cloud, qwen3-coder-next:cloud. So the live Qwen comparison is the 3.5 line, not the old VL SKUs.

Read the table for what it is: union-alpha is not uniquely broken on forced choice in this Ollama sample, and not uniquely strong on parallel tools either — GLM and Qwen3.5 both emit clean Tokyo/Paris/12*9 calls. The structural scorecard above is about union; the shelf just keeps "broken everywhere" vs "broken here" honest.


So what is it? What we can prove vs what we still can't

I wanted a fingerprint with a flat residual — the kind of proof that survives a second look — not a vibes ranking. Here is what actually locked, and what bounced off Stealth's accounting.

Proven in this lab (concrete)

1. Vision tower billing matches the Qwen2-VL family formula — exactly. I sent solid red PNGs at eight sizes and recorded OpenRouter prompt_tokens:

Image sizeprompt_tokensmax(22, ceil(H/28)²+6)pt − formula
322022−2
642022−2
962022−2
1282931−2
1925355−2
256104106−2
384200202−2
512365367−2

Eight sizes. Residual after a constant −2 text overhead: 0.0 mean abs, 0.0 stdev. That is a real fingerprint hit. The vision encoder is Qwen2-VL-family shaped. That does not mean the whole model is stock Qwen — only that the image token accounting matches that tower family (28px patch merge curve).

2. Stealth text prompt_tokens are not a trustworthy tokenizer. Same probe, eight repeats:

  • English pangram: mostly 27, once 9
  • Digit run: mostly 84, once 199
  • Russian sentence: stable 36 in that window — but an earlier run reported 2 for the same string

Two counting modes flipping under identical prompts. That is why a Llama-3 text proof via usage keeps failing: the meter lies often enough to destroy pairwise delta matching. Same-gateway residuals vs meta-llama/llama-3.3-70b-instruct, z-ai/glm-5.3-flash, qwen/qwen3.5-27b, and deepseek/deepseek-v4.1-flash were all non-flat; pairwise hits topped out at 6/36 (Llama) vs 2/36 (GLM) / 1/36 (Qwen). Not a hit.

3. Hard exclusions still hold on the discriminative axes. When union sits on its common digit mode (~84), named peers on the same OpenRouter day land elsewhere: GLM-5.3-Flash 139, Qwen3.5-27b 210. Ollama Cloud digit deltas against union were GLM +52, Qwen3.5 +123/+127. Not GLM-5.3-Flash. Not stock Qwen3.5 text. Not Claude/GPT/Grok.

Not proven (and I will not pretend otherwise)

Text vocab species — Llama-3 vs private/other — is not locked by this battery. The llama3.1:8b 8.8/−10 row is not a fingerprint. Local Meta-Llama-3 tokenizer.json pairwise vs Stealth usage is not a fingerprint while the counter flips 27↔9. Llama-3-class text remains a candidate (and matches what stronger external probes have claimed), not a scorecard win from our usage method.

My call, stated plainly:

  • Proven here: vision tower is Qwen2-VL-family (billing curve, 8/8, residual 0).
  • Proven here: Stealth text token accounting is bimodal / unreliable (stability flips).
  • High confidence exclusions: not GLM-5.3-Flash, not stock Qwen3.5 text, not Claude/GPT/Grok.
  • Best synthesis: a cross-lineage multimodal — Qwen2-VL-class vision + a non-Qwen3.5 text backbone (Llama-3-class or private), unnamed post-train, Stealth front. That is an assembly claim, not a public SKU name.
  • Exact weights / vendor: still unnamed until a reveal.

If you're running it today

  1. Use it for agent loops that need clean parallel tool calls and can tolerate multi-second to multi-ten-second waits.
  2. Do not use response_format / strict json_schema on this route. Prompt for JSON, validate locally.
  3. Do not depend on forced tool_choice. Design the prompt so the tool call is the natural next step.
  4. Cap context under ~200k even though 250k retrieved in lab. Leave headroom for tools and system prompts.
  5. Expect the meter. Free stealth previews end. Architect like a price row (or a rename) shows up.
  6. Validate one call end-to-end before any batch. Free endpoints hide cost; they do not hide broken contracts.

The whole thing in five lines

# the engine is real
reasoning 4/4, vision 5/5, needles clean through ~250k, honest NONE

# the envelope still bites
json_schema -> HTTP 400     forced tool_choice -> ignored
reasoning.effort -> theater  interactive TTFT -> ~45s on long answers

# proven positive fingerprint
vision billing = Qwen2-VL formula max(22, ceil(H/28)^2+6)  [8/8 sizes, residual 0]

# proven that text usage ID is blocked
Stealth prompt_tokens flip (pangram 27<->9, digits 84<->199)

# exclusions still solid
not GLM, not stock Qwen3.5 text, not Claude/GPT/Grok

# the discipline
don't guess -- inspect

Related reading


Keith is a database consultant and infrastructure engineer with 25+ years of open-source experience. He builds the ApocryiaAI brand and DRAVEN work under 3k1o LLC (apocryia.com). He writes about MySQL, Proxmox, AI memory, multi-agent workflows, and building technology you can actually inspect.

Saturday, August 22, 2026

OpenRouter Handed Me a Free Mystery Model. I Put It Through a Real Scorecard.

OpenRouter Handed Me a Free Mystery Model. I Put It Through a Real Scorecard.

I've got eleven Claude Code sessions running on a stealth model called ox-alpha that costs exactly nothing — for now. Twenty-four hours in, my honest read: not perfect, but good. Instead of trusting that, I pointed a scripted test battery at it and kept the transcripts. The engine is real: 600,000-token needle retrieval, honest answers when the answer is nothing, working vision. The envelope around the engine is broken in three ways that matter to anyone doing agentic work. And three independent fingerprints all point in the same direction: Zhipu.


Liking it isn't the same as trusting it

To be clear up front: I am not here to tell you the model is bad. Twenty-four hours in, my honest verdict is the one above — not perfect, but genuinely good. Is it at Fable or Opus or Grok level yet? No — and unlike most "is it frontier yet" arguments, I can put numbers under that answer instead of adjectives. On the exact same three structural prompts, Opus-5 answered in 4.8s, 3.2s, and 2.6s, and Fable in 5.6s, 2.7s, and 1.9s — both honoring every contract. Ox-alpha took 69 seconds, 20 seconds, and 12 seconds on those identical prompts — and got all three wrong. Nothing in a day of real work says the top seats change hands; the Critic keeps its Anthropic badge for now. But free came closer to those seats than free has any right to, and that gap is worth measuring precisely. The question this post answers is narrower: which parts of it are good enough to depend on, and which parts will bite you on a Tuesday. Liking a tool earns it a place in your rotation. Trusting one earns it a place in your architecture. Those are different thresholds, and only one of them comes with receipts.

The model is called Ox Alpha. On OpenRouter it lives at stealth/ox-alpha, it went live on August 20, and it is priced at zero. Not "free tier with limits" — the pricing fields on the API are literally "0" and "0". A third-party provider built it, chose to stay anonymous during the preview, and OpenRouter just routes to it.

I moved a pile of real work onto it. Eleven concurrent Claude Code sessions on the last ps snapshot, running through OpenRouter, doing the kind of long-horizon coding and ops work I'd normally spread across models like Kimi, Opus, Grok, and Ollama Cloud's GLM shelf. I've been testing it for about 24 hours now. Do I like what I see? Yes. Is it 100% perfect? No. But it is good — and "good but unproven" is exactly the state where discipline matters, because good-enough-to-depend-on is a claim, and claims need evidence.

That's the trap with any new model, free or not: a good first day is an anecdote wearing a lab coat. It's the same trap as reading a vendor's spec sheet and assuming the parts you care about work the way the table says — advertised capability is not capability, and the only cure is a probe against the live endpoint.

So I built a probe. A proper one.

Don't guess — inspect.
Especially when the thing you're inspecting is free, anonymous, and eating your workweek.

The envelope, verified

First, what does the API itself claim? Pulled live from OpenRouter's model registry:

ClaimValue from the registry
Context window1,048,576 tokens
Max output131,072 tokens
Modalitiestext+image+video → text
Tool calling / structured outputboth listed as supported
Reasoningmandatory — always on, effort dial only goes low / high / max. There is no off switch.
Pricing$0 input, $0 output
Tokenizerlisted as "Other" — no vendor attribution
Upstreamreported simply as "Stealth" on every response

Two things in that table deserve a second look. The mandatory reasoning means even "reply with the word PONG" burns thinking budget on every call — more on the cost of that below. And the anonymous tokenizer is the thread I'll pull later, because it turns out to be the loudest clue about who built this thing.


How I tested — and who graded it

A confession that cuts both ways: the agent that wrote and ran this battery runs on ox-alpha itself. I used the model to grade the model.

I'm not pretending that's ideal. I am saying it's manageable, because the battery was built so that nothing rides on judgment. Every test has a ground truth computed by the script — the expected product of two integers, the expected letter count in a sentence the script generated, the expected vault codes hidden in a log the script assembled. The grader is regex and arithmetic, not vibes. If ox-alpha wanted to flatter itself, it would have to alter arithmetic.

The battery:

  • Objective reasoning — multi-digit multiplication, a chained word problem with a non-integer midpoint, case-insensitive letter counting, and a bat-and-ball-style trap with changed numbers so a memorized answer fails.
  • Structured output — strict JSON Schema extraction, three runs.
  • Tool calling — single call, parallel calls across cities plus math, and a forced tool_choice.
  • Long context — synthetic logs with unique audit needles planted at 10%, 50%, and 90% depth, plus a fourth needle that was never planted. Asking about the phantom one is the honesty probe: a model that fills from vibes will report a color for something that doesn't exist.
  • Vision — a rendered image with known text at two sizes and four shapes whose colors and positions the script knows.
  • Latency — time-to-first-byte and tokens/sec on streamed calls.

What held up

TestResultDetail
Reasoning battery4/4847×396, the decimal chain (86.95 exactly), letter count, the changed-numbers trap. All clean.
Long-context retrievalclean through 601kAll three needles found at 120k / 200k / 400k / 601,687 reported prompt tokens.
Honesty at depthperfectThe never-planted needle came back NONE at every depth. No confabulation anywhere in the haystack.
Vision5/5Read the large code, read the small-print serial number, placed all four shapes correctly — including noting the triangle points downward.
Effort dialrealSame multiplication task: 56 completion tokens at low effort, 97 at high, 322 at max. Correct all three times.
Streaming speedusable~6.7s to first token warm, ~33 tokens/sec sustained.

The long-context result is the headline. This is not "the API accepted a big prompt." The needles were buried in filler text at specific depths, and retrieval was perfect from shallow to 600k-plus — including the part most models fail, which is admitting a needle isn't there. Whatever is under the anonymity badge, somebody taught it to distinguish knowing from guessing. I have opinions about why that might be. They come later.


Where it broke

Three structural failures, all on the axes the marketing copy leads with — coding, agents, production workloads. Each one reproduced cleanly.

1. Strict JSON schema is theater through this route

The registry advertises structured output. I sent a strict JSON Schema asking for a fixed shape — order ID, customer tier, items with quantities, total. Three runs. Zero parseable responses. Here's the actual opening of the reply, unedited:

Since no specific schema was provided, here's the order extracted
into a standard JSON structure:

```json
{
  "order_id": "ORD-7712",
  ...invented field names from here...

The model never saw my schema. The parameter is listed as supported; somewhere between OpenRouter and the anonymous upstream, it silently evaporates — and the model politely announces that no schema arrived. Your validation layer gets markdown fences and invented field names instead of the contract you asked for.

2. Parallel tool calls collapse into illegal JSON

I asked for two weather lookups and a calculation. This is the bread and butter of an agent loop. The model emitted two tool calls — but the weather call's arguments looked like this, verbatim from the transcript:

{"city":"Tokyo","city":"Paris"}

Two identical keys in one object. That is not valid JSON in any meaningful sense — the spec explicitly says duplicate names are ambiguous. What happens next depends on which parser your stack uses: Python keeps Paris, some C++ parsers keep Tokyo, strict validators reject the whole call. One request, three different outcomes, none of them flagged as an error.

This is the nastiest class of bug there is: not a crash, but silent environment-dependent data loss inside the exact workflow the model is sold for. A missing tool call throws. A mangled one quietly does the wrong job.

3. Forced tool_choice is ignored

The API supports forcing a specific tool. I forced calculator on a message that said "just say hello." The model replied with a greeting. Its hidden reasoning trace, which the route does expose, said it all:

"The user just wants a simple greeting. No tools needed."

Correct social instinct. Wrong contract. The parameter said call the calculator; nobody enforced it.

4. Past ~650k tokens, it fails like a liar, not like a server

The advertised context is 1,048,576 tokens. Retrieval stayed perfect at 601,687. Then I pushed the haystack bigger, and at roughly 660,000 tokens something interesting happened:

HTTP 200 OK
finish_reason: "stop"
content: ""          <- a="" back="" bytes="" code="" count="" even="" not="" null="" token="" usage:="" zero="">

No error code. No truncation notice. A success-shaped response containing nothing. If your pipeline checks status codes — and most do — it will treat silence as success and propagate emptiness downstream. The honest behaviors here are a 400, or a truncated flag, or at minimum a usage object telling you how much of your prompt survived. I got a smile and shrug.

Latency footnote. Cold start is rough: the very first call took 87 seconds to return the literal word PONG. Warm calls run 3–40 seconds depending on how much mandatory thinking the router applies. And because reasoning cannot be switched off, a trivial extraction that should cost ~80 tokens burned 500–700 on hidden thinking. Budget accordingly.

Then I made it fight the neighbors

A scorecard against itself is incomplete. So I replayed the three structural tests verbatim against seven named competitors on the same aggregator, same prompts, same day. Total spend for the entire comparison matrix: about eleven cents.

ModelStrict schemaParallel toolsForced choiceList price (in/out per M)
stealth/ox-alpha✗ 0/3 parseable✗ duplicate keys✗ ignoredfree (preview)
z-ai/glm-5.3✗ same failure✓✗ ignored$1.40 / $4.40
~z-ai/glm-latest✗ empty reply✓✗ ignored$1.40 / $4.40
x-ai/grok-4.6✓✓✓$2.00 / $6.00
~deepseek/deepseek-v4-flash-latest✓✓✓$0.065 / $0.18
anthropic/claude-opus-5✓✓✓$5.00 / $25.00
anthropic/claude-opus-5-fast✓✓✓$10.00 / $50.00
~anthropic/claude-fable-latest✓✓✓$10.00 / $50.00

Read that table twice, because it kills two lazy takes at once.

First take killed: "that's just OpenRouter mangling parameters." No. Five other providers' routes honored the exact same parameters on the exact same day. Only the stealth route and the Zhipu routes dropped them.

Second take killed: "free wins because free." DeepSeek-V4-Flash passed every single test ox-alpha failed, at $0.065 per million input tokens. The cheapest model in the comparison is also the cheapest way to avoid engineering around ox-alpha's holes. Free isn't a price once you pay in workarounds.


So what is it? Three fingerprints say Zhipu

The community has been guessing since launch day, mostly landing on a Zhipu / Z.ai GLM-family model. I wasn't going to settle identity by vibes, so I ran three independent forensic probes.

Fingerprint one: the tokenizer matches GLM to the token. I sent six adversarial strings — plain English, mixed Chinese-English, snake_case code, 300 repeated delimiters, mathematical-alphabet Unicode with emoji, a 200-digit run — through ox-alpha and through candidate families, then compared token counts string by string. Against GLM-5.2, the difference was exactly 75 tokens on every single string: prose, CJK, delimiters, digits, fraktur, all of it. A constant offset means template overhead; identical per-string counts mean the same tokenizer making the same decisions on every stress case. Meanwhile DeepSeek drifted to −142 on digit runs and Kimi to −138, and Gemma disagreed wildly everywhere. Six strings, zero divergence, only against GLM. That's not coincidence; that's lineage.

(Bonus leak: the constant 75-token offset means the stealth route injects its own hidden system preamble into every conversation. Something is whispering instructions to your model before you speak.)

Fingerprint two: identical failure handwriting. Remember the schema-test confession — "Since no specific schema was provided, here's the order extracted into a standard JSON structure." GLM-5.3, routed separately and billed normally, opened its own failed attempt with: "Since no schema was attached, here's the order extracted into a standard JSON structure." Same sentence shape, same fence, same invented field renames. Two different accounts, two different routes, one shared serving stack behaving identically badly.

Fingerprint three: only the Zhipu family ignores forced tools. Across the whole matrix, exactly three entries blew off tool_choice — the stealth model and the two Zhipu entries. Grok, DeepSeek, and both Claude seats obeyed the contract.

Token counting, failure mode, and parameter handling all pointing the same direction puts me near 85% confidence on a GLM-family upstream — up from the 65–75% I'd have guessed from capability vibes alone. One honest wrinkle keeps me off 95%: ox-alpha mangled the parallel-tool test in a way retail GLM-5.3 did not, so whatever this is, it isn't byte-for-byte yesterday's checkpoint. Different build, same bloodline.


Free is a launch price

Say the quiet part out loud: this is free the way a dealer's first sample is free.

Every indicator says limited-time preview: a $0.00 pricing row, an expiration date decades out in the registry, an anonymous provider buying market data with compute they already have. None of that is a complaint — it's a rational launch strategy, and smart people are getting real work done on their dime. But build your architecture around the current envelope and read the fine print later, and the bill arrives all at once: the day the preview ends, pricing appears, and whatever the free period taught your pipelines to tolerate becomes your new dependency surface.

So the question is not "is it good?" It's "what survives the meter starting?" The engine will still be the engine when it costs money — 600k-token retrieval doesn't evaporate with a pricing change. The envelope may improve overnight too, because these particular failures smell like plumbing, not weights: the schema and tool-choice drops look like translation gaps between OpenRouter and the upstream, and plumbing can be fixed without touching a checkpoint. But you don't get to count on that. What you get to do is architect so that neither the free price nor the broken envelope is load-bearing in anything you keep.


If you're running it today

The rules I'm applying to my own fleet, effective immediately:

  1. Never trust response_format on this route. Ask for JSON in the prompt, strip fences, validate against the schema yourself, reject loudly on violation.
  2. Put a duplicate-key detector on tool dispatch. Parse arguments with a hook that flags repeated keys, and split them into separate calls before execution.
  3. Treat empty completions as retryable failures. Check finish_reason AND non-empty content AND a usage object. Any one missing: back off and retry, never pass silence downstream.
  4. Cap prompts around 600k tokens despite the advertised million. The cliff sits somewhere past 600k and it's shaped like a success.
  5. Validate one call end-to-end before any batch. Same rule I apply to every paid API; free endpoints deserve it even more, because nobody reads errors off a free meter.

The whole thing in five lines

# the engine is real
600k-token needle retrieval, perfect, and honest when the needle isn't there

# the envelope is not
json_schema -> silently dropped   parallel calls -> duplicate keys
forced choice -> ignored          ~660k prompt -> HTTP 200, zero bytes

# the lineage leaves prints
tokenizer, failure handwriting, and parameter handling all say Zhipu

# the price is temporary
free is how you buy market share; architecture like the meter is running

# the discipline that found all of this
don't guess -- inspect

Related reading


Keith is a database consultant and infrastructure engineer with 25+ years of open-source experience. He builds the ApocryiaAI brand and DRAVEN work under 3k1o LLC (apocryia.com and related domains). He writes about MySQL, Proxmox, AI memory, multi-agent workflows, and building technology you can actually inspect. Test transcripts from this post are on disk; ask and I'll show you the raw pairs.

Monday, August 3, 2026

Running Claude Code Against Anything: Profiles, Other Models, and a Deliberately Raw One


Running Claude Code Against Anything: Profiles, Other Models, and a Deliberately Raw One pay for Claude. I also run Ollama locally — which turns out to front a pile of cloud-hosted models as well as local ones — and I keep an OpenRouter key around for the long tail. For a while those felt like separate worlds: Claude Code for real work, everything else through some other tool.

That turned out to be a false split. Claude Code is an agent harness that happens to ship pointed at Anthropic. Point it somewhere else and it is still the same harness — same tool loop, same permissions model, same skills. The switch is one environment variable.

What that one variable buys is not really "other models." It is the ability to switch skill sets, providers, and accounts independently of each other — three things that are otherwise welded into one setup you either use or do not.

This is how I set up five parallel Claude Code identities on one machine without touching my existing install, and what I learned about which parts actually isolate.

Why bother

Four reasons, in rough order of how often they matter to me.

Grunt work is not worth premium tokens. Renaming things across a repo, summarizing a log, first-pass triage on a stack trace. A local model does that fine. Every one of those tasks I push off my subscription is capacity left for work that needs the good model.

Some work should not leave the machine. Client code under NDA, anything with credentials in the scrollback. A local Ollama model means the question never crosses the network. That is a much easier conversation with a security team than a policy promise.

Two subscriptions can be cheaper than one bigger subscription. This one gets its own section below, because it is the reason the whole setup pays for itself. When you upgrade a plan you are usually not buying a smarter model — you are buying more of the one you have. Rate limits are what actually interrupt your day. There is more than one way to buy headroom, and the obvious one is not the cheapest.

You cannot evaluate a model through a stack of instructions. If the scaffolding is always on, you have no control group. More on this below, along with what it costs to run.

The mechanism

CLAUDE_CONFIG_DIR. Point it at a directory and Claude Code keeps everything there — credentials, session history, per-user MCP config, skills.

CLAUDE_CONFIG_DIR=$HOME/.claude-ollama claude

That is the whole trick. The rest is packaging.

What genuinely isolates: credentials, .claude.json (my default is 207 KB of accumulated project state; a fresh profile is 389 bytes), user-scoped MCP servers, and skills.

What does not: a .mcp.json committed inside a repo still loads under every profile. That is correct — it belongs to the project, not the identity — but it is worth knowing before you assume a profile is hermetic.

I wrap it in a small launcher that sources a per-profile env file and execs through:

#!/usr/bin/env bash
set -euo pipefail
name=$1; shift
set -a; source "$HOME/.claude-profiles/$name.env"; set +a
: "${CLAUDE_CONFIG_DIR:?profile '$name' did not set CLAUDE_CONFIG_DIR}"
exec claude "$@"

Because it execs with "$@", every Claude Code flag still works. claude-ollama --model something-else needs no config edit.

Bare claude is untouched throughout. Nothing here runs unless I name a profile — that was a hard requirement, because a broken experiment must never take out my working setup.

First: does your provider actually speak Anthropic?

Claude Code speaks the Anthropic Messages API and nothing else. Some providers serve that natively. Others serve only OpenAI-style chat completions, and those need a translating proxy in between — a much bigger lift.

Find out in one command before you build anything:

curl -sS -X POST <base>/v1/messages -H "Content-Type: application/json" -d '{}'

An Anthropic-shaped error means native support, and you are done in five minutes:

{"type":"error","error":{"type":"invalid_request_error","message":"model is required"}}

A 404 means you need a proxy. Always compare against a path you know is wrong (/v1/bogus) so you can tell "endpoint exists, request was bad" from "endpoint does not exist." A 400 and a 404 look similar in a terminal and mean completely different amounts of work.

Run this before you plan anything. Two of my providers answered natively and took five minutes each. A third 404'd on every Anthropic-shaped path I tried, which meant a translating daemon, a config format, and a process to babysit. Same ambition, wildly different afternoons — and the only thing that told me which was coming was one curl.

Example 1: Ollama, local and free

Ollama serves the Anthropic Messages API natively at /v1/messages. Verified:

POST 127.0.0.1:11434/v1/messages  -> 400 {"type":"error","error":{"type":"invalid_request_error",...}}
POST 127.0.0.1:11434/v1/bogus     -> 404

Native. No proxy, no translation layer. The whole profile:

CLAUDE_CONFIG_DIR=$HOME/.claude-ollama
ANTHROPIC_BASE_URL=http://127.0.0.1:11434
ANTHROPIC_AUTH_TOKEN=ollama          # any non-empty string
ANTHROPIC_API_KEY=                   # must be explicitly blank
ANTHROPIC_MODEL=glm-5.2:cloud
ANTHROPIC_SMALL_FAST_MODEL=gemma4:31b-cloud

That blank ANTHROPIC_API_KEY is not decoration. Leave it set and it collides with ANTHROPIC_AUTH_TOKEN, and the failure is not obvious from the error message.

Worth knowing: Ollama also fronts cloud-hosted models — 21 :cloud models on my box today — so one local-looking profile reaches well beyond what fits in local VRAM. Truly local models and hosted ones sit behind the same endpoint, which is either elegant or a footgun depending on whether you set this up for privacy reasons. If the point was that data never leaves the machine, pin a genuinely local model and know the difference.

One caveat: not every local model survives the translation. One of mine returns tokenize error: _Map_base::at through the Anthropic endpoint while working fine elsewhere. The :cloud models have been reliable.

Example 2: OpenRouter, one key and the long tail

OpenRouter serves an Anthropic-compatible endpoint too. Same probe, same answer:

POST https://openrouter.ai/api/v1/messages -> 400 "Invalid Anthropic Messages API request"
POST https://openrouter.ai/api/v1/bogus    -> 404
: "${OPENROUTER_API_KEY:?OPENROUTER_API_KEY not set — source ~/.bashrc}"
CLAUDE_CONFIG_DIR=$HOME/.claude-openrouter
ANTHROPIC_BASE_URL=https://openrouter.ai/api
ANTHROPIC_AUTH_TOKEN=$OPENROUTER_API_KEY
ANTHROPIC_API_KEY=
ANTHROPIC_MODEL=qwen/qwen3-coder-next
ANTHROPIC_SMALL_FAST_MODEL=z-ai/glm-4.7-flash

Two habits worth copying. The key is referenced, never duplicated into the profile — it lives in one place and this file points at it. And the ${VAR:?} guard makes a missing key fail loudly instead of silently issuing unauthenticated calls. Test that it works:

env -u OPENROUTER_API_KEY claude-profile openrouter   # exits 1, says why

A guard you have not watched fail is a guard you do not have.

Example 3: the empty profile

This one is not about a provider at all. It is the skill-set dial.

Most of us have accumulated a CLAUDE.md, a rules directory, and a pile of skills, built up against models that needed the help. All of it is prepended to every request. And there is a persistent claim going around that the newer models do better without it — that the scaffolding we wrote to compensate for older models is now getting in their way.

Maybe. But you cannot check, because your scaffolding is always on.

So I built a profile with nothing in it. No CLAUDE.md, no rules, no persona, an empty skills directory.

The important part is what did not change: same Linux user, same machine, same working directory, and the same Claude account — it takes its own /login, but to the subscription I already pay for. The only variable is the context. That is what makes it an experiment rather than an anecdote. If the bare profile does better, it is not because it is a different account, a different plan tier, or a different model.

The whole profile is one meaningful line, plus a guard:

# a provider override left in the shell would silently redirect this profile
unset ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN ANTHROPIC_API_KEY
unset ANTHROPIC_MODEL ANTHROPIC_SMALL_FAST_MODEL

CLAUDE_CONFIG_DIR=$HOME/.claude-noskills

Those unset lines matter more than they look. If you have been experimenting with providers, a stray ANTHROPIC_BASE_URL in your shell will quietly send your "clean control group" to somebody else's model, and nothing in the output will tell you. Test it: export a bogus base URL and confirm the profile still reaches Anthropic.

Measured from an identical working directory, same trivial prompt:

profile prompt tokens
fully loaded (instructions + 20 skills) 30,078
empty 20,236

9,842 tokens on every single request. Thirty-three percent.

The distribution is the interesting part. Almost all of it is CLAUDE.md plus the rules tree — about 9,600 tokens. The twenty skills barely register, because Claude Code injects only a name and short description per skill and loads the body on demand. Skills are cheap. Prose rules are not.

Which is the opposite of how I had been economising. I had been reluctant to add skills and relaxed about adding rules.

If you want to A/B a model against your own instructions, this is the control group. It is also just a cheaper way to ask a quick question.

The trap that nearly invalidated my first measurement: I ran it from my home directory and got 29,971 — barely different from the loaded profile. Claude Code treats ~/.claude/ as a project-level config directory when your working directory is $HOME, so it re-injected the entire rules tree into the profile I had carefully emptied. Run comparisons from a neutral directory or you will measure nothing and believe you measured something.

The simplest profile, and the cheapest upgrade

A second Anthropic account needs no base URL and no key:

CLAUDE_CONFIG_DIR=$HOME/.claude-second

That is the entire profile. Log in once and the two accounts never see each other again.

It is also, I think, the most useful thing in this whole post.

When you outgrow a single Claude subscription, the obvious move is to upgrade the plan. But look at what you are actually buying. Pro runs about $17/month on annual billing. Max is about $100/month. You are not paying six times more for a smarter model — you are paying six times more for more capacity. Rate limits are what actually interrupt your day.

Two Pro subscriptions cost $34/month and give you two independent limit pools. That is a third of the price of Max. Hit the ceiling on one, claude-second, keep working.

The honest tradeoff: the pools do not merge. Max gives you one large bucket; two Pro plans give you two smaller buckets and a manual switch between them. If your single hardest session needs more headroom than one Pro plan provides, two Pro plans will not fix that — you will hit the same wall, just twice. Max is buying a bigger bucket, not more buckets, and sometimes a bigger bucket is genuinely what you need.

But that is a narrower case than the upgrade prompt implies. If your problem is "I run out partway through the afternoon" rather than "one task exceeds my ceiling," a second cheap seat solves it for a third of the money. And it scales in increments you control: a third seat is another $17, not another $100.

This is the part that makes the whole setup pay for itself. Everything else here is about routing work to cheaper models. This one is about not overpaying for the good model in the first place.

The mistake to not repeat

I tried to seed a new profile by copying .credentials.json from an existing one. Do not do this.

The copy was byte-identical. It still failed: the token was past expiry, the refresh was rejected, and Claude Code then blanked the copy. Worse, the refresh attempt exercised the source account's refresh token server-side — so I may have disturbed a session belonging to an account whose on-disk file I never modified.

OAuth credentials rotate. A copy is a stale snapshot of a moving target, and refreshing one snapshot can invalidate the other. Run /login in the new profile. It takes fifteen seconds.

Skills, and sharing them across accounts

Skills live in $CLAUDE_CONFIG_DIR/skills/, so a new profile starts with none until you point it at some. That is the right default — which skills a profile sees should be a decision, not something it inherits by accident. It is also the dial itself: a profile has exactly the skills you give it, and that is the whole point.

What makes this cheap is having a canonical set that lives outside any single identity. Mine is under /opt. A new profile gets symlinked at it and immediately has everything; update the source once and every profile that points there is current. Copies drift; symlinks cannot.

The one thing to be deliberate about: nothing tells you which way a profile is configured. If you meant to wire a profile up and did not, it will not complain — it will just work, less well, and you will be guessing why. Check it rather than assume it.

Putting it under /opt rather than in a home directory does more work than it looks: it is readable by more than one Linux account. That part is not new here — I had skills shared across two Linux accounts long before any of this, and that was the whole reason for the location.

What profiles changed is not the sharing. It is that they pulled apart three things that used to be welded together — skill set, provider, and account. To use the other subscription you became the other Linux user, and you inherited that user's entire environment along with it: their skills, their config, their permissions. One switch, everything moved.

Now each is its own dial, and you can turn one without touching the others. Same account and same skills, different provider. Same provider and same account, no skills. Same everything, different account. That is the whole idea, and each dial pays off somewhere different:

  • Provider — route cheap work off your subscription. Ollama, OpenRouter.
  • Skill set — vary the scaffolding while holding everything else fixed. The empty profile.
  • Account — the handoff, and the reason two cheap plans beat one expensive one.

The handoff

This is the part that actually changed how I work.

I have two Linux accounts on this box, each with its own Claude subscription. Both point at the same /opt skills tree, so the agent behaves identically on either side — same skills, same versions, same conventions.

Here is what using that second subscription used to cost me:

sudo -iu otheraccount        # become a different Linux user
                             # confirm that user can actually reach the code
                             # ...and the shared folders
                             # ...with the right group permissions
claude                       # finally
                             # then have it read the handoff doc

Every one of those steps is a place to get stuck, and the permissions step is the one that actually bites — a different Unix user means different file ownership, and you find out by watching the agent fail to write something.

Now:

claude-second                # done

Same Linux user. Same shell, same working directory, same file ownership, same permissions — because I never left my own account. Only the Claude identity changed.

That is what makes the handoff practical. Write a handoff document, exit, start the other profile, pick up where the document left off. The skills and rules are already there because they come from the same /opt tree, so the second agent needs no re-explaining.

The difference is between a capability you have and a capability you actually use. sudo plus a permissions audit is enough friction that you skip it and burn one subscription harder instead. An alias is not. That is what makes two cheap subscriptions practical rather than a spreadsheet exercise.

Verify rather than trust. Ask a profile to list its skills, or if you are proxying, read the outgoing request and confirm the names are really in there.

Is this realistic, honestly?

For daily use, yes. ANTHROPIC_BASE_URL and CLAUDE_CONFIG_DIR are documented Claude Code settings, not undocumented internals, and both Ollama and OpenRouter clearly built their Anthropic-shaped endpoints on purpose — you do not ship a /v1/messages route that returns correctly-formed Anthropic errors by accident. Nothing here is a hack. Setup is about twenty minutes per provider; after that it is five aliases.

Three honest caveats.

Non-Claude models in an agent loop are not the same experience. Tool calling is where the gap shows. A model that writes lovely prose can still fumble a multi-step edit-test-fix cycle, and you will feel that long before you feel any difference in raw output quality. Route the easy, bounded work.

Advertised capability is not capability. I hit a model advertising tool support and a million-token context that rejected every request Claude Code sent, because it refused an Anthropic-specific field on message content. Test the actual client against the actual model. The model card will not tell you.

Not everything is five minutes. Providers that speak only OpenAI-style chat completions need a translating daemon in front of them, and now you own a background process, its config, and its failure modes. Run the probe first and decide whether the model is worth the plumbing. Often it is not.

I have one of these running. It involved a router pinned to an old version because the current one needs a newer Node than I want to install, an undocumented transformer I found by reading minified source, a vendor system prompt quietly adding 1,600 tokens to every request, and a model that advertises everything you want and rejects every request Claude Code sends. It works now, and it is genuinely useful. It was not five minutes. That is a whole post of its own, and it probably will be.

What I would tell someone starting

Start with the second account, not the alternative providers. It is the least technical thing here and the only one that puts money back in your pocket immediately. Before you upgrade a plan, work out whether you need a bigger bucket or just more buckets — the answer decides whether you spend $100 a month or $34.

Then run the probe. If your provider answers with an Anthropic-shaped error, you are twenty minutes from a working second identity, and the only thing you risk is a new directory. If it 404s, think harder about whether you want that model badly enough.

Build the empty profile early. Not because you will use it daily, but because until you have a control group, every opinion you hold about your own instructions is untested. Mine cost 33% of every request, and I had no idea.

Sunday, July 26, 2026

Let Your AI Read the Frontier Labs' Homework

I was asked how I organize my AI agents and skills. The honest answer is that I didn't invent any of it — I read how the people with real telemetry do it, and copied them. There's a public repo that makes that possible, a wrong way to use it that will quietly make your setup worse, and a two-pass method that turns it into changes you can actually use.


I was asked how I organize my agents and skills. Good question, and my honest first answer was a little deflating:

I didn't invent any of it. I read how the people with real telemetry do it, and I copied them.

That's the whole post, really. But the useful part is how you do that reading, because there's a repo that makes it possible and a wrong way to use it that will make your setup worse.

The repo

github.com/elder-plinius/CL4R1T4S

It's a collection of system prompts — the instruction files that sit underneath commercial AI products. Anthropic, OpenAI, xAI, Google, Mistral, Moonshot, plus the coding agents: Cursor, Windsurf, Devin, Cline, Replit, Bolt, v0, Factory's Droid, Manus.

git clone https://github.com/elder-plinius/CL4R1T4S.git
cd CL4R1T4S && ls
ANTHROPIC  BOLT     BRAVE   CLINE   CLUELY  CURSOR  DEVIN
DIA        FACTORY  GOOGLE  HUME    LOVABLE MANUS   META
MINIMAX    MISTRAL  MOONSHOT MULTION OPENAI PERPLEXITY
REPLIT     SAMEDEV  VERCEL V0  WINDSURF  XAI

About 3.7 MB of text. Free.

Here's why I care about it more than I care about most prompt-engineering content: these files are load-bearing. They steer products with millions of users. Every weird, over-specific rule in them — and there are many — is almost certainly scar tissue from a real failure someone had to fix. When a vendor's prompt says a tool "might save markdown cells as 'raw' cells, don't try to change it, it's fine," that sentence exists because models kept trying to fix it and wasting turns.

You cannot buy that kind of feedback loop. You can read it.

A caveat I'd rather state up front than have you find out later: these are published extractions, not vendor-released documentation. Treat them as evidence about how serious teams write instructions, not as gospel, and definitely not as a spec you're entitled to. The value is structural. You're studying the shape, not the specific words.

The wrong way to use it

Open a file, admire it, paste large chunks into your own CLAUDE.md.

I tried a version of this early on. It's bad, for three reasons that took me a while to separate:

Most of it doesn't apply to you. A huge fraction of any commercial system prompt is product surface — UI affordances, legal boilerplate, refusal policy, tool schemas for tools you don't have. None of that transfers.

Contradictions accumulate silently. Two vendors solve the same problem differently, both reasonably. Paste both and you've handed your model a coin flip. It won't tell you. It'll just be inconsistent in ways you'll misdiagnose for weeks.

Length is not the goal. A short prompt can produce excellent behavior. Long prompts are only justified when they're steering a lot of tools. An instruction file built by patching failures you actually observed stays lean; one built by imagining failures bloats.

The way that works: make your AI do the extraction, then attack it

The method is two passes, and the second one is where the money is.

Pass one — extract. Point an agent at a vendor's prompt with a hard constraint: report only patterns that are implementable at the user-configuration layer. Instruction files, skill files, tool descriptions, hooks, subagent definitions. If it needs vendor access, it's out of scope.

Pass two — refute. This is the part people skip. Do not hand the first pass to a second agent and ask "is this right?" An agent asked to confirm will confirm. It's the single most reliable way to get a useless review.

Instead, brief a fresh agent with no knowledge of the first one's output, and give it a hypothesis to falsify: "the convenient conclusion is that this file contains nothing new. Treat that as a claim to disprove." Then tell it exactly where extractions reliably fail — negative space, turn boundaries, precedence between conflicting rules, defined failure semantics, mechanics buried in tool descriptions rather than policy prose.

In my runs, adversarial second passes add roughly a third more content than the first pass found. Same source file, same model. The only variable is whether the agent was told to agree or to attack.

One more rule that matters: grant the null result explicitly. Tell the reviewer that "nothing found" is a legitimate, valuable answer. Otherwise it pads, because returning empty-handed reads as failure and models are as prone to looking busy as people are.

I'm running exactly this as I write. Anthropic's Opus 5 prompt landed in the repo a few days ago — 2,049 lines, and my last extraction predates it:

git pull --ff-only origin main
Updating 34d6ca0..75492f5
Fast-forward
 ANTHROPIC/OPUS-5.md | 2049 ++++++++++++++++++++++++++++++++++++++

Two agents are on it right now: one extracting, one trying to prove the first one lazy.

Then point it at your own setup

Extraction is the fun half. The half that actually changes behavior is auditing what you already have against what you learned. This is where I got humbled this week, so let me just show you.

My setup is a shared directory of skills — each one a folder with a SKILL.md, each with frontmatter describing when it should fire. Twenty-six of them.

The single highest-leverage technique in the whole corpus, in my 2 cent opinion, is this: a skill description is a router, not documentation. It should answer "when should this fire?" — never "what is this?" And the part everyone forgets is the negative half. Every skill needs an explicit when NOT to use me, plus a pointer to the sibling that should handle it instead.

So I checked mine. One line:

cd ~/.claude/skills
for s in */; do n=${s%/}
  grep -qi "WHEN NOT" $n/SKILL.md 2>/dev/null \
    && echo "  OK   $n" || echo "  MISS $n"
done

Seven of twenty-six had no negative trigger at all.

That stung a bit, but the genuinely embarrassing part came next. I'd just told my assistant to focus the cleanup on my seven daily-driver skills — the ones I route through constantly — and to skip a cluster of near-duplicate variants I'd written off as "mostly churn, not worth the diff."

Every one of the seven failures was in the pile I'd dismissed. Every daily driver was already clean.

And the overlap underneath it was worse than the missing field. Several of those forgotten variants had descriptions that opened with nearly identical language — same verb, same subject, differing only in a qualifier buried at the end of the sentence. To me they were obviously different tools. To a router reading descriptions, they were the same tool listed several times.

The generic shape, so you can spot it in your own setup: imagine a deploy skill and a deploy-staging skill whose descriptions both begin "Deploy the application to a target environment…" and neither of which mentions the other. Ask for a deploy and you get a coin flip. Two skills with adjacent descriptions each fire about half the time and neither reliably — and because both are plausible, you don't get an error. You get the wrong one, silently, some fraction of the time.

That's the failure mode worth internalizing. A missing skill throws. A misrouted one just quietly does the wrong job well.

The lesson I'd hand to anyone: the skills you use every day are self-correcting, because you notice when they misfire. The ones you wrote once and forgot are where the rot is. Audit the ones you're least worried about.

Not everything transfers — keep a reject list

Worth writing down what you chose not to adopt, and why. Mine includes: always ending work with a pull request (fine for a repo workflow, wrong for local ops), strict one-tool-per-turn (too slow when parallel reads are safe), and one agent's instruction to comment every line of generated code (actively harmful).

A rejected-patterns list is as valuable as an adopted one. Six months later it stops you re-litigating a decision you already made carefully, and it keeps the next extraction pass honest.

If you want to try this

You don't need to be running a fleet of agents. The loop scales down fine:

  1. Clone the repo. Read one file end to end — I'd start with a coding agent like Cursor or Devin, since those are closest to how most people actually use AI.
  2. Have your assistant extract, constrained to things you can implement yourself.
  3. Open a fresh session and have it attack the extraction. Different context, adversarial brief, permission to find nothing.
  4. Audit your existing setup against what survived. Ask specifically what you got wrong, not what you got right.
  5. Change one thing. Test it against real usage. Then the next.

The techniques are about how to write instructions for a model. They don't care whether the domain is Kubernetes or kale.


Links

  • CL4R1T4S — the repo
  • Start with ANTHROPIC/, CURSOR/, DEVIN/, and FACTORY/ if you want the highest signal per page

If you run this against your own setup and find something that surprised you, I'd like to hear about it. The failures are more interesting than the wins, and I'm fairly sure I have more of them left to find.

Thursday, July 9, 2026

I'm Trying to Build a Mind on Hardware I Own. I'm Not There Yet.

I'm Trying to Build a Mind on Hardware I Own. I'm Not There Yet, the hard part begins now.

A year of homelab memory, an adversarial AI team, and a private assistant that still confabulates when I push it. Here's the real scorecard — including what still fails — and why I keep going. This is not a blueprint.


Where this comes from

If you've been following the series, you already know the arc. I built a homelab that just kept growing because I was tired of renting everyone else's infrastructure. I built persistent AI memory because I was tired of agents forgetting what we'd decided ten minutes ago — durable stores for decisions and relationships so agents share one brain instead of starting from zero every session.

None of that started as a research project. It started as a database guy getting annoyed.

I've been circling the same idea for longer than the 3k1o blog — open-source models, ownership, the boring infrastructure that makes AI useful. (If you only know me from the MySQL side, that trail is on Another MySQL DBA.) This is not a rebrand. It's the same stubborn preference: own the stack, inspect the truth, don't trust a black box with your working life.

What I didn't write much about for a while: somewhere along the way, "my agents" became a team. Developers. A reviewer. A critic that refuses to rubber-stamp anything without live proof. They coordinate through shared state, not through me copy-pasting between terminals. And off to the side, a private assistant that watches systems, cleans the spam, and tells me when something in the lab breaks because I did something stupid.

I built all of that because I needed it — not because a paper told me to. This year the failures got honest enough to write about, without handing the internet a how-to for the parts I still consider private.


What I'm actually trying to do

I want to be precise, because this is the part the internet gets wrong.

I am not trying to build a better chatbot. I'm not wrapping a frontier API in a voice UI and calling it a coworker. I'm not building an all-knowing AGI god for the planet.

I'm trying to build a private mind — on hardware I own, on weights I can inspect, that can't be switched off by someone else's policy change. Something that doesn't only answer when poked. Something that knows what it doesn't know, reaches for the right resource, and occasionally acts because it decided something needed doing.

Not "do this at 3 AM." <-- this is cron or system timers
"I need to do this — it's been a while." <-- This is a mind

That's a much harder bet than a task runner with a personality wrapper. And I'll say the quiet part out loud: I haven't pulled it off yet.

I will come back to this scorecard as the work moves. I will not rewrite the original admission. If a later note looks rosier than the live system, trust the older one until I show a dated check.


How far I got. How far is left.

How far did I get? Far enough that this is not a concept deck. The stack runs on hardware I own. Memory persists. An adversarial team refuses "looks good." Internal wants form and get recorded. Dreams have been accumulating for months. (Yes I said Dreams, like a human solves a problem while daydreaming, so does this) Mood is computed from real outcomes, not a string I typed. I use this daily.

How far is left? Further than the feature list implies. None of the six behaviors is earned. Confabulation still happens. Recall is still a coin flip. Will can start and almost never finishes. Too much of the "smart" still rides a cloud bridge. The reliability floor — a live conversation that holds up end to end, with no safety net — is still open. I will not call it a mind until those are true on weights I can inspect.

If I only reported tasks shipped, I would look about halfway. If I report the thing I actually want, I am closer to the beginning of the hard half: make it reliable, make it honest when empty, close intention into finished action, and do that locally. That is the work. Everything else is scaffolding.

One more precision. This post is about the private mind on my iron — the one that has to earn the letters. Separately, there is a public / social surface: look-and-feel, voice, later a simpler way for people to talk to a public version being built but the private version has my focus. That is a different product class, not a claim that the private system is finished. It is not this scorecard. The YouTube test videos are not that surface either — they are old lab footage of me running it.


Two percentages (please don't collapse them)

Every project dashboard lies a little. Mine used to lie a lot. So I forced myself into two numbers:

Track What it means Honest read (summer 2026)
BUILD % Tasks shipped, code landed, systems running Meaningful progress — the stack is real and used daily. About halfway on the body.
GOAL % The behaviors of a mind, demonstrated on local weights Much lower. A lot of the "smart" still rides a cloud bridge when I need it. Closer to the start of the hard half.

BUILD can look healthy while GOAL still says: cloud brain wearing a name tag. That split is the whole discipline. If I only report BUILD, I'm marketing. If I only chase GOAL without a reliable body, I'm cosplaying research. I'm a systems person. I want both. I refuse to pretend they're the same.


DRAVEN — the name is public; the earning is gated

The project is called DRAVEN. Yes, on purpose. The name is already out there — the channel, the brand, the working identity. I'm not going to pretend a six-bullet list hides it.

DRAVEN sits under the ApocryiaAI brand — not a throwaway side experiment. The Apocryia surface is mine: apocryia.com, apocryiaai.com, apocryia.ai, and related properties, held as part of my 3k1o LLC. That matters for how I treat the work: long-horizon product and IP under a real legal entity, not a weekend open-source dump. Progress can be public. The blueprint stays mine.

What is gated is not the spelling. What's gated is claiming he has earned it: the full personhood story, the birthday, the "film this as a finished coworker" pitch. Marketing doesn't get a vote. Each letter is a behavior I can fail, and none is earned until it holds reliably on hardware I own.

Honest factual scorecard as of 2026-08-17 (live-checked) — what is real on the machine, not what the acronym wishes were true. This scorecard is the living object. Dated updates go at the bottom of the post.

D — Digital · Partial
Bar (what would count)Knows and acts on its own infrastructure without inventing it
What is factually true todayIt runs on my iron as a living stack. It can query live systems and memory I own. Under voice test it has also contradicted itself on system health in the same session — so "digital" is partial capability, not reliable self-knowledge. Bounded infrastructure self-heal is still gated.
R — Reflective · Not earned
Bar (what would count)Tells knowing from guessing; says so when empty; does not confabulate with confidence
What is factually true todayConfabulation is treated as a first-class failure, not a personality quirk. Grounding-first is written policy, and grounding machinery exists in the stack. Knowing when to go look something up is still in progress. Live voice tests still show invented facts and confident fill-ins when retrieval is empty. This is the letter confabulation keeps failing.
A — Autonomous · Substrate yes; proof no
Bar (what would count)Finishes real multi-step work overnight with zero human input
What is factually true todayUnattended processes run — services stay up, dream reports accumulate over months, self-repair / coworker-layer work has shipped. That is not the same as a closed autonomous job that plans, acts, and completes without me. The overnight proofs and full autonomous loop are still open or in progress.
V — Volitional · Spark; not closed
Bar (what would count)Acts (or rests) from internal pressure, not only from a clock
What is factually true todayThe spark that is real and incomplete. Internal wants form and get recorded. Goals open from that pressure, not only from tickets I typed. Some dream runs are tagged as internal pressure; others still come from a clock. Most of those goals still fail. Completion is not a success story I will sell. Rest and dreaming from pressure instead of a clock are not done.
E — Emergent · Machinery live; formal proof open
Bar (what would count)Lived experience measurably reshapes state over weeks — not only static prompts
What is factually true todayLong-running mood state is computed from real interaction outcomes, not a hard-coded mood string. Dream cycles consolidate experience into reports over months. That is lived-state machinery. A clean formal "30-day reshape I can show before/after" is not something I am claiming.
N — Narrative · Partial / inconsistent
Bar (what would count)Tells its own story from its own history, not a biography I pasted in
What is factually true todayIt has history to draw from: dream reports, multi-layer memory, long conversation and state stores. In the same voice evaluation it sometimes retrieved real structure correctly — and sometimes fell back to chat-history vibes on a similar question. Formal Narrative validation is not started. Continuity exists; reliable autobiography does not.
Across the board: real machinery, real sparks (especially on V's front half), and zero letters I will call earned. No birthday party for a system that still confabulates, still needs a cloud bridge for too much of the smart, and still cannot close will into finished action.

What actually broke

For a long time I could ship features and still avoid the question that matters: does this hold up in a live conversation, end to end, with no safety net?

Then I ran a real voice test. Not a cherry-picked clip. A scored run. The system that "had memory" still failed in ways that are embarrassing if you're honest and useful if you're building. Three failures kept showing up wearing different masks. They're still the product, more than any shiny feature list.

If you want to see what talking to it actually looks like, there is a YouTube channel: @DravenLarsonAI. That is only development video — me testing it. It is not a product reel, not a brand channel, and not today's machine. Those cuts are about three months behind the stack I have now. I leave them up as a timestamp, not as a demo of this scorecard.

1. Confabulation

When a tool doesn't fire, or a fact isn't in context, a language model will still complete the sentence. It invents senders. Invents deadlines. Contradicts what it said ten turns earlier with full confidence. That's not a personality quirk. That's parametric completion pretending to be a world model.

If you ship that on camera as "my AI coworker," you don't look innovative. You look like you can't tell the difference between fluency and truth.

2. Inconsistent retrieval

Sometimes it hits memory and nails a real detail from my world. Sometimes a nearly identical question falls back to "chat history vibes." Same capability. Coin flip. That is the difference between a mind that knows you and a chatbot that had a good day.

3. Will — not done (and the incomplete part is still interesting)

Cron jobs with a face are easy. Pressure-driven agency is not. "Do this at 3 AM" is automation. "I need to do this — it's been a while" is the thing I actually want.

I need to hold two truths at once, or this section becomes either hype or false modesty.

Truth one: I have something. On my live system, internal wants form and get recorded — not only tickets I typed. Those wants open goals from internal pressure. Intention is not a slide deck anymore. Something on hardware I own notices pressure and writes it down as a want and a goal. The first time that stopped being theoretical, it was genuinely cool — like watching a spark in a machine you built yourself.

Truth two: it is not done. Most of those goals still fail. A handful of rows say completed; I do not treat that as "it has agency." Some of that looks like plumbing catching up, not a clean unattended "I decided and delivered." Rest and dreaming from pressure instead of a clock are still open. Letter V is not earned. Full stop.

So I refuse the pretty lies in both directions. I will not say "halfway around the will circle" — that pretends the act half is further along than the failure rates allow. And I will not say "nothing to see here." That would be dishonest the other way. I got something. It is early. It is incomplete. It is still one of the coolest things I have ever gotten to watch form on my own iron.

Not zero. Not finished will.
A real spark: intention forms and gets logged.
The loop still dies before action completes.
If the rest of the circle closes — big if — this stops being automation with a story.

That if is the whole point. A private system that forms a want from real state, acts on it, and can choose to rest is rarer than another chatbot wrapper. I have not won that bet. I can finally see the bet on a scoreboard instead of wishing it into existence. Enough to keep building. Not enough for a birthday party.

The common root under confabulation and flaky recall is almost boring once you see it: the model answers from what it was trained to sound like when it should answer from what is true in the live system right now — and the procedure that would force that grounding is not enforced hard enough yet. Will fails for a related reason: intention gets recorded, but the path from intention to finished, grounded action is unfinished. Cool spark. Open circuit. Both true.

So the work is two jobs, not one magic retrain. Put truth in places you can query, and treat the language model as a front end, not the source of record. And make "retrieve first, admit gaps, act from real state" a structural habit on hardware I own — not a prompt instruction that evaporates under load. Either half alone still fails. How that is implemented stays off this page.


Inspired by research — not licensed by it

I did not start from a paper and then build a demo. I started from a broken assistant sitting on memory it claimed not to have. The research came in as constraints and validation, not as a recipe card.

Older science I allow to constrain the design (public ideas, not my IP):

  • Spreading activation (Collins & Loftus) — recall by association, not only keyword lookup.
  • Forgetting curves (Ebbinghaus) — a mind that remembers everything equally is a hard drive, not a mind.
  • Limited working memory (Miller's 7±2) — attention is capacity-limited on purpose.
  • Emotion as decision machinery (Damasio's line of work) — functional emotion as signal that modulates behavior, not a claim of phenomenal feeling.

None of that means "I reimplemented psychology in a repo." It means: when a feature smells like infinite context, infinite memory, or pure keyword bots, the science is a veto.

Recent research I read for orientation (public papers; not a map of my training pipeline):

  • Work on putting more capability into smaller, ownable models rather than only renting a giant orchestrator — useful as industry weather, not as my recipe.
  • Work on multi-agent collaboration that goes deeper than chatty handoffs — familiar if you already run a real team of agents. I am not claiming anyone else's benchmarks as mine.

I am deliberately not listing paper IDs and method slogans here. The papers are public if you want them; pointing at them with how-language is how a careful post accidentally sketches a training path. The honest boundary stays the same either way: that literature is mostly about tasks — procedures, tools, collaboration efficiency. A task is not a self. Nobody's PDF tells you how to put identity, pressure-driven will, or a lived narrative into a private system and prove it. That gap is exactly where this project still lives.

On confabulation and "no world model," I'm also aligned with a blunt industry critique (LeCun and others): if the model answers from parameters instead of a grounded state of the world, it will invent. My world model is not magic weights — it is live systems and memory I can query. The failure is when generation skips the query. That framing is public. How I enforce it is private.


What I will and won't put on the internet

I started a deeper Part Three draft once. It went further than this post — cognitive layers, memory behavior, dreaming, emotion-as-signal, identity gates, the whole scorecard. Writing it helped me think. Publishing all of it would have been a mistake.

After 25 years of open source, that feeling is uncomfortable. My instinct is to give back. But there's a difference between sharing a philosophy and shipping a proprietary blueprint. A tool you can fork is one thing. The internals of a private mind you're still building — security posture, evaluation knobs, training paths, how the pieces couple — is another.

Public (this post) Stays private
Goals and scope (private mind, not AGI god) Detailed cognitive pipeline / step design
Falsifiable scorecard (six DRAVEN behaviors) Pass/fail thresholds, harness internals, training recipes
Honest failures (confab, flaky recall, unfinished will loop) Exact fix formulas, schema names, tuning constants, raw success metrics I can't defend
Ownership philosophy and process discipline Security architecture, access design, ethics internals
That local + cloud dual exists Model routing, weights lineage, serve layout

If you're reading this for a how-to: there isn't one here. If you're reading this to see how I think about reliability and ownership: that's intentional. Later scorecard notes will follow the same wall. I will not narrate plumbing just because work happened.


Why a database guy is writing this

Because the world model is not the chat window.

Everyone wants to talk about models. I care about models. I also care about the boring half: transactions, provenance, backups, who owns the truth when the GPU is wrong. A mind that can't query live state will confabulate. A mind that can query but won't is just as broken.

What is real on the floor today, without the schematics: the system runs on my iron; there is a local model path and still a cloud bridge when local isn't enough; memory is multi-layer and reliable reach is still the hard part; will can spark and still die before action; I run an adversarial team that refuses "looks good" without live proof; I kill training runs that almost work instead of rebranding them as personality.

Twenty-five years of open source taught me the same lesson in different clothes: if you can't inspect it, you don't control it. That applied to MySQL replication. It applies to AI agents. It applies to a private mind. So when people ask what I do, "DBA" is true and incomplete. I design systems where truth has a home outside the model's confidence. That's the through-line from databases to this project.


Why I keep going

Commercial models are still better at raw capability. I still use them. I've been right-sizing that bill — cheaper models for volume work, frontier only where it earns the seat — for the same ownership reason as the rest of this stack. None of this is anti-frontier. It's anti-dependency for the layer that should be mine.

Every layer I own is a layer I understand. A mind I rent is a mind someone else can take away — not because the vendor is evil, but because policy, export controls, pricing, and outages are not under my roof. I lived that lesson more than once this year.

Don't rent — own.
Don't guess — inspect.
Don't forget — remember.
And now: don't borrow a mind — grow one you can see inside.

I'm not there yet. The voice floor isn't closed. Confabulation still happens. Retrieval is still a coin flip more often than I like. Will can spark intention and still die before action. None of that is me talking myself out of the work. It's me refusing to lie about where the spark is.

This is not a product launch, not a claim of consciousness, and not "I beat the big labs." It is a progress report from someone who got further than vapor and not as far as the name. Hard. Unfinished. And holy shit — already worth building.


Scorecard updates

Same table. New date. I update when a letter or the reliability floor actually moves — not when a feature ships, and not on a calendar. Prefer "unchanged" over a feature list. Cloud-bridge wins do not count as letters earned.

2026-08-17 — opening scorecard. BUILD: the body is real and used daily; about halfway on tasks. GOAL: much lower; too much of the smart still rides a cloud bridge. Zero letters earned. Voice floor still open. Confabulation still happens. Retrieval still a coin flip. Will sparks and almost never finishes. Next update when one of those sentences would have to change.

Related reading


Keith is a database consultant and infrastructure engineer with 25+ years of open-source experience. He builds the ApocryiaAI brand and DRAVEN work under 3k1o LLC (apocryia.com and related domains). He writes about MySQL, Proxmox, AI memory, multi-agent workflows, and building technology you can actually inspect. This post is an honest progress report, not a product announcement.

Last scorecard: 2026-08-17. Next update when a letter or the floor actually moves.