Thede Technologies

All posts

Notes from the AI Gatherings, Vol. 2

Second in the series - a new machine that bought capability instead of speed, prompt order as a performance lever, hooks over instructions, and the planet-sized gap between solo AI and an org-scale rollout.

Same table, same rules as Vol. 1: a few of us at Switchyards in Nashville on a Friday morning, comparing notes on building with AI. My own work by name, everyone else’s as patterns.

Three things from this week.

The model the laptop couldn’t load

Someone at the table bought a big-memory machine. This week they ran the benchmark. Same model, same prompt, laptop versus new box. Eighteen minutes became fifteen. You didn’t buy speed. You bought capability.

What the machine did buy was the ability to run a 120-billion-parameter model at all, one the laptop cannot load at any speed. That’s the real purchase. Not throughput. Access to a tier of intelligence that was off the table. If you’re sizing local hardware, the question isn’t tokens per second. It’s what’s the largest model I want resident in memory.

The prompt cache matches from the beginning and dies at the first character that differs, so an edit near the top forces everything after it back through the math on every run. Stable material first. Instructions, schemas, reference, examples. Volatile material last. The record, the question, the thing that changed.

“I no longer trust you to do this”

That sentence was said at the table, to Claude.

The setup. A rule in a CLAUDE.md file is a request. A hook is different. It’s a shell command the harness runs at a fixed point, before a tool call, after one, on session start, and a pre-tool hook can inspect the pending action and block it. So after one instruction too many got ignored, an attendee said the sentence above, and Claude wrote the hook, ran it, and demonstrated it blocking itself.

One limit surfaced too. Hooks fire around tool calls and lifecycle events, not between the model’s prose and your screen. There’s no hook that makes it less chatty. An attendee went looking for exactly that one and came home empty.

An agent has no cost model unless you give it one. One person’s enrichment tool was firing eighteen web searches in parallel, per record, until it was told the calls cost money. Another had every subagent inheriting the most expensive model available until it was handed a budget. Neither agent was misbehaving. Neither had been told.

If it matters, don’t leave it to persuasion.

The same model, an entirely different planet

I show up on Fridays and talk about personal leverage. Sitting across from someone rolling AI out to a company of several hundred people, I realized how little of what I know transfers.

Credentials pasted into chat channels. One-off apps deployed to cloud accounts the company doesn’t formally have. So the framing that works isn’t “you can’t do this.” It’s “here, take the keys safely.”

Two threads from that conversation are still open. Nobody has a title for the person who owns model choice, evals, and routing. Every company has someone quietly doing it. And nobody measures whether code review still happens. Every metric we have counts what shipped. None counts whether a human pushed back.


That’s Vol. 2. Same closing as last time. If you’re in Nashville on a Friday morning and this sounds like your kind of table, find me and ask.

All posts