Local Inference on Thede Technologies

Notes from the AI Gatherings, Vol. 2

Tue, 13 Oct 2026 09:00:00 -0500

Same table, same rules as Vol. 1: a few of us at Switchyards in Nashville on a Friday morning, comparing notes on building with AI. My own work by name, everyone else’s as patterns.

Three things from this week.

The model the laptop couldn’t load

Someone at the table bought a big-memory machine. This week they ran the benchmark. Same model, same prompt, laptop versus new box. Eighteen minutes became fifteen. You didn’t buy speed. You bought capability.

An Agent Reads My Email

Thu, 06 Aug 2026 00:00:00 +0000

Every morning at five, an agent reads yesterday’s email and writes me a briefing. Most mornings I read the briefing instead of the inbox.

My friend Kyle Reinford just published the pattern he uses - Cloudflare Email Workers catch messages at the edge and file them into R2, so a model never touches his real Gmail. Mine is the same idea from the other end: he guards the front door, I process at home. Apple Mail already syncs my accounts to disk. A script reads the last 24 hours of messages and hands each body to a local model - Qwen3.6-35b-a3b, running in LM Studio on my Mac Studio. No agent harness, no tool loop - just one chat call to localhost per message. All the model gets to do is fill out a tiny form: what is this, does it need me, one-line summary. A renderer turns the day’s forms into a static page. Nothing leaves the house.

Notes from the AI Gatherings, Vol. 1

Fri, 24 Jul 2026 00:00:00 +0000

A format note, since this is the first one. Most Friday mornings I meet a few friends at Switchyards in Nashville to compare notes on building with AI. No agenda, no demos-or-else, no recording - just a couple of hours of “here’s what I tried, here’s what surprised me.” It’s become the most useful standing appointment on my calendar, because the things you learn tool-to-tool never make it into anyone’s launch post. One of those mornings already sent me home to rebuild my entire engineering workflow. This series is my notebook from those mornings - short, occasional, and anonymized. The rule is simple: I write about my own work by name, everyone else’s not at all. What you get is the patterns.

The Machine I Can't Turn Off

Wed, 07 Oct 2026 09:00:00 -0500

I told my partner the other night that I can never turn the Mac Studio off.

There’s a job on that machine at 11:45 at night. One at 3:30 in the morning. One at five, one at seven. Every one of them is an agent. It wakes up, reads something, thinks about it with a model that runs entirely on the chip under my desk, writes something down, and goes back to sleep.

Can a Local Model Read My Inbox? A Bake-off

Sat, 27 Jun 2026 00:00:00 +0000

How much of my daily AI work actually needs to leave my machine? The frontier models still win the hard, open-ended thinking. But most of my real workload isn’t that - it’s repetitive, high-volume, and touches data I’d rather keep home: my mail, my documents, decades of personal history.

Email was the perfect test. I was building a system that reads my inbox for me - each message becomes a small structured record, so a daily brief can be written from records instead of raw mail. That per-message reading should run locally, if local is good enough.

Getting good output from Lens

Tue, 05 May 2026 00:00:00 +0000

A Lens daily summary derailed by reasoning artifacts: **People:**, **Patterns:**, **Drafting structure:**, “Constraint Check”, bleeding into the narrative

People pick up Lens, connect it to LM Studio, point it at whatever model they downloaded first, and expect polished daily snapshots. Sometimes they get them. Sometimes the same setup on a different machine produces output that reads like the model thinking out loud, or worse, fabricating events that didn’t happen.

Lens itself doesn’t change between those outcomes. The model, the quantization, and a few LM Studio settings do. This is the field guide I wish I’d shipped with the first build.

Why personal AI belongs on hardware you already own

Mon, 27 Apr 2026 00:00:00 +0000

Tim Cook stepped down this week. The next CEO of Apple is John Ternus, the hardware engineer who ran the Mac transition to Apple Silicon. The new Chief Hardware Officer is Johny Srouji, who built the chips that made the transition possible. Two hardware people at the top of the company, in the same month the cloud AI labs are losing money on their best customers.

Nate B. Jones made the case that this is Apple declining the race the rest of the industry is running and lining up for a different one. I think he’s right, and I want to add what I can only add from inside it: this is the race I’ve already been running on a four-year-old machine under my desk, and the people I trust are starting to run it too.

A general-purpose MoE multimodal beat every dedicated vision model on my father's handwriting

Thu, 23 Apr 2026 00:00:00 +0000

I’ve been transcribing a multi-generational archive of handwritten family letters on my own hardware. Posts one and two covered why and how. This post is the surprise.

I assumed the right tool for a vision task was a vision model. If you’re reading handwriting, you reach for something labeled “VL.” If you can find one fine-tuned for OCR and handwriting, even better.

I was wrong, at least at the sizes I can run locally in 2026.

Running real models locally on a Mac Studio that isn't new anymore

Thu, 23 Apr 2026 00:00:00 +0000

I have a multi-generational archive of handwritten family letters in my house, and I wanted to read it without sending the private content to a cloud provider. The first post is the why. This one is the how, on the specific hardware I already own, with the software I landed on.

My Mac Studio is almost four years old. M1 Max, 64 GB of RAM. It’s proven more than capable of running large language models locally. It keeps private content in the house, and it costs a lot less than any cloud provider.

What family archives are for, now that the AI can read them

Thu, 23 Apr 2026 00:00:00 +0000

I have three collections of letters sitting in my house.

The oldest is a stack from my great-grandparents, written between 1910 and 1913. The second is from my grandparents, written during World War II. The third is from my father, letters he wrote from the moment I was born until the moment his mother died.

Three generations, each one writing to the next, each one leaving behind something I’ve never fully read.