The Agentic Software Factory
Planning, memory, verification, and personal software from builders using agents every day.
“The difference between vibe coding and agentic engineering is planning, architecture, and caring about the output.”
— Wes McKinney riffing on Jesse Vincent, ep-1 @ 11:51
We recently aired the first episode of Show Us Your Agent Skills, our new live show. Wes McKinney (creator of pandas, POSIT), Jeremiah Lowin (Prefect), and Randy Olson (Good Eye Labs) each walked Thomas Wiecki and me through the tack they actually use: the tools they’ve built, the skills they’ve written, and the habits that make agent-driven work hold together day to day. The full episode runs about two hours.
This post pulls out some of our favourite parts, for anyone who doesn’t have two hours to spare.
There are lots of ways we could have sliced this, but two themes kept surfacing:
Verification. Once agents are producing more work than you can personally inspect, the question becomes how to check it. Wes automates code review; Randy encodes visual judgment into a skill that critiques its own charts.
Personal software. Jeremiah’s Cardboard, OpenClaw memory, and Wes’s local tooling are all built for one user’s workflow. The point is not that everyone should use these tools, but that agents make this kind of bespoke software newly cheap.
Put together, the episode was less about any one tool than about the systems builders are wrapping around agents: review loops, editable memory, living skills, and software built for one person’s workflow.
This is an experiment. We’re trying to figure out how to communicate how people are actually building. For each guest we’ve given some context, drawn out the principles behind their setup, and described what a session looks like. We don’t expect anyone to jump in and adopt all of these tools, but hopefully the ways of thinking and the principles behind them will help in your own work.
Think of this as field notes, not a linear recap: jump around, follow what catches your eye, and tell us what you want more of. Check out the show website where you can find all live streams, upcoming episodes, and more.
👉Want to learn how to apply agentic engineering to the world of data science? Come build the future of Agentic Data Science with us in our upcoming course. It’s a live cohort with hands on exercises, capstones, and reusable agent skills, OSS code, and notebooks that will 10x your data science projects. Sign up here and use the code ADSVG10 for 10% off. Hit reply to enquire about group discounts.👈
Jump to:
What it means to use an agent as a second brain with Jeremiah
How to review a million lines of AI-generated code
Wes McKinney generates about a million lines of code every six months, across roughly a dozen projects, at 1.3 to 1.4 billion tokens a day. He reads almost none of it himself.
A daemon called RoboRev does. It runs as a post-commit hook on every project, firing a code review through Codex (GPT 5.5, reasoning xHigh) every time the writer agent commits. Findings accumulate in a per-repo ledger, and by the time a pull request merges, the code has been read by agents four or five times. With four or five projects in flight at once, multiple reviewer runs are usually grinding in the background.
The split between generation and review is asymmetric on purpose. Wes generates with a roughly three-to-one mix of Claude Code and Codex; the reviewer model is allowed to be slower and stronger because it runs once per commit. On the show, he pulled up a macOS widget showing $21,765.80 for what his last thirty days of usage would have cost at API rates.
Around RoboRev, Wes has built the rest of the stack:
Agents View: search across hundreds of sessions.
Middleman: local GitHub replacement, recent activity pinned to the top of pull requests.
Kata: local issue tracker, built after Beads “destroyed some of my Git repositories, very annoyingly.” ep-1 @ 13:25
Wes told us why he built Middleman to replace Github with trademark dryness:
“GitHub is in leagues with big scroll. They just really want you to have to scroll to the bottom of a pull request with like 100 comments.” ep-1 @ 24:21
On top of all of that he runs four or five Superpowers spec interviews in parallel across git worktrees.
His own job has shifted: he makes structural decisions about whether the work is scoped and structured correctly, and lets the agents read the code.
Vibe coding vs agentic engineering
Wes opened by quoting Jesse Vincent, the creator of Superpowers:
“The difference between vibe coding and agentic engineering is planning, architecture, and caring about the output.” ep-1 @ 11:51
He then picks it up in his own words:
“I’ve been building systems to do agentic engineering and not just vibe coding. I think when I started out, it was just like, ‘Okay, I’m going to open Claude Code in the terminal and type prompts and see what happens.’ And at a certain point, you’re like, ‘Wait, okay, this is not creating things that are good.’ How do we manage all this at scale? If you work on six projects in parallel, how do you keep the train from crashing into the side of a mountain?” ep-1 @ 11:59
At a million lines of code in six months, no human can read everything that’s being shipped. Either the agent is the planner, the architect, and the reviewer too, or you’re vibe coding at scale and putting garbage into production. Every piece of Wes’s rig is designed to be the first case.
The principles that make it work
Slow plan, long implement. Superpowers runs a long, detailed spec interview before any code is written, then hands off to a sub-agent execution skill that can grind for hours. Wes has run a single plan for fourteen hours straight, with forty-five tasks in it. The structure is what makes parallel projects possible: a slow plan is something you can step into and out of while it’s being formed, and a long implementation runs unattended while you’re elsewhere.
“One of the pros and cons of Superpowers is it generates amazing software, but it also takes a long time to generate very detailed implementation plans. The idea is that it doesn’t really trust leaving that much up to the agent in terms of making decisions.” ep-1 @ 26:33
Commit every turn. Wes’s hard rule, written into CLAUDE.md and AGENTS.md on every project: the agent commits at every turn. RoboRev’s post-commit hook fires at fine granularity, the review ledger stays accurate, and you can see at a glance what’s been read and what hasn’t. A long implementation that committed once at the end would defeat the entire rig.
RoboRev reads every line; you read for structure. RoboRev reads every line of generated code, every commit, multiple times before merge. The reviewer is Codex (GPT 5.5, reasoning xHigh), which Wes considers the strongest code reviewer available.
“I almost don’t read code now.” ep-1 @ 27:14
His own attention goes to structural questions: is the scope right, is the complexity warranted, is this the right thing to build at all.
“Sometimes the agents will go off the rails and build something that’s completely inappropriate, and so then you have to nuke that.” ep-1 @ 27:52
The daemon handles correctness. By the time a pull request merges, the code has been read by agents four or five times minimum.
Drain the ledger inline, not at the end. For long plans, Wes asks the sub-agent-driven-development skill to invoke the roborev-fix skill every five tasks. The agent pauses, picks up open findings from the ledger, fixes them, and continues. Without that, a fourteen-hour run buries the ledger so deep that the context is gone by the time anyone reads it.
What a session looks like
Four or five projects in flight at once, each in its own git worktree on a large monitor. At any given moment they’re at different stages, and you drop in on whichever needs you:
Spec interview. You answer the agent’s questions about scope and constraints, then leave it to keep planning.
Implementing. You don’t watch. You trust the long run.
Ready to merge. You check the cross-cutting dashboard, fold any outstanding RoboRev findings back into the session, merge what’s clean.
Wes kicked one of these off live on the show: a Spicy Takes dashboard prompted with a single sentence, with the agent starting to write the spec while RoboRev reviewed the spec commits in the background.
“I’m thinking about ideas for how I could turn this into more of an automated software factory and build even more software, but the trouble is that I’m bandwidth limited in terms of my ability to make decisions throughout the day. And so I feel like I’m already in all these spec interviews, like I’m already at my decision-making bandwidth. Like I can’t make any more decisions. It’s like, ‘Don’t ask me another question. I can’t make another decision today.’” ep-1 @ 30:04
The bottleneck is no longer code generation, or even line-by-line code review. It’s the builder’s ability to make structural decisions throughout the day: what is scoped well, what is too complex, what needs another pass, and when the agent has gone off the rails.
That velocity comes with a safety tradeoff. Wes runs in YOLO mode, but he is clear that the ergonomics are not solved yet:
“It’s sort of this balance between, do you want to be productive or do you want to be safe? And I think right now, to be maximally productive with agents requires you to be a little bit unsafe.” ep-1 @ 31:32
What it means to use an agent as a second brain
Jeremiah Lowin opens his segment by telling us what he’s really interested in right now: “custom software or just-in-time software,” built for one user. The second brain workflow below is the daily practice and Cardboard and Prefab at the end of his segment are paradigms.
He starts most workdays with a voice memo. Sometimes recorded on the morning commute (kids drop-off, around 30 minutes in the car), sometimes much shorter on a home-office day. He talks through what he’s thinking about, what he wants to do, what’s on the horizon, and drops it into the customised memory substrate of a harness called OpenClaw.
The premise
Asked what he loves most about working with agents, Jeremiah doesn’t say speed or autonomy. He says:
“That’s what I really love, is just pouring information in and then working to get it out.” ep-1 @ 38:02
Asked later what would scare him most if his agent skills got leaked, he gives the same answer!
“It’s the same second brain that I love so much. It’s all there, right? All my deepest and darkest fears and how I hate spiders and all that is probably in the brain of this thing.” ep-1 @ 42:38
The leverage comes from feeding context in for weeks or months before the moment you need an answer. The morning voice memo is the daily form of that habit and the customised memory substrate in OpenClaw is what makes asynchronous pour-in / picked-up-later work mechanically.
The principles that make it work
Feed the brain before you need it. Jeremiah talks the day’s work into a voice memo before he opens his laptop, and OpenClaw drops the transcript into the memory layer his agents read from. The memo is an update on what he’s done that the agents wouldn’t otherwise know about:
“In a previous era of LLMs, I’d be like, ‘That’s my system prompt for the day,’ and it kind of affects everything. Now with more modern memory systems, it’s sort of this update on what I’ve accomplished that might be out of sight for my agents, and that’s been a huge unlock for me.” ep-1 @ 37:22
Months later, those deposits are still there:
“There’s a talk I’m giving in three weeks for PyData London, so I can feed in something tonight, close it, don’t worry about it, talk to the agent about 1,000 other things, and then I can come back and we can actually pick right up because of the memory substrate there.” ep-1 @ 1:02:54
Pick a substrate whose memory you can edit. Jeremiah picked OpenClaw specifically so he could reach into the agent’s memory and change what it remembers:
“This is one of the reasons that I use an OpenClaw, for example, so that I can go muck around with its memory, in a way that works for me.” ep-1 @ 36:04
If the operator can’t edit what the agent remembers, the second brain is the vendor’s, not theirs.
Pick one tool for thinking, another for coding. Jeremiah does anything that draws on accumulated context inside OpenClaw: morning memos, ongoing planning, project state that has to persist across weeks. He writes code in Claude Desktop and Codex Desktop. He picked separate tools rather than asking one to do both jobs.
“I use OpenClaw as my main personal interface because of how I’ve customized its memory. When I’m working on code, I use Claude Desktop and Codex Desktop, which I migrated to from the CLIs mostly because of how much better it is at managing parallel sessions.” ep-1 @ 1:03:08
What a session looks like
Three states define the day:
Morning. Voice memo recorded somewhere you already are (the car, the kitchen, a walk). Talk through the day. Drop it into OpenClaw.
Through the day. Agents work in the background, reading from the same memory layer. You’re not watching them.
When you come back to one of them. Threads pick up via the memory substrate; a conversation you fed in weeks ago resumes where it left off.
You stay in the memory layer for thinking and planning. You move to the desktop apps when it’s time to write code.
For more on this territory, episode 2 has Thomas Tunguz’s local-first agents workflow, including why memory is still very much unsolved. Episode 3 has Eleanor Berger’s personal agent harness workflow with Hermes and GPT 5.5.
The anatomy of a skill
Jeremiah also talked us through several of his agent skills and talked us through how skills function:
“Skills are shockingly simple for how effective they are. They have two front matter: a name, that’s really important, that’s how you invoke it; and a description. And the description is always going to be seen by the agent.” ep-1 @ 51:03
A skill is a markdown file with two pieces of frontmatter: a name (used to invoke it) and a description. The description sits in the agent’s context all the time. The rest of the file is hidden until the agent decides to invoke the skill.
“Everything down here is only seen by the agent, if and when it decides to invoke the skill. So your description actually really matters... That progressive disclosure... is the magic of skills. You can have tons and tons of skills that are all shown as these little tiny descriptions, and then the agent’s like, ‘I need this skill. I need this skill.’” ep-1 @ 51:20
Each skill costs only its description in context until the agent reaches for the body. A skill body is markdown by default, but bundles can also include scripts and other business logic the agent invokes from inside the skill.
“It’s a polite note to your agent, and usually it does what the skill says.” ep-1 @ 52:14
Jeremiah’s ship-it skill exists for exactly one reason: by default, when you tell Claude “ship it”, Claude tries to merge the code. The ship-it skill is a polite note saying “ship it” means open a pull request.
explain came out of Jeremiah’s FastMCP work. He’s the core maintainer of the project, and the volume is lopsided:
“Code is so cheap and it’s just kind of getting lobbed over. There’s this real imbalance as a maintainer.” ep-1 @ 41:21
He runs many agents in parallel on contributions:
“I’m running 10 different agents on 10 different things over 10 different timeframes, and I’m cracking open my laptop with a coffee and I’m like, ‘What’s this one?’” ep-1 @ 46:55
The one sentence at the centre of the file:
“Talk to me like you’re explaining this to your colleague who knows about your project but wants to understand what you just did.” ep-1 @ 46:45
The full file is roughly 80 lines of markdown and is published in the show’s companion repo. The rest of it does three things:
Asks for a guided tour, not a diff. Start with the conceptual model (what is this system trying to do, what are the key abstractions), then the formal behaviour (invariants, contracts, the mental model someone needs). After that, what changed conceptually, what it enables, and what future work is teed up.
Explicitly rules out the failure modes. A “What NOT to do” section bans line-level diff narration, file-by-file summaries, mechanical descriptions of what was modified, changelog-style output. Without this, the agent defaults to those formats and the skill stops being useful.
Gets called by name from other skills. Jeremiah’s ship-it, github-reply, and skill-creator all invoke explain as a building block (”use your explain skill to post a reply”).
“This skill has become my workhorse. It is referenced in every other skill I have.” ep-1 @ 48:01
What MCPs and CLIs actually are
Because Jeremiah maintains FastMCP, we also asked him to spell out where skills, MCP servers, and CLIs fit.
Asked when to use skills versus MCP servers, Jeremiah split the two cleanly:
“Skills are awesome ways to steer behavior. They go into the agent’s brain in the exact same way that a message from you does... MCPs are great ways to distribute business logic from a central place.” ep-1 @ 48:57
The popular MCP-vs-CLI debate, he argued, is a non-question for most of MCP’s actual users:
“The MCP versus CLI style debate, which I think is stupid if you’re an individual, use whichever one you prefer, and non-starter if you’re an enterprise. We’re not installing a bunch of CLIs on people’s machines. We never have. We never will. It’s a nightmare. We’re going to distribute the business logic centrally.” ep-1 @ 49:21
Most people’s first exposure to MCP is third-party servers like the GitHub MCP server in their IDE. That’s a small slice of where it actually runs:
“Overwhelmingly, the use case for MCP is distributing internal business logic to internal teams in enterprises.” ep-1 @ 49:58
Personal software
Early in his segment, Jeremiah names a theme that runs under all of it:
“I don’t know if my way would work for anyone else. I think that’s kind of a theme of what I want to talk about today: custom software or just-in-time software or something like that.” ep-1 @ 36:16
The clearest example is Cardboard, the slide software he demos at the end of his segment. It lays out talks as acts → beats → slides, with a fixed colour scheme for speaker notes. The screen is read-only by design:
“This is not an interactive UI. This is read only... I interact with this entirely over an API or an MCP server, talking to it from any agent I want that can connect to it.” ep-1 @ 1:02:07
Read-only UIs need no form validation, no edit conflicts, no permissions model. The expensive parts live in the agent layer, which Jeremiah didn’t have to write. He drives Cardboard from OpenClaw by voice memo and conversation. The whole thing is days old at the time of recording (”came off the agent presses last night”).
“Purely for me, like no one else should use it.” ep-1 @ 1:00:43
“This has been really fun to have a piece of software that exactly makes talks the way I want them, the way I like to give them, and I don’t think anyone else would use it.” ep-1 @ 1:01:54
Prefab, the other piece Jeremiah demos, is one step broader than Cardboard. It’s a Python front-end framework for MCP apps, no backend required, that he spun out of FastMCP a few weeks before the show:
“I desperately wanted to create MCP apps in Python, and that meant I needed a Python front-end framework that didn’t require a backend, which almost every one of them assumes a very specific backend, which we don’t get here. We have an MCP server.” ep-1 @ 57:39
Prefab is what you reach for when the personal software you want to build is a dashboard rather than a one-off custom tool.
When Thomas asks whether this approach could be a model others adopt, Jeremiah told us:
“I’m into personal software basically in the most true sense. And so in a funny way I think that’s also resulting in very personal decisions of how you achieve that software.” ep-1 @ 56:33
He extends the point to the rest of the stack:
“So you have your harness, you have OpenClaw, you have Pi, whatever you Claude, whatever you want to use as your base substrate, and then the way people pile on functionality features, customize it, skills, is deeply personal.” ep-1 @ 56:47
Cardboard is built only for Jeremiah, edited the way he wants to edit it, in the vocabulary he uses for talks. The choice to build it on OpenClaw with voice memos through MCP is itself personal: OpenClaw fits because he’s customised the memory layer to work that way. The artifact and the stack underneath it are both shaped by the same one user.
Building an agent skill that grades its own work and the generator-evaluator pattern
Randy Olson publishes a data-viz post every morning, run by run from a single high-signal chart workflow. The chart is drawn, critiqued, and iterated on by the agent using the skill. The skill pulls the data, sketches several chart variants, grades its own output against Tuftean criteria (Edward Tufte-style judgment about clarity, data-to-ink ratio, annotation discipline, and whether the chart reveals the data), and hands the surviving versions to Randy. He picks one and posts it.
Randy moderates r/dataisbeautiful, co-founded Good Eye Labs, has worked in AI and ML for 15 to 20 years, and has been writing data-viz code since the early 2010s. The skill encodes most of that taste in markdown.
What makes it work is the verifier loop: Python checks what a script can measure (DPI, file integrity, basic constraints), and an LLM acting as a Tuftean judge evaluates what a script can’t (annotation placement, data-to-ink ratio, whether the chart actually says what it’s supposed to).
In that sense, Randy’s skill is personal software in that it encodes his judgment: which sources to prefer, which chart variants to try, which mistakes to reject, and how to revise when the output falls short. The key here is the reusable and iterative judgment loop.
The premise: tell it how to check it
“You don’t want to just tell it what to do, you also want to tell it how to check it.” ep-1 @ 1:29:32
This is Randy’s framing for the whole skill. The pattern is generate-and-verify: the skill produces a chart, then runs two gates on it before declaring it done:
A deterministic Python verifier checks what scripts can measure (DPI, file integrity, basic constraints).
A Tuftean LLM-as-judge handles what scripts can’t.
“You can’t programmatically assess images.” ep-1 @ 1:29:59
The Tuftean evaluator returns a pass-or-fail verdict and, on failure, feedback about how to improve the chart. That feedback becomes the agent’s next set of fix-it instructions, and the skill rewrites the chart and runs both gates again.
“Even still, if an eval flips or is wrong 80% of the time, it’s still directionally valuable.” ep-1 @ 1:41:37
LLM-as-judge has known variance; generate-and-verify still beats generate-and-hope.
The agent decides when the chart is good enough.
What a session looks like
Each morning, with one idea in hand:
Pick the idea. One sentence. “The history of marriage and divorce in the USA.” “No horse has broken Secretariat’s 1973 Kentucky Derby record.”
Fire the skill. The skill walks itself end to end: sets up an isolated working directory, searches the web for an authoritative dataset (CDC and other government and educational sources preferred), pulls the data, generates three chart variants in parallel, scores them, picks one, runs the verifier loop on the winner.
Walk away while it runs. In Randy’s demo, dataset discovery took five to ten minutes; the rest is fast.
Come back. Read the chart and the agent’s log. Check that the log matches the chart.
Do the last-5% human pass. Almost entirely one problem: detecting when an inline annotation overlaps a line. “If anyone solves this consistently, you’re a billionaire.” ep-1 @ 1:33:09. Adjust if needed.
Publish, then edit the skill. Anything the verifier missed, anything the design checklist could catch next time. The skill is a living document, not a frozen file.
Three principles for designing skills
Skills are polite notes, not programs. This is Jeremiah’s framing: a skill is a markdown file that politely asks the agent to do a thing. The agent usually does what the skill says; sometimes it doesn’t. The point is that writing one is cheap, and you can iterate fast as you notice new failure modes. ship-it is the example: a polite note saying “ship it” means open a pull request, not merge.
Deterministic in scripts, ambiguous to the LLM. This is Randy’s framing:
“A skill plus an LLM is kind of a program.” ep-1 @ 1:17:32
What he means: when you write a skill, you’re writing the deterministic spine of something program-like. The LLM fills in the parts you can’t write as deterministic code. Anything you can specify exactly (DPI, file integrity, axis units, dataset URLs) goes in scripts the skill invokes. Anything that needs judgment (which chart type fits this data, whether the chart is Tufte-compliant, what to do when the verifier fails) goes to the LLM. The skill is the seam.
Skills are living documents. Randy names where the editable surface actually lives:
“Even the harness, unless you’re hardcore and making your own, is kind of fixed too. The skills are really the thing that can evolve with you.” ep-1 @ 1:21:07
Both Jeremiah and Randy edit their skills constantly. Randy tweaks his data-viz skill after most mornings: the phases, the design checklist, the verifier criteria. The evaluator is part of that loop too: each time the skill misses a bad chart, he can update the checklist, verifier, or judge prompt so the next run catches more of what he would have caught manually. Jeremiah edits explain and his other skills as he notices new failure modes (which is most of why ship-it exists). The markdown-file format is what lets either of them treat the skill as a living document rather than a frozen artifact.
If you take one thing away from this post on how to build with agents, make it the loops: plan before the agent writes, review what it ships, keep memory editable, encode your taste into skills, and improve the checks every time they miss something.
Links
Check out the show website where you can find all live streams, upcoming episodes, and more. The companion repo has skills, configs, workflows, and more from each episode.
👉Want to learn how to apply agentic engineering to the world of data science? Come build the future of Agentic Data Science with us in our upcoming course. It’s a live cohort with hands on exercises, capstones, and reusable agent skills, OSS code, and notebooks that will 10x your data science projects. Sign up here and use the code ADSVG10 for 10% off. Hit reply to enquire about group discounts.👈
How You Can Support Vanishing Gradients
Vanishing Gradients is a podcast, workshop series, blog, and newsletter focused on what you can build with AI right now. Over 70 episodes with expert practitioners from Google DeepMind, Netflix, Stanford, and elsewhere. Hundreds of hours of free, hands-on workshops. All independent, all free.
If you want to help keep it going:
Share this with a builder who’d find it useful