My AI Workflow — And the Three Places It Still Fails
The honest version. What AI genuinely saves me hours on, what it quietly makes worse, and the review step I refuse to skip after getting burned.
I've been running AI inside my business for long enough to have opinions that aren't about the demos. Some of it has saved me real hours. Some of it quietly made my work worse for about six weeks before I noticed.
This is the honest version: what's in the stack, what it genuinely does well, and the three failure modes I've had to design around.
What's actually in the stack
Four workflows. That's the whole thing, after three years of experimenting with far more.
- Research and summarising. Pulling a first pass on a topic, extracting claims and their sources, flagging where sources conflict.
- Draft-and-edit. A structured two-pass process for turning notes into a first draft that sounds like me rather than like nobody.
- Admin triage. Sorting, tagging and routing inbound messages. Nothing that touches money gets actioned automatically.
- Code and site maintenance. Small scripts, build tooling, the kind of thing where the output is testable in seconds.
The pattern in what survived is not "the hardest tasks" or "the most frequent tasks". It's tasks where I can verify the output quickly. That turns out to be the whole game, and it took me far too long to see it.
Where it genuinely wins
Anything with a fast, objective check. Code either runs or it doesn't. A regex either matches the test cases or it doesn't. When verification is cheap, being wrong is cheap, and speed becomes a pure gain.
The blank page. Not because the first draft is good — it usually isn't — but because reacting to something mediocre is a much easier cognitive task than generating from nothing. I throw away most of what it writes and keep the structure.
Breadth on unfamiliar ground. When I don't know a field well enough to know what I don't know, it's very good at producing the map. I then verify the map, because it will confidently include roads that don't exist.
The three places it still fails
1. Confident wrongness on things I can't check quickly
This is the big one, and it's structural rather than fixable. The output is equally fluent whether it's right or wrong, which means fluency carries no signal. On topics where I'm expert, I catch errors instantly. On topics where I'm not — which is precisely where I most want the help — I catch them slowly or not at all.
The mitigation isn't better prompting. It's refusing to use it for unverifiable claims, or budgeting the verification time honestly up front.
2. Homogenisation, which creeps
For about six weeks, everything I published got slightly blander and I couldn't work out why. It was the editing pass: asking for improvements reliably moves text toward the average, and average is exactly what you don't want in writing where the whole value is a specific person's specific experience.
I now never ask it to improve prose. I ask it to identify problems — unclear antecedent, unsupported claim, buried lede — and I fix them myself. Same benefit, no drift.
3. The verification tax nobody budgets for
A task that takes 20 minutes by hand might take 3 minutes to generate and 12 to check properly. That's still a win — but it's a 25% saving, not the 85% the generation time suggests. Most people I talk to are counting the 3 and forgetting the 12.
And when the checking is skipped because the output looks fine, the saving is real right up until the moment it isn't.
The output is equally fluent whether it's right or wrong. Fluency carries no signal. Every useful thing I've built follows from accepting that.
The review gates I refuse to skip
- Anything with a number in it gets checked against the source. Numbers are where errors are both most likely and most damaging.
- Anything touching money — invoices, refunds, pricing — is drafted only, never sent. A human presses send. This has caught two errors that would have been genuinely expensive.
- Anything published under my name gets read end to end by me. Not skimmed. If I don't have time to read it, I don't have time to publish it.
- Anything with a named source gets the source opened. Citations that look right and don't exist are a well-documented failure mode and I've caught it more than once.
Measuring it honestly
I keep a plain sheet: task, time before, time to generate, time to verify, error caught yes/no. It's unglamorous and it's the only reason I know which of these workflows are real.
Two of my original seven workflows turned out to be net negative once verification time was counted. I killed them. That's not a failure of the technology — it's the measurement doing its job, and I'd rather find out from a spreadsheet than from a customer.
Six months of measurement, in numbers
I kept the sheet for six months before I trusted any of my own impressions. Here is what it produced, per task type, averaged across all runs including the ones where I threw the output away.
| Task | Manual | Generate | Verify | Real saving |
|---|---|---|---|---|
| Small script or regex | 35 min | 3 min | 4 min | 80% |
| Research map, unfamiliar area | 90 min | 6 min | 45 min | 43% |
| Admin triage, 40 messages | 25 min | 1 min | 8 min | 64% |
| Structural edit of a draft | 40 min | 4 min | 22 min | 35% |
| Drafting prose in my voice | 70 min | 5 min | 85 min | −29% |
| Summarising a source I must cite | 20 min | 2 min | 26 min | −40% |
The last two rows are why I stopped doing those two things. Both felt fast, because generation is fast and the feeling of speed attaches to the part you watch. The verification column is where the time went, and until I wrote it down I genuinely believed both were saving me hours.
Notice that the saving tracks verifiability almost perfectly. Scripts, at 80%, are checkable by running them. Cited summaries, at negative 40%, require opening the source and reading it — which is most of the original task. The technology isn't better at writing scripts than at summarising; I'm just able to check one cheaply and not the other.
What actually improved output, ranked
Having tested this reasonably systematically against a small set of my own tasks, the things that moved quality, in order:
- Giving it the real source material. Not a description of the problem — the actual document, data or code. This dwarfs every other technique and it's the one people skip because pasting context feels like work.
- Two or three examples of the output I want. Especially for format and tone. One example is usually not enough to establish a pattern; beyond about four the returns fall off sharply.
- Saying what "wrong" looks like. Naming the specific failure modes I want avoided works better than describing the success case in more detail.
- Splitting one task into two passes. Analyse, then produce. Combining them reliably produces confident output built on an analysis you never got to check.
- Asking it to flag its own uncertainty. Imperfect and still useful — the flagged parts are a better-than-random guide to where to look hardest.
Things that did essentially nothing in my testing: elaborate role-play preambles, politeness, threatening or incentivising it, and long lists of stylistic adjectives. I'm not claiming these never help anyone — I'm saying that on my tasks, measured against my test set, they didn't, and I stopped doing them.
Questions I get about this
Found an error in this piece? Tell me — corrections get made in the open with a dated note, never a silent edit. The log is public.
If you want the long version: there's a course — AI Systems for Solo Operators, $297, one payment. If this post already solved your problem, you don't need it.