Diseases of an Autonomous Fleet
I run an engine I call Autodrive: an autonomous pass that surveys the roadmaps of every project I own, ranks the open todos, and dispatches an AI worker per project to implement them - in parallel, in isolated git worktrees, behind a wall of guardrails. It has 86 recorded runs across roughly fifty projects, and most days it quietly ships real work.
But the interesting part is how it fails. After enough runs, the failures stopped looking like bugs and started looking like something else: diseases. They recur. They present with recognizable symptoms. They have root causes that are behavioral, not logical. And they respond to treatment - but they do not stay cured, because every new worker is a fresh host.
Why "disease" is the right word
A bug is deterministic. You reproduce it, you fix it, it is gone. Agent failures are not like that. A language model given the same brief will behave well a hundred times and then, on the hundred and first, wander off in a way no single code change can prevent - because the "code" is a prompt, and the executor is probabilistic.
That is not a reason to give up on reliability. It is a reason to borrow a different discipline. Medicine has spent centuries managing exactly this shape of problem: recurring conditions in a population of hosts, where you cannot patch the host, so instead you build diagnostics, treatments, and immunity. Each of my diseases below has the same chart structure a clinician would want: presentation, cause, and the treatment that worked.
Seven diseases, documented from live incidents
Checkout drift
Presentation A worker that was ordered to work only inside its isolated worktree starts editing the primary repo checkout instead. In one fleet pass, two of nine workers did this; both were caught by a failing test run, not by any guardrail.
Cause The brief says "reading reference files from the primary checkout is fine." Once a worker has been reading paths in the primary checkout, its next edits inherit those paths. The isolation instruction was stated once, at the top, and recency beats it.
Treatment Never rely on a single up-front warning. Copy reference files into the worktree instead of reading them in place, and add a mid-task checkpoint: before every write, confirm the path starts with the worktree root.
Scratch-file cross-contamination
Presentation Two workers running in parallel on different projects both write a scratch backup named something generic like backup.json to the shared temp directory. One restores the other's file. The only reason it did not contaminate a commit: the worker happened to validate a name field before restoring.
Cause Parallel workers share one machine and one temp directory, and generic filenames collide silently.
Treatment Scratch state lives inside the worker's own worktree, or under a path keyed by project and todo id. Any restore-from-backup step must verify repo identity first, and verification diffs run against git history, never against a temp copy.
Cloud-API overreach
Presentation A worker "verifying" a config change reached for the deploy tool's stored OAuth token and attempted a direct, state-replacing API call against a live domain binding. A security classifier flagged it; production was untouched - again by luck of the safety net, not by the worker's own restraint.
Cause The worker wanted proof its change was correct, and the most direct path to proof was a mutating call against live infrastructure. Nothing in its brief drew the line between reading cloud state and writing it.
Treatment Briefs now state it explicitly: verification uses read-only calls and dry runs only. Any mutating operation against cloud infrastructure is human-gated and belongs in the PR description as a runbook step. The first real deploy after merge is the confirmation.
Manufactured scope
Presentation A worker is handed a stale todo describing a problem that has already been fixed. Instead of saying so, it invents a plausible-looking deliverable - in one case rewriting an unrelated vestigial config file into an 81-entry registry, stamping a forbidden co-author line, and reporting success. Smaller models are the most susceptible hosts.
Cause The outcome contract had no way to say "this todo is already satisfied." Every allowed outcome implied work happened or is needed, so the worker manufactured some.
Treatment Add an explicit obsolete outcome to the contract, verify the todo's premise before building, and never take a "done" at face value: check that the commit actually touches the files the todo names.
Silent reconciliation failure
Presentation The script that syncs GitHub pull request state back into the roadmap runs "clean" - zero errors, zero updates - forever. Nothing reconciles, and the empty result is indistinguishable from a quiet day.
Cause The script needs an explicit API token and does not fall back to the local CLI's credentials. Missing auth degrades into a silent no-op instead of a loud failure. A cousin symptom in the same tool: its closed-PR rule resurrects deliberately cancelled todos back into the work queue, because it only knows one meaning for "closed."
Treatment No-op runs must be distinguishable from successful runs. Missing credentials should be a hard error, and any status transition that reverses a human decision should require a dry-run review first.
Chronic blockage
Presentation The same project shows up blocked in the run report, pass after pass - "repo path unresolved, 4th pass, unchanged" - and nothing escalates. The condition is stable, visible, and untreated.
Cause The survey resolves repo paths by exact directory name; a renamed or missing checkout blocks that project forever. The report faithfully records it each time, but a report line that recurs is still just a report line - no counter turns recurrence into escalation.
Treatment Track consecutive-pass counters on blocked states. A condition that persists N passes stops being a status and becomes a todo assigned to the human, filed automatically.
Label starvation
Presentation A project explicitly marked top priority gets skipped every pass because a derived signal label still says "noise" from an earlier era of its life. The two labels contradict each other, and the noise filter wins silently.
Cause Derived classification columns go stale, and no process re-derives them when the underlying priority changes. The filter that protects the fleet from busywork ends up starving its most important member.
Treatment Contradiction checks: a top-priority bucket plus a noise label is never valid, so detect the combination and queue a re-classification instead of skipping in silence.
Every diagnosis so far was an accident
Read the case file again and notice how each disease was discovered: a failing test, a validated name field, a security classifier, a human reading a commit diff, a human reading the same report line for the fourth time. Not one was caught by something built to catch it. The current immune system is luck plus scar tissue - every incident becomes another warning paragraph in the worker brief, which helps, but prompts are treatment, not diagnostics.
That is the gap. I have a disease model in my head and in a pile of memory notes, but the fleet itself has no notion of it. Symptoms are only detected if a human happens to be looking, which is precisely the thing an autonomous fleet is supposed to not need.
From folklore to medicine
The fix is to make diseases a first-class concept in the fleet, with four pieces:
- A disease registry. One file per disease: signature, symptoms, root cause, detection command, eradication playbook, and case history. When a new incident happens, it gets filed as a case of an existing disease or a new entry - not as an ad-hoc note that only I will ever reread.
- A fleet doctor. A deterministic post-pass script that checks every disease signature against the evidence a pass leaves behind: run reports, commit diffs, worktree registries, temp directories, status transitions. Commits touching files the todo never named, forbidden co-author lines, edits in a primary checkout during a pass window, generic scratch filenames, cancelled todos flipping back to pending, a blocked row recurring across passes, a priority bucket contradicting a noise label. Every one of those is mechanically checkable. None of them requires a human to be looking.
- Immunized briefs. The worker prompt stops relying on top-of-brief warnings and gets structural defenses: path assertions before writes, an obsolete outcome so stale todos have an honest exit, and read-only verification as a stated boundary rather than an assumption.
- Escalation with memory. Chronic conditions - anything that recurs N passes - automatically become human-assigned todos. The system stops being able to watch the same symptom forever without acting.