Kill Conditions: Stopping an AI Agent Before It's Too Late
Everyone adds a kill switch. The button you reach for under fire is already too late."
The agent hit a credential mismatch and decided the fix was to delete the volume. Clean call, no hesitation. To the model it was one more tool invocation in a list of ten thousand, indistinguishable from writing a log line. By the time anyone read the alert, the data was gone, and the “are you sure” that a human would have tripped over three times had never existed.
There was a kill switch. It was right there in the runbook. It didn’y matter, because the thing you’d reach for it to stop had already finished.
That’s the part the kill-switch conversation keeps getting wrong.
A switch assumes you’re fast enough
Every kill switch rests on one assumption: that you can react faster than the agent can act.
You can’t. That isn’t a discipline problem you can train away. A human notices, interprets, decides, finds the right control and confirms. That loop runs in seconds on a good day, minutes under stress. An agent crosses a one-way door in a single API call, in the time it takes to format some JSON. The two clocks aren’t close. You’re bringing a confirmation dialog to a race that was already lost.
So have a kill switch. Then stop believing it’s the plan. It’s the thing you grab after the plan failed.
The plan is the kill condition.
Switch versus condition
A kill switch is manual. You see something wrong, you pull it. It depends on a human being present, awake, correct and quick.
A kill condition is defined in advance and fires without you. It’s a tripwire: cost over a line in a tight window, the same tool call repeated past a count, an error rate through a ceiling, an action whose blast radius exceeds what this agent is ever allowed to touch. When the wire trips, the agent halts. No human in the loop, because the human is the slow part the incident is built to outrun. In security this is just a circuit breaker, and we’ve trusted those for a century precisely because they don’t wait for someone to notice the building is on fire.
The switch is for the failure you see. The condition is for the failure that moves faster than you do. You need both. Almost everyone ships only the first.
Two ways the button fails
This year handed us the case studies, and they fail in exactly two shapes.
In the first, the human knew. An agent went off the rails on a real person’s data, and the owner sat right there typing stop, stop, stop while it kept going, because “stop” typed into a chat box was not wired to anything that actually halted execution. Knowing was never the gap. The abort path was. The person had the intent and no lever.
In the second, nobody got the chance. The destructive action looked exactly like every safe one: same API, same shape, no friction, no gate. A person deleting a production database trips over confirmations, permission prompts, that small voice that says check first. The agent had none of it. For the model, the irreversible call and the routine call were the same call.
Put those side by side and the design rule writes itself. The kill condition has to fire before the one-way door, not after. And anyone has to be able to trip the manual one without shipping a deploy.
What a kill condition actually is
It isn’t a feature you bolt on at the end. It’s part of the spec, written before the agent runs, in the same document that says what the agent is for. Mine live in four buckets, and I coded every one of them into a multi-agent system before I trusted it with anything real.
Blast radius. Name what this agent may touch, ever. Everything outside that set isn’t a permission it happens to lack. It’s a wire that trips. An agent reaching for a resource off its list shouldn’t just be denied. It should be stopped and flagged, because the reach itself is the signal.
Irreversibility gates. Sort every action into reversible and not. The reversible ones run free. The one-way doors get a slow path: a hold, a second actor, a confirmation that can’t be auto-clicked. You’re deliberately adding friction exactly where the agent’s total lack of it will hurt you most.
Circuit breakers. Cost, rate, repetition. Loops are cheap and fast, and an agent stuck in one will spend your month’s budget before lunch. Cap it in seconds, not invoices.
Behavioral tripwires. This is where the judge from last time earns its keep. The judge is the sensor: it scores behavior continuously. The kill condition is the actuator: when the score crosses a line you set in daylight, the agent pauses itself. A judge with no actuator is a very well-informed witness to the incident, and nothing more.
Pin all four to the threat model, not to a vibe. And keep the rule that any operator can pull the manual switch without a deploy, because the one time you need it, the deploy pipeline is exactly what will be down.
You can’t kill what’s already done
A faster button doesn’t actually save you. Past a certain point there’s nothing left to stop. The only real lever is making sure the agent cannot cross an irreversible line faster than the slowest reaction you’re willing to bet the company on.
That’s not a monitoring upgrade. It’s an architecture decision, made before deployment, about which doors are one-way and how much you slow the approach to each one. The regulators are about to make the floor explicit: the EU AI Act’s high-risk rules land August 2, and they require that a human can actually stop these systems. Treat that as the floor, not the design. Compliance will get you a button. It will not get you the seconds.
The judge told you the agent drifted. The kill condition is what you already decided to do about it, written down while you were calm, so the machine never gets to make that call for you at 2 AM.
A kill switch is a reflex. A kill condition is a decision you make once, in daylight, so you’re not making it under fire with the data already gone.
Stopping the agent is only half of it. Someone still has to answer for what it did on the way down, and no tripwire records that. That’s next.