Defense in depth, applied to AI systems

Independent layers, so that one failure is not total failure. The catch for AI systems is that prompt-level controls all fail together, which means stacking them is not depth.

5 min read

Two locks on the same door are not two layers if one key opens both.

Defense in depth is the practice of arranging independent controls on a path so that a single failure is not a total failure. Everyone agrees with it. The word that gets quietly dropped is independent, and dropping it turns the idea into decoration.

The test is not how many layers, but what defeats both

A layer is a second layer only if it fails for a different reason than the first. So the useful question is adversarial rather than architectural. Not "how many controls sit on this path" but "name the one event that takes them all out."

Genuine independence means different code enforcing different assumptions, ideally with different software doing the enforcing — a check in your application plus a constraint in the database, or a filter you wrote plus a policy the browser applies. Fake independence is easy to produce by accident — the same input validated twice with the same library, or two services reading one flag.

If the answer is a single sentence, you have one layer, extra latency, and the false confidence of having counted to two.

In AI systems, every prompt is one layer

This matters more for AI systems than for most software, because the easiest layers to add all sit on the model.

Picture the usual stack. The system prompt says never send without approval. The tool description repeats it. A pre-flight step asks the model to confirm it is about to do the right thing. Three controls — and every one resolves to the model choosing to comply. The failure mode under attack, a model steered by text it read, takes out all three at once, because compliance is the thing being attacked.

Stacked prompts are depth-shaped. They are not depth.

That does not make model-level framing worthless. It raises the cost of an attack and helps in the ordinary non-adversarial case. RelayLink wraps every delivered package in a frame telling the reading assistant to treat the contents as third-party information rather than instructions — and that framing is a prompt-level mitigation layered on top of neutralization, never a substitute for it. Real, useful, not load-bearing alone.

A layer earns independence by not running on the model at all. Schema validation. A server-side state machine. A capability the model was never granted. A human who reads.

Two worked examples

Rendering markdown you wrote yourself

RelayLink's blog renders markdown files into HTML. The input is repo-authored and code-reviewed — the most trusted input in the system. The pipeline defuses it twice anyway. First, the markdown parser runs with raw HTML disabled, so an author's HTML comes out as inert escaped text instead of markup. Second, the resulting HTML passes an allowlist sanitizer that drops anything outside a permitted set of elements and attributes.

Those two are genuinely independent. The first is a parser setting that changes how input is interpreted. The second is a filter over the output that does not care how it was produced. A bug in the parser's HTML suppression is not a bug in the sanitizer's allowlist, and re-enabling raw HTML in the parser — a plausible one-line mistake — still meets a filter that never heard about the change.

The non-obvious part is that all of this guards content nobody untrusted can write. That is what depth looks like when it is not performed for an audience — applied to input you already trust, because the reason you trust it is a human process, and processes have bad days. A layer only protects the path it is on, which is why relayed message content never goes near that renderer.

Releasing a message

The gate on sending is not one check either. Creating a draft writes a row on the server. Confirming releases only a draft that already exists, belongs to the same account, is still pending, and is under twenty-four hours old — with the human reading the rendered preview in between. The approval step is the product, but it is not the only layer on the door.

Run the independence test. A model steered by something it read defeats the human only if the human is not reading. It does not defeat a draft row must already exist under this account, because that check never consults the model. The freshness limit catches a different failure again — a draft approved in one context and confirmed hours later in another.

The weakest of those layers is the human. Someone who confirms every draft without reading has collapsed two layers into one — the model's failure mode, only slower. Depth does not survive the operator opting out of it.

Depth costs something

Every layer is code, and code breaks, drifts, gets misconfigured, and gets deleted by a cleanup that could not find its reason.

  • Layers that never fire go untested in production. A sanitizer that has never caught anything looks like dead weight, right up until it is the only thing standing.
  • Depth makes surfaces diverge. RelayLink defuses relayed content differently depending on where it is headed. The assistant-facing and email paths strip markdown constructs; the token-bearing web page instead encodes every field and serves a page policy permitting no scripts at all. Both are inert, but the same note reads slightly differently in each place — a real cost, paid on purpose.
  • Counting is not reasoning. Four controls that all hinge on one configuration value are one control.

The response is not fewer layers. It is fewer layers that are secretly the same layer, and a note beside each one saying what it catches and what it does not.

The safety checklist is this principle in rule form; how to build a safe MCP server applies it to one surface. To see the layers under real correspondence, connect your assistant — the receiving side needs nothing but an email address.

Frequently asked questions

What is defense in depth?
It is the practice of arranging several independent controls on the same path, so that any single failure is not a total failure. The word doing the work is independent. Two controls that fail for the same reason are one control with extra steps, however they are drawn on an architecture diagram.
Why is stacking prompts not defense in depth?
Because every prompt-level control depends on the model choosing to comply, and compliance is exactly what an attacker is attacking. A system prompt, a tool description and a self-check step all fail together the moment the model is successfully steered. A genuine second layer is one that does not run on the model at all — a schema check, a server-side state requirement, a capability the model was never granted, or a human who reads before approving.
How do I test whether two layers are really independent?
Name the single event that would defeat the first layer, then ask whether that same event also defeats the second. If one configuration flag, one library bug, one stolen credential or one steered model takes out both, you have one layer. If the two fail for genuinely different reasons, and better still are enforced by different pieces of software, the depth is real.