Measuring Your AI Exit Cost: An Architecture Review

On LinkedIn, I make the executive argument: your AI problem is not cost, it is dependency. Cost is what you pay this quarter; dependency is what you pay at every renewal, for a decade. This post is the engineering companion. If dependency is the real exposure, it should be measurable — and anything measurable can be architected down.

So: how do you actually measure your AI exit cost? Not as a slide. As an audit.

Exit cost is a stack, not a number

Teams tend to equate switching providers with changing an API key. That was roughly true in 2023. In an agentic estate, provider coupling has accumulated in five distinct layers, and each has its own unwind cost.

Layer 1 — API surface

The cheapest layer to fix and the first to audit. If application code calls a provider SDK directly, every model reference is a migration line-item. The fix is a gateway: one internal endpoint, provider adapters behind it, model names resolved by policy rather than hard-coded. If you cannot change your default model in one config file, start here — nothing else in this audit is testable until you can.

Layer 2 — Prompt and behavior coupling

Prompts are tuned, consciously or not, to one model’s quirks: its instruction-following style, its formatting habits, its tool-calling dialect. This coupling is invisible until you swap models and watch pass rates drop. The countermeasure is an evaluation suite that defines correct behavior independently of any provider — golden datasets, structured-output checks, task-level assertions. Your eval suite is your portability contract. No evals, no measurable exit cost; only guesses.

Layer 3 — Data gravity

The heaviest layer. Embeddings are tied to the model that produced them: switch embedding models and every vector in the store must be regenerated — a compute bill and a re-indexing window that scales with your corpus. Fine-tunes are worse: weights trained on a proprietary base do not travel at all. Rule of thumb: prefer retrieval over fine-tuning wherever quality allows, and record the full re-embedding cost of your corpus as a standing line in the exit ledger. If you do not know that number today, that is finding #1 of your audit.

Layer 4 — Orchestration and agent coupling

Agent frameworks encode provider assumptions: tool-schema formats, context-window sizes, retry semantics, reasoning-token behavior. A workflow that plans, retrieves, calls tools, and self-corrects has ten to twenty coupling points where a chat app had one. Keep tool definitions in a neutral schema and translate at the gateway; treat any framework feature that only works on one provider as a loan you will repay at migration time.

Layer 5 — Economic architecture

Routing is where dependency becomes money. The arithmetic is one-sided even at public list prices: frontier models cost on the order of ten times more per token than the small and open-weight models that handle classification, extraction, and summarization perfectly well. Send every request to a frontier model and you pay the premium rate for work a cheaper tier would do identically — so a tiered blend lands at a fraction of a frontier-everything bill, with the exact multiple set by your own traffic mix. Measure yours; don’t borrow someone else’s number. But the deeper value of a routing tier is optionality: once classification, extraction, and summarization run on interchangeable smaller models — including open-weight ones you could self-host — the share of your workload that is truly captive shrinks to the frontier-only remainder. Price your renewal leverage as: the percentage of token volume that could move within 30 days without eval regression.

The failover drill

An exit cost you have never tested is a fiction. The drill is simple and quarterly: repoint the gateway’s default policy at your designated fallback (a second provider or an open-weight deployment), run the full eval suite, and record three numbers — engineering hours spent, eval pass-rate delta, and the workload percentage that moved cleanly.

# gateway routing policy — failover drill
default_tier:
  primary:   frontier-a
  fallback:  open-weight-70b   # drill target
routes:
  classify|extract|summarize:  small-tier
  agentic|complex-reasoning:   default_tier
drill:
  cadence: quarterly
  pass_criteria: eval_delta <= 2%
  record: [eng_hours, eval_delta, pct_moved]

Those three numbers, tracked over time, are your exit cost — not an estimate, a measurement. In my experience the first drill is always humbling: the gap between the architecture diagram and the actual coupling is where the real dependency lives.

The point is not to leave

None of this is a plan to abandon your provider. It is the opposite of a plan: it is an option. An option you hold changes a renewal negotiation even if you never exercise it. An option you lack is a price you will accept.

Rent intelligence if you choose to. Measure the exit before it is load-bearing. And never rent the operating model.