Your AI Governance Program Has a Blind Spot, and It Isn’t in Any of the Systems You’re Governing
Executive Summary. Most AI governance programs are built to answer one question well: is this system allowed to go live, and does it stay compliant once it does. That question gets asked of every model, every agent, every use case on the register — and answered thoroughly. It is not the same question as “what does our entire governed estate actually run on, and what happens to all of it on the day that changes.” An organization can have a mature, well-staffed governance program that evaluates every deployed system individually and still have no answer for what happens if the provider underneath a third of those systems changes its pricing, its terms, or its availability overnight. That isn’t a gap in execution. It’s a gap in what the program was ever built to look at.
Governance Answers “Is This System Compliant.” It Doesn’t Ask “Compliant on What”
Every mature AI governance program we’ve seen asks a consistent set of questions of a system before and after it goes live: was this approved, does it meet the relevant risk tier’s obligations, is someone accountable for what it does, is it still compliant as regulation or usage evolves. These are the right questions, and most organizations now ask them reasonably well — that discipline has genuinely improved over the past two years.
What none of those questions ask is what the system is actually built on, and whether that foundation is something the organization actually chose to depend on or something it backed into one integration at a time. A model API, a specific provider’s agent framework, a particular vendor’s fine-tuning pipeline — these choices get made at the point of fastest delivery, by whichever team is shipping that quarter’s use case, almost never by anyone looking at the portfolio as a whole. Each individual choice is defensible. The aggregate is a dependency structure nobody designed and almost nobody can describe without pulling together a list no single document currently holds.
The Risk That Doesn’t Show Up Until the Day It Matters
Dependency concentration is a strange kind of risk, because it’s genuinely invisible under normal conditions. A governance program checking each system individually will find every one of them compliant, every one of them performing, every one of them properly owned — right up until the thing underneath several of them at once changes, and the organization discovers how much of its AI estate was quietly resting on one provider’s continued cooperation.
The change doesn’t have to be dramatic to matter. A pricing model shifts and the economics of three use cases that were approved on different assumptions stop making sense simultaneously. A provider deprecates a model version and every system built against its specific behavior needs re-validation at once, not on a schedule the organization controls. A provider’s terms of service change in a way that affects data handling, and what was a single contract review becomes a review of everywhere that data handling pattern was reused. None of these show up in a governance check that evaluates one system at a time, because none of them are about any single system. They’re about the shape of the dependency across all of them, which is a property of the portfolio, not the system.
Why This Keeps Happening Even in Disciplined Organizations
This isn’t a failure of rigor. It’s a structural blind spot that follows directly from how governance programs get built. A governance function is almost always organized around systems — a register of use cases, each with its own risk tier, owner, and compliance status. That structure is correct for the questions it was built to answer. But a register organized by system has no natural row for “everything that depends on provider X,” because provider dependency cuts across systems rather than living inside any one of them. Finding it requires a different query against the same data, one that almost nobody runs because the register was never designed to answer it.
The result is an organization that can produce, with genuine confidence, a complete and accurate answer to “is every one of our AI systems individually compliant,” while having no answer at all to “how exposed are we, in aggregate, to any single provider’s decisions.” Both can be true at once, and usually are, in exactly the organizations whose individual-system governance is best.
What a Dependency View Actually Requires
Three things distinguish an organization that has an answer to this from one that doesn’t, and none of them require rebuilding the governance program that already works.
A cross-system inventory of what each system actually depends on — not just “approved,” but the specific model, API, framework, or pipeline each system runs on, maintained as its own view across the whole portfolio rather than buried inside each system’s individual record. This is the piece most organizations are missing entirely, not because it’s hard to build, but because nobody’s governance structure asked for it.
A concentration threshold, decided in advance. At what share of the governed estate depending on one provider does that dependency itself become something the organization treats as a risk, with its own owner and its own mitigation plan — not an abstract policy statement, but a specific number someone agreed to before it mattered, the same way a financial portfolio sets a concentration limit before a single position gets too large to hedge.
A named owner for the dependency, separate from the owners of the individual systems. Each system already has an owner accountable for that system’s compliance. Almost no organization has a separate owner accountable for the portfolio’s exposure to any single provider — which means when a provider-level change does happen, the first conversation is about who should have been watching for it, instead of what to do now that it’s here.
A Short Illustration
Picture an organization with twelve AI systems in production, each individually governed, each passing every review on schedule. Eight of them, built by different teams over eighteen months for different reasons, all run on fine-tuned variants of the same provider’s model family — not because anyone decided to standardize on it, but because it was the fastest path for each team at the time they built. No single system’s governance record would ever surface this, because no single system’s record is supposed to know what seven other systems are also running on. Only a portfolio-level dependency view would show that two-thirds of the governed estate shares one point of failure, and only a named owner for that specific risk would be positioned to do anything about it before the provider — not any of the twelve systems — is the thing that changes.
Governance That Checks Deployments Is Necessary. It Was Never Sufficient.
None of this is an argument against the governance work organizations have already built. Checking whether each system is compliant, owned, and properly risk-tiered is real work, and most organizations do it better than they did two years ago. The argument is narrower: that discipline answers a question about each system in isolation, and the newer, quieter risk sits one layer underneath — in what all those individually compliant systems collectively depend on, and whether anyone is accountable for that dependency as its own thing. Our own AI Vendor Governance Scorecard exists for exactly that layer: scoring dependency concentration across governance dimensions so the question has an owner and a number, not just a hope that it never comes up. The renewal cycle is where that scoring becomes a concrete decision: see What to Check Before Renewing an AI Vendor Contract for 2027 for the specific things to check before the terms lock in for another year.
The Question Worth Adding to the Next Governance Review
Most governance reviews already ask, system by system, “is this compliant.” Before the next one, it’s worth adding a different question, asked once about the whole portfolio rather than once per system: if our largest AI provider changed its terms, its pricing, or its availability tomorrow, how much of what we’ve built would need to change with it, and who would be the first to find out? If the honest answer is “we’d find out the same way everyone else does,” that’s the gap this piece is describing, and it has nothing to do with how well any individual system is governed.