Most AI Governance Is a Dashboard. Real AI Governance Is a Decision.

A model in production and a model that is governed are not the same claim. Most organisations only have evidence for the first one.

Executive Summary

Four failures sit underneath almost every AI programme audit finding: deployment decisions that were never formally made, human review processes that quietly eroded after go-live, business outcomes nobody re-confirmed after month one, and compliance obligations tracked by assumption rather than by record — and a monitoring dashboard, however good, answers none of them.

Monitoring Answers a Different Question

Ask most programme teams how they know an AI model in production is under control, and the answer is a dashboard. Accuracy holding steady. Latency within range. Drift within tolerance. All correct, all reassuring, and all beside the point.

A monitoring platform — MLflow, Azure ML, SageMaker, Vertex AI, Prometheus and Grafana, Arize, WhyLabs — tells you what the model is doing. It does not tell you whether the human review process designed at go-live is still being followed. Whether the business owner reviewed last month's outcomes. Whether the override mechanism is actually being used, or quietly abandoned because it slows people down. Whether the organisation's EU AI Act or DPDP Act obligations for this specific model are being actively tracked, or assumed to be someone else's job.

These are governance questions, not operational ones. They live in a different layer, and that layer does not instrument itself. It requires someone to ask it, on a schedule, and to write down the answer.

The Decision Nobody Formally Makes

Most AI models do not fail on go-live day because the model was wrong. They fail because nobody made a documented decision that the model, the infrastructure, the oversight mechanisms, and the business function around it were actually ready — and because that gap only becomes visible in an incident review, an audit, or a regulatory inspection, by which point it is expensive.

A production readiness decision should rest on five things holding at once: the model performing against a business-defined threshold on held-out data; the serving infrastructure tested under realistic load with monitoring, rollback, and disaster recovery confirmed; human review and override mechanisms designed and signed off, not assumed; the business unit formally accepting real outputs, not a summary; and the documentation and compliance record complete enough to survive scrutiny. Score any one of those honestly as absent, and everything else becomes theoretical.

Nineteen criteria, five weighted dimensions

• Model Performance Validation (30%) — accuracy against threshold, edge case and failure mode testing, bias assessment, stability across runs.
• Operational Infrastructure (25%) — load-tested serving infrastructure, monitoring and alerting confirmed live, retraining pipeline tested end-to-end, rollback tested, disaster recovery confirmed.
• Human Oversight and Controls (20%) — review process formally signed off, override mechanism operational and logged, named incident response owner, documented accountability.
• Business Readiness (15%) — formal business sign-off on real outputs, user training completed, adoption and outcome tracking in place.
• Documentation and Compliance (10%) — model card complete, data lineage documented, regulatory obligations confirmed or a remediation plan in place with a named owner and date.

Each criterion is scored 1 to 5 against a defined scale, from Not in Place to Optimised, and the weighted result produces one of three outcomes: Recommended for Deployment at 4.0 or above, Conditional Go between 3.0 and 3.9 with named gaps to close before go-live, or No-Go below 3.0.

A high average is not readiness. That is not a gate. It is a spreadsheet pretending to be a decision.

Which is why the scoring logic includes a second, independent trigger: any single criterion scored 1 blocks deployment outright, regardless of the weighted average. A strong 4.3 cannot absorb a missing rollback procedure or an oversight mechanism that was never designed. One critical absence should not be diluted into an acceptable number — it should stop the deployment and name exactly what is missing.

The Part That Actually Erodes — Governance After Go-Live

The gate is a point-in-time decision. What happens in month four is a different problem entirely, and it is the one most programmes have no instrument for at all. Review processes that were rigorous at launch get skipped under delivery pressure. Override mechanisms exist but nobody checks whether they are being used, or logged, or investigated when usage spikes. A retraining decision gets made informally in a chat thread instead of documented anywhere an auditor could find it.

None of this shows up on a monitoring dashboard. The model can be performing exactly as designed while every governance control around it has quietly stopped functioning.

Fifteen indicators, four categories, every month

• Oversight and Controls (GV1–GV4) — is the human review process still operating, is the override mechanism in use and logged, has the escalation path been tested, has any model change gone through the change governance process.
• Platform and Operations (GV5–GV8) — is the monitoring platform confirmed operational, were this month's alerts reviewed by a named owner, was a retraining decision made and documented if one was due, were incidents reviewed and closed.
• Business and Adoption (GV9–GV12) — did the business unit formally review outcomes, was adoption data reviewed, was user feedback captured, has the business owner confirmed satisfaction.
• Risk and Compliance (GV13–GV15) — were compliance obligations reviewed, is the risk register current, does the governance documentation reflect anything that changed this month.

Each indicator takes one of four answers — Confirmed, Partially Confirmed, Not Confirmed, or Not Applicable — mapped straight to green, amber, red, or grey. Not Applicable responses are excluded from the score entirely, which matters: a governance instrument that lets convenient non-answers inflate a health score is not measuring anything. The result is a Governance Health Score from 1.0 to 3.0, with three plain-language states — Healthy, Watch, or At Risk — and it is designed to be completed by a Programme Manager or AI Governance Lead directly, without needing an ML engineering background to answer a single question on it.

What This Replaces, and What It Deliberately Doesn't

This is not a case for building yet another monitoring layer. Organisations with a functioning MLOps platform already have accuracy, drift, latency, and cost covered — rebuilding that manually in a spreadsheet adds friction without adding governance value. The instrument described here sits above that platform, not beside it, and it stays out of the technical layer on purpose: no metric values are re-entered, no model internals are assessed, no attempt is made to replace observability tooling.

It is equally deliberate about what it is not. The compliance criteria that reference the EU AI Act and India's DPDP Act are structured prompts to act — confirm the obligation has been reviewed, name an owner, set a date — not a legal opinion. Binding compliance determinations still require qualified legal counsel. And it is scoped to one model per instance: a workbook shared across multiple models produces a trend line that means nothing and an evidence record nobody can rely on.

Who Actually Owns This

The two modules were built for different points in the accountability chain, and that is intentional. The Pre-Production Gate requires coordination — a Programme Manager pulling evidence from ML engineering, infrastructure, and the business owner to score all nineteen criteria honestly. The Monthly Governance Review does not. It is designed to be run end-to-end by a Programme Manager or AI Governance Lead with no ML background required, because governance oversight and model engineering are different skill sets, and conflating them is exactly how the oversight layer stops getting checked.

Management consultants run the Gate during client deployments instead of building a bespoke checklist from scratch. AI PMOs standardise it across a portfolio and use the Executive Dashboard as the recurring steering committee artefact — a five-minute agenda item instead of a scramble the day before the meeting. CIOs and engineering directors get a formal, evidenced recommendation to sign against, not a verbal assurance.

Where This Sits in the Programme

A governance gate is only as credible as the investment case it is checking against. The model being assessed here is the one that was funded on a specific business case, usually built and defended before a single line of the model existed. Viksya's AI ROI Calculator is where that investment case gets built; this toolkit is where the resulting deployment gets governed once it is live — the funding decision and the production decision are two separate instruments, and treating them as one is how weak business cases survive all the way to a steering committee without anyone re-testing them.

A funded model is not a governed model. Those are two different sign-offs, made by two different instruments, at two different points in the programme.

The Practical Case for Running Both Modules

None of this is complicated in mechanism — a scored Excel gate, a monthly dropdown review, an auto-calculated dashboard, no macros, no code, one file per model. The discipline is in the cadence: run the gate before every deployment and every material model change, and run the monthly review in the first five working days of the month, every month, for as long as the model stays in production. Organisations that skip the second half tend to discover the gap the same way — in an audit, after an incident, or when a regulator asks a question nobody in the room can answer with a document instead of a memory.

The AI Production Governance Toolkit is built to make sure that question always has a documented answer.

About the Toolkit

The AI Production Governance Toolkit (v2.0) is a Microsoft Excel workbook from Viksya: a 19-criterion pre-deployment gate across five weighted dimensions with an automatic No-Go trigger, and a 15-indicator monthly governance review across four categories, both rolled up into an auto-calculated Executive Dashboard with a 6-month trend view. Six tabs, no macros, no code, compatible with Excel 2016 and above. A full .pdf User Guide is included.

Available at https://viksya.com/ai-production-governance-toolkit