Most AI projects don’t fail because the model was wrong. They fail because nobody could say who approved the agent, what data it used, what it cost, or whether it still works as it did at launch. A proof of concept can skip those questions. A production service can’t.
We use a one-page framework to make sure none of them gets missed. It has nine layers. Each one answers a single question, and together they cover an AI capability from the idea to the invoice. It’s written for Microsoft-based estates, but the structure works on any platform.
Read it top-down for control and bottom-up for evidence:
In one line: governance directs, controls enforce, observability evidences, FinOps optimises, value delivers.
This is where decisions are made. It covers the architecture review board, the AI risk council, business strategy and use cases, policies, investment and prioritisation, lifecycle governance, and accountability.
The key idea is that governance is a process to follow, not a meeting to pass. Every agent follows the same gated lifecycle: idea, prototype, sandbox, pilot, production, then ongoing operation. Each gate has a clear approver and clear evidence. Use cases are scored on business impact, technical feasibility, compliance risk and scalability, so priorities aren’t set by whoever speaks loudest.
For the architect: if you can’t name the approver for your agent’s next gate, you’re not ready for it.
This is defence in depth, applied to AI. Identity comes first: single sign-on, role-based access, just-in-time privilege and conditional access. Then come private networking, keys held by the organisation, and threat protection.
What’s new is identity for agents. An agent is a security principal like any other. It needs its own identity, a named human owner, least-privilege permissions, and access reviews like any privileged account. When an agent acts on a user’s behalf, the audit trail should still show which human it was acting for.
For the architect: treat every agent as a new privileged account, because that’s what it is.
AI is only as trustworthy as the data it reads. This layer finds out where sensitive data lives, labels it, stops it leaking, keeps records for the right length of time, and makes them available for legal holds and investigations.
Two ideas matter most:
Ownership is deliberately spread out. Privacy, records, legal, security and data teams each own their own part. No single team can credibly own all of it.
For the architect: your design should answer “what did this AI touch?” without anyone having to work it out by hand.
This layer controls how people and agents reach models and tools. The main pattern is an AI gateway in front of every model, tool and grounding call the organisation owns. In one place, the gateway:
The layer also covers prompt security (defending against prompt injection), governance of connectors, and model risk management, meaning an approved list of models with owners and retirement dates.
Know your boundary. Built-in assistants such as Microsoft 365 Copilot don’t pass through your gateway. Govern them through data controls instead.
For the architect: if a call to a model you own doesn’t pass through the gateway, you can’t limit it, measure it or explain it.
These are the platforms that do the work: productivity copilots, low-code and code-first agent platforms, data and analytics agents, AI services, and custom integrations. They draw on connectors to your content and systems.
A useful way to think about use is ask, build, delegate:
The more autonomy, the stronger the human approval step needs to be.
Underneath all three sits a context layer: shared, permission-aware access to documents, business data and knowledge. Without it, every agent team rebuilds its own data plumbing, and each copy is another thing to secure and keep up to date.
For the architect: design the context once and reuse it across agents. Don’t let each agent keep a private copy of the truth.
Normal monitoring asks whether the request succeeded and how fast. For AI, that’s not enough. A fast, successful answer can still be wrong, ungrounded or unsafe. So this layer watches two things:
We send each kind of signal to the tool that answers its question:
| Question | Where the signal goes |
|---|---|
| How did the system behave? | Application monitoring |
| What data did it touch? | Data governance tooling |
| Is anyone attacking it? | The security operations centre |
| What agents do we have? | A single agent registry |
Use one open standard (OpenTelemetry) for telemetry, so a trace looks the same whichever platform produced it.
For the architect: observability is a prerequisite for cost control, not a separate workstream. You can’t manage spend you can’t see.
AI cost scales with usage. Each call costs almost nothing, but volume adds up quickly, and a badly designed agent can cost five to ten times more than a well-designed one.
This layer tags spend, sets budgets and alerts, attributes cost to individual agents and teams, and drives optimisation. There are usually two billing streams:
Neither admin console shows both, so bring them together in one cost model. The standard control is the same everywhere: a budget, an alert at 80% of it, and a hard stop at the limit.
For the architect: the cheapest tokens are the ones you design out. Keep prompts tight, limit retrieved context, cap response length and use smaller models where they’re good enough.
This is the landing zone every workload inherits:
Get this right once, as infrastructure as code, and every new agent starts compliant instead of fixing it later.
For the architect: if a new AI workload needs a special network or policy exception, question the design before approving the exception.
AI output varies, so you can’t test it once and move on. This layer keeps curated test sets: golden question-and-answer pairs, adversarial “red team” prompts and robustness checks. They all run through one evaluation harness on every change to a prompt, model, tool or knowledge source.
Decide in advance which results are hard stops and which are watch items. Feed failures from production back into the test sets. Model retirements are the change teams most often forget: when a model version is retired, the replacement needs a full re-test, not a quiet swap.
For the architect: treat evaluation results as release evidence, the same as unit tests are for code.
The layers aren’t new ideas. Identity, data protection, monitoring and cost control are familiar disciplines. The value comes from putting them on one page, so that for any AI capability you can ask nine questions and expect nine clear answers:
If any answer is “we’re not sure”, you’ve found your next piece of design work.
