Azure OpenAI (Internal Assistants) governance guidance

Reviewed 3 October 2026. Aicura is not affiliated with Microsoft.

About Azure OpenAI (Internal Assistants)

Azure OpenAI (Internal Assistants) gives an organisation access to OpenAI and other AI models inside its own Azure environment, so its teams can build assistants for staff and customers on the organisation’s own information. Its reach grows as teams build. A simple assistant drafts and answers questions, a grounded assistant searches internal documents and agents go further by calling tools, connecting to other systems and acting on the organisation’s behalf. Because the organisation builds these assistants itself, it also holds most of the levers that make them safe, and good governance is what lets it pull them consistently as the number of assistants grows.

Where governance makes the difference

• Information reaching wider than intended. Assistants grounded on internal documents and data can surface that information to anyone who can use the assistant. Without deliberate choices about what each assistant may draw on and who may use it, information can reach people it was never meant for.
• Assistants and agents multiplying without oversight. Any team with access can deploy an AI and build an assistant on it. Without a register and a review step, the organisation can end up with assistants nobody owns, running on AI nobody approved and connected to tools nobody assessed.
• Content steering the assistant. An assistant follows instructions in what it reads, so documents, emails or web pages it retrieves can influence what it says or which tools it calls.

---

For the executive team

The organisation builds it, so the organisation owns the outcome

Azure OpenAI (Internal Assistants) is a platform for building assistants rather than a finished product. Each assistant the organisation builds is its own system, with its own purpose, its own data and its own users. That makes accountability a matter of internal choice, and each assistant needs a named owner and a clear purpose before it reaches staff or customers.

Customer-facing assistants need a higher bar

An assistant that answers customers speaks for the organisation. Its answers, its tone and the information it draws on carry the same weight as any other customer communication. Hold customer-facing assistants to a higher standard of review and testing than internal ones, and agree in advance who decides when one is ready.

Oversight needs to keep pace with change

New assistants, new connections to internal systems and newer versions of the underlying AI arrive continually. The vendor also retires older AI versions on a published timetable, and some deployments move to newer versions automatically. Oversight works best as an ongoing practice that notices these changes rather than a one-off approval at launch.

Questions worth putting to management

• How many assistants and agents are built on Azure OpenAI (Internal Assistants), and who owns each one?
• Which assistants are used by customers, and how were they tested before release?
• What information can each assistant draw on, and who decided that?
• How would the organisation know if an assistant gave a harmful answer or an agent took an action it should not have?
• How does the organisation learn about changes to the underlying AI, and who decides how to respond?
• If a regulator, customer or insurer asked how these assistants are governed, what would be shown to them?

What good reporting looks like

Expect regular reporting on the assistants and agents in use and who owns them, which ones serve customers, the main risks and how they are changing, spend against budget, changes to the underlying AI and how they were handled, any incidents or near misses and the significant decisions taken, with who made them and why.

How Aicura can help

• One register of every assistant and agent built on Azure OpenAI (Internal Assistants), each with an owner, an accountable executive and a technical contact.
• Board reports drafted from the governance record, covering use, risks, spend, AI changes, incidents and the decisions taken.
• A public trust centre where customers and partners can see which AI assistants the organisation runs and how it governs them.

---

For risk, compliance and assurance

Set where each assistant sits in the risk appetite

Decide which classes of information assistants may draw on and which decisions their output may inform. Internal drafting help and a customer-facing assistant sit at very different points on the scale. Output that feeds decisions about people, such as customer outcomes, eligibility or employment, needs review by someone who knows the subject before it is relied on.

Assess each assistant on its own terms

Assess each assistant separately, covering its purpose, its users, the data it draws on, the tools it can call, whether its output affects people and who reviews it. A shared platform assessment is useful groundwork, but it does not replace an assessment of what each assistant actually does.

Reassess when anything changes

A new data source, a new tool, a wider audience or a change to the underlying AI all move an assistant away from what was assessed. The vendor publishes a lifecycle for each AI version and can move some deployments to a newer version automatically. Each of these changes needs to trigger a fresh look rather than wait for the annual cycle.

Make the data and location decisions explicitly

Decide where prompts and responses may be processed. The vendor offers deployment choices that keep processing within a single Azure geography, within a wider data zone or across any region, and the right answer depends on the information involved. Check how the vendor’s terms handle prompts, responses and any stored data, including conversation history and files kept for assistants, then decide how long the organisation keeps its own records of assistant activity and who may access them.

Confirm which protections apply

Check which of the organisation’s Azure subscriptions and deployments carry the commitments the organisation is relying on, and confirm the vendor’s content safety settings in place for each deployment are the ones agreed. Staff using consumer AI tools on personal accounts sit outside those arrangements, so find that use and bring it into the approved setup.

Set limits and name who acts

Agree the limits that matter, such as unusual spend, harmful content being blocked repeatedly, sensitive information appearing in prompts or an agent calling tools outside its purpose. Name the person who acts when one is crossed and what they are empowered to do.

Be ready to show governance to others

• Policy. Maintain an AI use policy that covers building assistants on Azure OpenAI (Internal Assistants), including who may deploy AI, what data assistants may use, how customer-facing assistants are approved and how output affecting people is reviewed. Tie each commitment to the control that carries it out.
• Accountability. Record committee and management decisions with their reasons when they are made. Have assistant owners confirm periodically that their assistants still match what was approved.
• Operating evidence. Keep records that controls actually ran, such as completed assessments, approvals before release, test results, blocked requests and interventions.
• External scrutiny. Prepare for internal and external audit, certification against standards such as ISO/IEC 42001, regulator enquiries and insurer questions about AI controls.
• Customers and partners. Customers using an assistant may ask whether they are talking to AI, how their information is handled and where it is processed. Have clear answers ready and supported by evidence.
• Incidents. Be able to reconstruct which assistant version, which AI version, which data, which prompt and which controls were in place at the time.

How Aicura can help

• Risk and impact assessments for each assistant against recognised frameworks, run again when its data, tools, audience or underlying AI changes.
• The organisation’s own position on each AI version held alongside the vendor’s lifecycle, with notice to every affected assistant when a version is changed or retired and the response recorded.
• Limits on activity and spend, with responses the organisation chooses, from alerting the owner to pausing an agent.
• Every approval, decision and control action kept as evidence that can be verified without relying on the organisation’s word.

---

For technology and security

Discover every assistant and deployment

Inventory every Azure OpenAI (Internal Assistants) resource, AI deployment and agent across all subscriptions, along with the applications that call them. Record the AI version, deployment type, data sources, tools and identity behind each assistant. Look beyond the central platform team, because any team with the right Azure access can deploy an AI and start building.

Prepare the data each assistant draws on

Grounded assistants retrieve from indexes, file stores and connected systems. Decide what goes into each index, keep access controls on source documents meaningful at retrieval time and keep the most sensitive content out of assistants that do not need it. Where an agent acts on behalf of a user, prefer passing the user’s own identity so the agent sees only what that person can see.

Put deployments and agents under change control

AI deployments are Azure resources, and agents can be defined through the SDK or API from a deployment pipeline, with versions kept for each agent. Keep deployments and agent definitions in source control and run a preflight check on every release, so a new assistant, AI version, tool, data source or deployment type is checked against agreed rules before it goes live. Use platform policy to block deployment types or regions the organisation has not approved.

Control who can deploy AI and enable tools

Limit who can create AI deployments, build and publish agents and connect tools. Give each agent its own identity with only the access its purpose needs, and review every tool connection and its credentials before it is switched on.

Govern traffic in the request path

Applications call Azure OpenAI (Internal Assistants) through a documented API, so their traffic can pass through an AI gateway on the way to the organisation’s own endpoint and the gateway can read it. Use the gateway to redact personal information before it leaves, screen inbound content for injected instructions, confirm each assistant uses an approved system prompt, restrict agents to approved tools and AI versions, cap spend per assistant and stop an agent when needed. Put the vendor’s own gateway capabilities and content filters to work alongside it rather than relying on either alone.

Keep the endpoints private

Bring deployments onto private networking where the organisation’s requirements call for it, require calls to come through the gateway and use managed identities in place of shared keys. That way traffic that bypasses the agreed path is refused rather than merely noticed.

Feed everything into monitoring

Bring platform logs, gateway records, agent traces and usage and cost data into monitoring. Apply the limits risk has agreed and set responses ranging from alerting the owner to tightening an agent’s restrictions or stopping it. Subscribe to the vendor’s lifecycle notices and check AI version status programmatically, so a scheduled upgrade or retirement reaches the assistant’s owner well before it takes effect.

Test against the live deployment

Red-team and evaluate each assistant before release and on a schedule afterwards, against the deployed assistant with its real data, tools and AI version. Test again before and after any AI version change, and watch for changes in behaviour between tests.

How Aicura can help

• One inventory of every assistant and agent, with the AI versions, tools and identities behind each one, plus discovery from Azure inventories and code repositories of deployments not yet recorded.
• Preflight checks in the deployment pipeline before an assistant, AI version, tool or deployment type goes live.
• A gateway in the request path with guardrails derived from the agreed controls, covering redaction, injection screening, approved system prompts, tool and model limits, budgets and the ability to stop an agent.
• A collector inside the network for platform logs and agent telemetry, plus scheduled red-teaming and evals against the live deployment with drift watched between runs.

---

Getting lasting value from Azure OpenAI (Internal Assistants)

Azure OpenAI (Internal Assistants) pays off when every assistant built on it is governed as an ongoing practice rather than approved once at launch. Three things matter whatever the role. The first is knowing every assistant and agent in use, what it can reach and who is accountable for it. The second is keeping assessments, controls and the request path in step with assistants and AI versions that keep changing. The third is being able to show, at any time, how decisions were made and that controls held. Organisations that get these right can build more assistants and put them in front of more people with confidence rather than caution.

---

Sources

• Microsoft Azure, Azure OpenAI in Foundry Models
• Microsoft Learn, Microsoft Foundry documentation
• Microsoft Learn, Understanding deployment types in Microsoft Foundry Models
• Microsoft Learn, Foundry Models lifecycle and support policy
• Microsoft Learn, What is Microsoft Foundry Agent Service?
• Microsoft Learn, What is Microsoft Foundry Control Plane?
• Microsoft Learn, Enforce Token Limits for Models
• Microsoft Learn, AI gateway capabilities in Azure API Management