Keyless AI: Using Entra ID Managed Identities for Azure OpenAI

You followed the quickstart. You grabbed the key from the portal, pasted it into a .env file, and your app worked. Now that key lives on your laptop, in your CI/CD secrets, probably in a Slack message from six months ago, and quite possibly in a git log you haven’t checked. It does not expire. It grants full access to every deployment in your OpenAI resource. You aren’t doing it wrong — you just followed the path Microsoft laid out for onboarding, not production.

Managed Identities remove the credential from the equation entirely. Your compute resource authenticates using its Azure identity. No static secret is created, stored, or rotated. The 32-character string simply stops existing.

1. Why API Keys Fail at Enterprise Scale

API keys are bearer tokens: possession equals authorization. They carry no identity context, no MFA requirement, no device compliance check. When you need to rotate one, every consuming application breaks simultaneously — which is why your team has been using “Key 1” for eight months without touching it.

Specific Risks for AI

Azure OpenAI keys grant access to every deployment within a resource. An attacker with a key can call any model, not just the one intended for a specific application. Because AI workloads often run at high token-per-minute (TPM) quotas, a leaked key can generate massive cost spikes and trigger data exfiltration before you notice the breach.

2. Managed Identities: Concepts and Types

Managed Identities work by calling the Azure Instance Metadata Service (IMDS) at 169.254.169.254. Your compute resource requests a short-lived OAuth 2.0 token, which the Azure SDK handles automatically.

User-Assigned vs. System-Assigned

System-assigned: Tied to the resource lifecycle. Delete the VM, and the identity is deleted with it.
User-assigned: Created independently and attached to one or more resources.

For production AI workloads, use User-Assigned Managed Identities. They decouple the identity lifecycle from the infrastructure, so you can pre-provision RBAC assignments and maintain identity stability across resource redeployments.

3. RBAC Roles for Azure OpenAI

Identity is only half the solution. You also need to grant the correct permissions at the correct scope.

The OpenAI User Role

For standard application workloads, assign the Cognitive Services OpenAI User role (ID: 5e0bd9bd-7b93-4f28-af87-19fc36ad61bd). This role allows inference calls — chat, completions, embeddings — but does not allow managing deployments or fine-tuning.

Scope your assignments at the resource level. Granting access at the Resource Group or Subscription level gives too much lateral access. Assign the role specifically to the Azure OpenAI account itself.

# Get the principal ID of your User-Assigned Managed Identity
MSI_ID=$(az identity show -n my-mi -g my-rg --query principalId -otsv)

# Get the resource ID of your OpenAI account
OPENAI_ID=$(az cognitiveservices account show -n my-openai -g my-rg --query id -otsv)

# Assign the role
az role assignment create \
    --role "Cognitive Services OpenAI User" \
    --assignee "$MSI_ID" \
    --scope "$OPENAI_ID"

4. DefaultAzureCredential: One Pattern Everywhere

The azure-identity library provides DefaultAzureCredential, a unified authentication pattern that works across all environments without changing a line of code.

DefaultAzureCredential Resolution Chain

The Python Implementation (v1.x)

First, ensure you have the necessary packages: pip install azure-identity openai. In production, DefaultAzureCredential uses the Managed Identity. In local development, it falls through to your Azure CLI login (az login). You get the same code path in both environments.

import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI

# Create the token provider for the Cognitive Services audience
token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), 
    "https://cognitiveservices.azure.com/.default"
)

# Initialize the client without an api_key
client = AzureOpenAI(
    azure_ad_token_provider=token_provider,
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    api_version="2024-02-15-preview"
)

5. CI/CD Federation: Secret-less GitHub Actions

GitHub Actions supports OIDC federation, letting your workflows authenticate to Azure without a client secret. The setup is two steps. Both are easy to miss.

GitHub Actions OIDC Federation Flow

Configuring OIDC

Create a User-Assigned Managed Identity.
Add a Federated Credential to the identity pointing to your GitHub repository and branch (e.g., repo:my-org/my-repo:ref:refs/heads/main).
Use the azure/login action in your workflow with the client-id, tenant-id, and subscription-id.

The azure/login action injects the necessary environment variables so subsequent Python or PowerShell steps using DefaultAzureCredential work automatically. You no longer need AZURE_OPENAI_API_KEY in your repository secrets.

6. Secretless Local Development

Your developers should not have API keys on their local machines either. Assign the Cognitive Services OpenAI User role to individual developer accounts or an Entra ID security group.

Once assigned, developers run az login once. DefaultAzureCredential picks up their personal identity, and they call the AI service without ever seeing a key. To avoid quota exhaustion on production resources, point local development to a dedicated “Dev” OpenAI account.

Hands-On Example: Migrating a Pipeline

To migrate a GitHub Actions pipeline from keys to identity:

Identity: Create a User-Assigned identity and grant it the OpenAI User role on your resource.
Federation: Create the federated credential for your GitHub repo.
Code: Update your script to use azure_ad_token_provider and DefaultAzureCredential.
Workflow: Update the YAML to use azure/login@v2 with OIDC.
Cleanup: Delete the AZURE_OPENAI_API_KEY secret from GitHub and rotate the key in the Azure portal to a random value before disabling it.

Key Takeaways

Keys are Bearer Tokens: They provide no identity context and are high-risk assets.
Managed Identity is Auditable: Every call is logged in Azure Monitor with the caller’s object ID.
DAC is the Standard: DefaultAzureCredential provides a single auth path for local, CI/CD, and production environments.
Scope is Critical: Always assign roles at the resource level, never higher.
Federation Removes Secrets: OIDC for GitHub and AKS eliminates the need for client secrets or API keys in deployment configurations.