Configuring Azure OpenAI Private Link: Keeping AI Traffic Off the Public Internet

You open your CI/CD pipeline logs, and there it is: a curl call to your-resource.openai.azure.com — a public FQDN, resolving to a Microsoft-owned IP, carrying your internal service names and proprietary logic over the public internet. TLS encrypts the session body, but the endpoint itself is still exposed. Anyone watching the wire knows you are calling Azure OpenAI and roughly when.

This is not a flaw — it’s a default designed for onboarding, not production. Every Azure OpenAI resource ships with a public endpoint. Even if you add IP firewall rules, you are still routing sensitive prompts over public IP space. Private Link removes this public endpoint entirely, giving the AI service a private RFC 1918 address inside your Virtual Network.

1. How Azure Private Link Works for AI Services

Azure Private Link creates a Private Endpoint — a virtual network interface (NIC) with a private IP address — for your Azure OpenAI resource.

Before choosing Private Link, it helps to see what the alternatives actually give you:

Private Endpoints vs. Alternatives

A Service Endpoint restricts access to a VNet, but the service still has a public IP. Private Link gives it a private address (e.g., 10.0.1.5) from your own subnet. That IP is the only address the service answers on once you disable public access.

The table makes the choice look obvious, but the alternatives keep surviving in real estates for a reason: they’re faster to turn on. Service endpoints take one subnet setting. IP firewall rules take a list of egress IPs. Both reduce exposure, and both leave a public endpoint that a security reviewer will find on the first scan. On-premises callers can’t use a service endpoint at all, so teams behind ExpressRoute end up with an IP rule as well.

The call I’d make: Private Link with public network access disabled is the end state for any workload that sends internal or customer data to the model. IP firewall rules are acceptable as a temporary step during migration, with a written end date. What I’d rule out is service endpoints as a permanent answer. You pay for the complexity of VNet integration and still have to explain why the public endpoint exists.

2. VNet and Subnet Design

Before you provision the endpoint, you need to prepare your network. Set privateEndpointNetworkPolicies to Enabled on the subnet. The default, Disabled, means NSGs and route tables ignore traffic destined for the Private Endpoint, so any NSG rules you write for the AI endpoint do nothing.

Subnet Strategy

A Private Endpoint can share a subnet with other resources, but a dedicated /28 subnet (16 IPs) for “AI-Private-Links” simplifies NSG management and prevents CIDR exhaustion.

import * as azure_native from "@pulumi/azure-native";

// Define a secure subnet for AI Private Endpoints
const peSubnet = new azure_native.network.Subnet("snet-ai-pe", {
    subnetName: "snet-ai-pe",
    resourceGroupName: resourceGroupName,
    virtualNetworkName: vnet.name, // Don't also declare subnets inline on the VNet, or each deploy fights over them
    addressPrefix: "10.0.1.0/28",
    privateEndpointNetworkPolicies: "Enabled", // NSGs and UDRs apply to private endpoint traffic
});

In a hub-and-spoke topology, place the Private Endpoint in the hub’s shared-services VNet. This allows all peered spoke VNets to reach the AI service through a single entry point, centralizing DNS and security monitoring.

Hub-and-Spoke Networking Topology

3. Provisioning the Private Endpoint

Deploy Private Link from code, not the portal, so every environment gets the same endpoint. You must target the account group ID for Azure Cognitive Services.

const privateEndpoint = new azure_native.network.PrivateEndpoint("pe-openai-prod", {
    privateEndpointName: "pe-openai-prod",
    resourceGroupName: resourceGroupName,
    location: location,
    subnet: { id: peSubnet.id },
    privateLinkServiceConnections: [{
        name: "conn-openai",
        privateLinkServiceId: openAiResourceId, // Full ARM resource ID
        groupIds: ["account"], // Required for Azure AI / OpenAI
    }],
});

If your OpenAI resource is in a different subscription than the VNet, the connection enters a “Pending” state and requires manual approval in the Networking blade of the OpenAI resource.

4. DNS Configuration: The Split-Brain Pattern

Private Link creates the private IP, but your applications still use the FQDN your-resource.openai.azure.com. Without a DNS override, that FQDN still resolves to the public IP — and your traffic fails the moment you disable public access. The Private DNS step is where most implementations fail silently.

Split-Brain DNS Resolution

Private DNS Zones

Azure provides a specialized Private DNS Zone: privatelink.openai.azure.com. When you link your VNet to this zone, any query for an OpenAI resource returns the private IP. A PrivateDnsZoneGroup on the endpoint automates the A-record registration.

const dnsGroup = new azure_native.network.PrivateDnsZoneGroup("openai-dns-group", {
    privateDnsZoneGroupName: "default",
    resourceGroupName: resourceGroupName,
    privateEndpointName: privateEndpoint.name,
    privateDnsZoneConfigs: [{
        name: "openai-config",
        privateDnsZoneId: dnsZoneId, // ID of the privatelink.openai.azure.com zone
    }],
});

For on-premises developers using VPN or ExpressRoute, configure a conditional forwarder on your local DNS server pointing openai.azure.com to an Azure DNS Private Resolver in your hub VNet.

5. Disabling Public Network Access

Once you verify the Private Endpoint, disable public access. Do not perform this step first — you will lock yourself out of the resource until the private path is functional.

Set publicNetworkAccess to Disabled. Any request not originating from your Private Endpoint is rejected with a 403 or a connection refusal.

Enforcement at Scale

Assign the built-in Azure Policy “Cognitive Services accounts should disable public network access” (ID: 0725b4dd-7e76-479c-a735-68e7ee23d5ca) in Deny mode at your Management Group level. This blocks any future AI deployments that skip Private Link.

6. Routing CI/CD Traffic

Microsoft-hosted GitHub Actions runners live on the public internet. They cannot reach your VNet. If you disable public access, those runners fail immediately.

Self-Hosted Runners

Deploy self-hosted runners inside your VNet (or a peered VNet). They inherit the VNet’s DNS and resolve the OpenAI private IP automatically. Azure Container Instances (ACI) or Azure Container Apps work well for hosting these runners in a serverless fashion within your network boundary.

# GitHub Actions snippet for self-hosted runners in a VNet
jobs:
  ai-task:
    runs-on: [self-hosted, azure-vnet]
    steps:
      - name: Call Private OpenAI
        run: |
          # This FQDN now resolves to 10.x.x.x automatically
          curl https://your-resource.openai.azure.com/status

Verification and Testing

1. Internal Resolution: From a VM inside the VNet, run nslookup your-resource.openai.azure.com. It must return the private IP (e.g., 10.0.1.5).
2. External Refusal: From your local machine (without VPN), run the same nslookup. It may resolve to a public IP due to the public CNAME chain, but any curl request must return a 403 error.
3. Audit Logs: Check the “Networking” blade in the Azure portal; “Public network access” should be “Disabled.”

Best Practices

• Centralize DNS: Maintain one privatelink.openai.azure.com zone in your Hub subscription and link it to all spokes.
• Automate Records: Always use privateDnsZoneGroups to manage A-records. Manual records drift and cause outages during resource recreation.
• Subnet Policies: Set privateEndpointNetworkPolicies: 'Enabled'. With the default Disabled, your NSGs will not filter traffic to the AI endpoint.
• No Ping: Azure blocks ICMP to Private Endpoint NICs. Use tcping or curl to verify connectivity.

Next Steps:

• Read Mastering Zero Data Retention: The Guide to Modified Abuse Monitoring in Azure OpenAI to stop Microsoft retaining the prompts that now travel over your private network.
• Return to The Ultimate Guide to Secure AI DevOps on Azure to see where network isolation sits in the full blueprint.

Sources

• Microsoft Learn: Configure Azure AI services virtual networks
• Microsoft Learn: Private DNS Zone for Cognitive Services
• Microsoft Learn: Azure Private Link Pricing
• GitHub: Self-hosted runners in a VNet