How to Use AutoGen with Azure OpenAI (Enterprise Security)
Connect Microsoft AutoGen to Azure OpenAI for enterprise-grade AI agents. Step-by-step setup with private endpoints, OAI_CONFIG_LIST, and deployment config.
Get more content like this on Telegram!
Daily AI tips, notes & resources — free
Enterprise teams moving from OpenAI's public API to Azure OpenAI get something valuable: data residency guarantees, private network access, compliance certifications (SOC 2, ISO 27001, HIPAA), and content filtering controls. Microsoft AutoGen supports this migration cleanly, but the configuration has quirks that trip up almost every team the first time through.
This guide walks you through the full setup — from creating your Azure OpenAI resource to running a multi-agent AutoGen system behind a private endpoint. Every configuration detail that matters for production is here.
Why Azure OpenAI for Enterprise AutoGen
Before getting into code, the motivation matters. Here's what Azure OpenAI provides that the public API doesn't:
- Data residency — your prompts and completions stay within a specified Azure region
- Virtual network isolation — private endpoints mean your traffic never traverses the public internet
- Managed identity auth — use Azure AD instead of API keys, eliminating secret rotation overhead
- Content filtering — configurable at the deployment level with audit logging
- Compliance scope — Azure OpenAI is in scope for HIPAA, FedRAMP, and other regulated frameworks
- SLA-backed availability — enterprise SLA with uptime commitments
- Cost management — integrate with Azure Cost Management for chargeback and budget alerts
For teams building agents that handle customer data, legal documents, financial records, or medical information, these aren't optional features. They're requirements.
Setting Up Your Azure OpenAI Resource
Start in the Azure portal. Create an Azure OpenAI resource with these settings:
Resource name: your-company-openai
Region: East US 2 (or your preferred region)
Pricing tier: Standard S0
Once created, deploy a model:
Deployment name: gpt-4o-enterprise
Model: gpt-4o (or gpt-4, gpt-4-turbo)
Version: Latest
Tokens per minute: 100K (adjust to your needs)
Content filter: Default (customize after testing)
The deployment name is critical — this is what you'll use in AutoGen's config, not "gpt-4o". Azure uses deployment names as namespaced identifiers so you can run multiple deployments of the same model with different configurations.
Installing AutoGen
pip install pyautogen azure-identity
azure-identity handles managed identity authentication — essential for production environments where you want to avoid storing API keys in environment variables or config files.
OAI_CONFIG_LIST: The Core Configuration
AutoGen uses OAI_CONFIG_LIST as its standard configuration format. For Azure, each entry needs four fields beyond the standard API key:
import autogen
import os
# Option 1: API key authentication (development/testing)
config_list_azure = [
{
"model": "gpt-4o-enterprise", # Your deployment name
"api_type": "azure",
"base_url": "https://your-company-openai.openai.azure.com/",
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
"api_version": "2024-02-01" # Use latest stable version
}
]
# Option 2: Multiple deployments with fallback
config_list_azure_multi = [
{
"model": "gpt-4o-enterprise",
"api_type": "azure",
"base_url": "https://your-company-openai.openai.azure.com/",
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
"api_version": "2024-02-01",
"tags": ["gpt-4o", "primary"]
},
{
"model": "gpt-4-turbo-enterprise", # Fallback deployment
"api_type": "azure",
"base_url": "https://your-company-openai.openai.azure.com/",
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
"api_version": "2024-02-01",
"tags": ["gpt-4-turbo", "fallback"]
}
]
# Load from JSON file (recommended for production)
# config_list_azure = autogen.config_list_from_json(
# "OAI_CONFIG_LIST.json",
# filter_dict={"api_type": ["azure"]}
# )
The JSON file format (preferred for keeping secrets out of code):
[
{
"model": "gpt-4o-enterprise",
"api_type": "azure",
"base_url": "https://your-company-openai.openai.azure.com/",
"api_key": "your-azure-api-key-here",
"api_version": "2024-02-01"
}
]
Store this file outside your repository and load it via path. Never commit API keys to version control.
Basic AutoGen Agents with Azure Config
import autogen
import os
config_list = autogen.config_list_from_json(
"OAI_CONFIG_LIST.json",
filter_dict={"api_type": ["azure"]}
)
llm_config = {
"config_list": config_list,
"temperature": 0.1,
"timeout": 120,
"cache_seed": 42, # Deterministic caching for cost management
"max_tokens": 4096
}
# Define agents
assistant = autogen.AssistantAgent(
name="Enterprise_Assistant",
llm_config=llm_config,
system_message="""You are an enterprise AI assistant. Follow these rules:
1. Do not reference any external URLs or services without approval
2. Summarize confidential data without retaining specific values in responses
3. Flag any requests involving PII or financial data for human review
"""
)
user_proxy = autogen.UserProxyAgent(
name="Enterprise_User",
human_input_mode="TERMINATE",
max_consecutive_auto_reply=5,
code_execution_config={
"work_dir": "enterprise_workspace",
"use_docker": True, # Isolate code execution in production
"timeout": 60
},
is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE")
)
# Start conversation
user_proxy.initiate_chat(
assistant,
message="Analyze our Q4 sales data and generate a summary report."
)
Managed Identity Authentication
For production deployments, replace API keys with Azure Managed Identity. This eliminates secret management entirely — the VM or container running AutoGen authenticates to Azure AD automatically.
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.core.credentials import TokenCredential
import openai
def get_azure_token() -> str:
"""Get a token for Azure OpenAI using managed identity."""
credential = DefaultAzureCredential()
token = credential.get_token("https://cognitiveservices.azure.com/.default")
return token.token
# For AutoGen, we need to provide a custom LLM client
class AzureManagedIdentityClient:
"""Custom OpenAI client using managed identity."""
def __init__(self, endpoint: str, deployment: str, api_version: str):
self.endpoint = endpoint
self.deployment = deployment
self.api_version = api_version
self.credential = ManagedIdentityCredential()
def create_client(self) -> openai.AzureOpenAI:
token = self.credential.get_token(
"https://cognitiveservices.azure.com/.default"
)
return openai.AzureOpenAI(
azure_endpoint=self.endpoint,
azure_deployment=self.deployment,
api_version=self.api_version,
azure_ad_token=token.token
)
# Use with AutoGen via custom model client
managed_client = AzureManagedIdentityClient(
endpoint="https://your-company-openai.openai.azure.com/",
deployment="gpt-4o-enterprise",
api_version="2024-02-01"
)
Private Endpoint Configuration
Private endpoints route traffic through your Azure VNet rather than the public internet. This is the configuration that satisfies most enterprise network security requirements.
Azure setup steps:
# 1. Create private endpoint (Azure CLI)
az network private-endpoint create \
--name openai-private-endpoint \
--resource-group your-rg \
--vnet-name your-vnet \
--subnet private-endpoints-subnet \
--private-connection-resource-id "/subscriptions/SUBSCRIPTION_ID/resourceGroups/your-rg/providers/Microsoft.CognitiveServices/accounts/your-company-openai" \
--group-id account \
--connection-name openai-private-connection
# 2. Create private DNS zone
az network private-dns zone create \
--resource-group your-rg \
--name "privatelink.openai.azure.com"
# 3. Link DNS zone to VNet
az network private-dns link vnet create \
--resource-group your-rg \
--zone-name "privatelink.openai.azure.com" \
--name openai-dns-link \
--virtual-network your-vnet \
--registration-enabled false
# 4. Create DNS record for private endpoint
az network private-dns record-set a add-record \
--resource-group your-rg \
--zone-name "privatelink.openai.azure.com" \
--record-set-name "your-company-openai" \
--ipv4-address PRIVATE_ENDPOINT_IP
AutoGen config with private endpoint:
# The base_url changes to use the private DNS name
config_list_private = [
{
"model": "gpt-4o-enterprise",
"api_type": "azure",
# Private endpoint URL — same hostname, resolves to private IP via DNS
"base_url": "https://your-company-openai.openai.azure.com/",
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
"api_version": "2024-02-01",
# Optional: disable SSL verification for self-signed internal certs
# (not recommended for production)
# "verify_ssl": False
}
]
From a machine on the VNet, the DNS resolution will return the private IP. From outside the VNet, the hostname won't resolve at all — which is the security behavior you want.
Multi-Agent Enterprise Pipeline
Here's a complete enterprise pattern — a GroupChat setup where multiple specialists collaborate on a task, all using Azure OpenAI:
import autogen
config_list = autogen.config_list_from_json("OAI_CONFIG_LIST.json")
llm_config = {
"config_list": config_list,
"temperature": 0.1,
"timeout": 180
}
# Define specialist agents
data_analyst = autogen.AssistantAgent(
name="Data_Analyst",
llm_config=llm_config,
system_message="You analyze data and identify trends. Always cite your sources."
)
security_reviewer = autogen.AssistantAgent(
name="Security_Reviewer",
llm_config=llm_config,
system_message="""You review outputs for security and compliance issues.
Flag any PII, financial data, or confidential information before it's included
in deliverables. Respond with APPROVED or FLAG: [reason]."""
)
report_writer = autogen.AssistantAgent(
name="Report_Writer",
llm_config=llm_config,
system_message="You write professional reports from approved analysis data."
)
# Human proxy with enterprise termination logic
enterprise_proxy = autogen.UserProxyAgent(
name="Enterprise_Proxy",
human_input_mode="NEVER",
max_consecutive_auto_reply=15,
code_execution_config={
"work_dir": "enterprise_output",
"use_docker": True
},
is_termination_msg=lambda msg: "REPORT_COMPLETE" in msg.get("content", "")
)
# Group chat configuration
groupchat = autogen.GroupChat(
agents=[enterprise_proxy, data_analyst, security_reviewer, report_writer],
messages=[],
max_round=20,
speaker_selection_method="auto"
)
manager = autogen.GroupChatManager(
groupchat=groupchat,
llm_config=llm_config
)
# Launch
enterprise_proxy.initiate_chat(
manager,
message="""Task: Generate a quarterly performance report.
Data source: /data/q4_metrics.csv
Requirements:
- Executive summary (3 bullet points)
- Department breakdown table
- Risk flags for any metrics below threshold
- All outputs must pass security review before inclusion
End with REPORT_COMPLETE when finished."""
)
Configuration Comparison: Dev vs Staging vs Production
| Setting | Development | Staging | Production |
|---|---|---|---|
| Auth method | API key | API key or MI | Managed Identity |
| Network | Public endpoint | Public or VNet | Private endpoint only |
| Content filtering | Default | Strict | Strict + audit log |
| Code execution | Local | Docker | Docker + isolated VNet |
| Cache seed | 42 (deterministic) | None | None |
use_docker | False | True | True |
| Timeout | 60s | 120s | 180s |
| Logging | Verbose | Structured | Structured + SIEM |
| API version | Latest | Pinned | Pinned + tested |
Always pin the api_version in staging and production. Azure OpenAI API versions change behavior, and auto-upgrading in production has broken agent workflows for teams that weren't paying attention.
Cost Management with Azure
One advantage of Azure OpenAI is integration with Azure Cost Management. Add tags to your resource for chargeback:
# Add metadata to requests for cost tracking
llm_config_with_tracking = {
"config_list": config_list,
"temperature": 0.1,
"extra_headers": {
"X-Department": "engineering",
"X-Project": "customer-support-agent",
"X-Environment": "production"
}
}
Combined with Azure's token quota management, this gives finance teams visibility into which agents are driving costs.
For more on agent patterns that work well in production, the Deploy AI model to production guide covers infrastructure concerns that apply directly to enterprise AutoGen deployments. The AI agents and the future of work piece gives broader context for the enterprise adoption patterns driving this migration.
If you're building agents that need long-term memory alongside this Azure setup, the Vector database guide covers Azure Cognitive Search as a vector store option that keeps all data within the Azure boundary.
The managed AutoGen pattern — private endpoints, managed identity, content filtering, audit logs — is what separates a proof of concept from a system that can be approved by enterprise security teams. The extra configuration work upfront saves enormous friction during security review.
Frequently Asked Questions
Can AutoGen work with Azure OpenAI instead of the public OpenAI API?
Yes. AutoGen natively supports Azure OpenAI through its OAI_CONFIG_LIST configuration. You provide the Azure endpoint, API key, deployment name, and API version, and AutoGen routes all LLM calls through your Azure resource.
What is OAI_CONFIG_LIST in AutoGen?
OAI_CONFIG_LIST is AutoGen's configuration format for specifying one or more LLM backends. It's a JSON array where each entry defines model, API type, base URL, API key, and API version. AutoGen iterates through the list and uses the first available model for each agent.
How do I set up private endpoints for AutoGen with Azure?
Create an Azure OpenAI resource with network restrictions enabled, configure a private endpoint in your VNet, update your DNS to resolve the OpenAI endpoint to the private IP, and set the base_url in OAI_CONFIG_LIST to the private endpoint URL. AutoGen itself requires no changes.
Does Azure OpenAI with AutoGen support content filtering?
Yes. Azure's built-in content filtering applies automatically to all requests. You can configure custom filters in the Azure portal at the resource or deployment level. This is a key enterprise advantage over the public OpenAI API.
What Azure OpenAI models work with AutoGen?
AutoGen works with any model available in your Azure deployment — GPT-4o, GPT-4, GPT-4 Turbo, and GPT-3.5-Turbo. The model name in OAI_CONFIG_LIST should match your deployment name, not the base model name, since Azure uses custom deployment identifiers.
Frequently Asked Questions
AiTechWorlds Team
✓ Verified WriterThe AiTechWorlds team is passionate about AI, technology, and education. We create high-quality, research-backed content to help you learn, grow, and succeed in the modern digital world.
Related Articles
5 AutoGen Agent Roles (Assistant, UserProxy, CodeExecutor)
Understand the 5 core AutoGen agent types — AssistantAgent, UserProxyAgent, CodeExecutorAgent, and more — with code examples and a comparison table for each role.
How to Deploy AutoGen Agents as APIs with FastAPI (2026)
Learn to serve AutoGen multi-agent systems as production REST APIs using FastAPI with async endpoints and real-time streaming responses.
Build a Code Debugging Agent with AutoGen (Auto-Fix PRs)
Build an AutoGen agent that reviews code, analyzes PR diffs, suggests fixes, and automates code quality improvements with a full working implementation.
How to Use AutoGen with Code Interpreter (Execute Python)
Learn how to set up AutoGen's code interpreter with LocalCommandLineCodeExecutor and DockerCommandLineCodeExecutor to safely execute Python in agent workflows.