Landing Zone Architecture
for AI Workloads

A prescriptive guide to designing Azure Landing Zones that host AI services securely, at scale, and aligned with the Cloud Adoption Framework (CAF). Covers management groups, subscriptions, networking, identity, governance, and platform services for enterprise AI.

Based on Azure CAF Enterprise-Ready
5 Layers
Architecture Depth
7 Patterns
Supported Topologies
Zero Trust
Security Model
CAF Aligned
Cloud Adoption Fwk

What is an Azure Landing Zone?

An Azure Landing Zone is a pre-provisioned, governed environment where workloads are deployed. It provides the foundational building blocks — networking, identity, governance, security, and management — so application teams can focus on building rather than plumbing.

For AI workloads, the landing zone must address unique requirements: high-bandwidth data pipelines, GPU compute isolation, model governance, Private Endpoint density, and OpenAI quota management.

Platform Landing Zone

Hosts shared services: networking, identity, DNS, monitoring, security tools. Managed by the platform team.

Application Landing Zone

Hosts workload-specific resources: AI services, compute, storage, app code. Owned by workload teams.

Sandbox

Unmanaged experimentation zone with no connectivity to production. For AI prototyping and hackathons.

Why AI Workloads Need Specialized Landing Zones

RequirementStandard WorkloadAI Workload
ComputeGeneral-purpose VMs, App ServiceGPU clusters (NC/ND series), Databricks VNet-injected, managed endpoints
Network2-3 subnets, basic NSGs5-8 subnets, /22 for Databricks, PE-dense /24 subnets, UDR for Firewall
Private Endpoints2-3 PEs6-12 PEs (OpenAI, Search, ML, Storage Ã-N, KV, ACR)
DNS1-2 Private DNS Zones5-8 zones (privatelink.openai, search, blob, dfs, vaultcore, ml, acr)
DataSingle storage accountMulti-account medallion (raw/curated/serving), feature stores, model registries
QuotavCPU quotasOpenAI TPM/RPM, GPU quotas, Databricks DBU, AI Search units
GovernanceCost tags, RBACModel lineage, responsible AI, content filtering, data classification
EgressNAT GatewayAzure Firewall with TLS inspection, threat intel, application rules

CAF Management Group Hierarchy

graph TB
    ROOT["Tenant Root Group"]
    ROOT --> PLATFORM["Platform"]
    ROOT --> LANDING["Landing Zones"]
    ROOT --> SANDBOX["Sandbox"]
    ROOT --> DECOM["Decommissioned"]
    PLATFORM --> MGMT["Management<br/>Log Analytics, Automation<br/>Defender for Cloud"]
    PLATFORM --> CONN["Connectivity<br/>Hub VNet, Firewall<br/>DNS, ExpressRoute"]
    PLATFORM --> ID["Identity<br/>Entra ID, Domain Ctrlrs<br/>Conditional Access"]
    LANDING --> CORP["Corp (Connected)<br/>VNet-peered workloads"]
    LANDING --> ONLINE["Online (Internet)<br/>Public-facing apps"]
    LANDING --> AI["AI & ML<br/>AI Workloads"]
    AI --> AIPROD["AI-Prod<br/>Production AI services"]
    AI --> AIDEV["AI-NonProd<br/>Dev/Test AI services"]
    CORP --> CORPSUB["App Subscriptions"]
    SANDBOX --> SBX["Sandbox Subs<br/>No connectivity"]
    style AI fill:#0d6efd,color:#fff,stroke:#0d6efd
    style AIPROD fill:#198754,color:#fff
    style AIDEV fill:#ffc107,color:#000
    style PLATFORM fill:#6c757d,color:#fff
    style CONN fill:#0dcaf0,color:#000
              

The AI & ML management group sits under Landing Zones and contains dedicated subscriptions for AI workloads. This separation enables:

  • AI-specific Azure Policies — enforce private endpoints on cognitive services, require content filtering on OpenAI, mandate diagnostic settings
  • Quota isolation — OpenAI TPM limits, GPU quotas, and Databricks DBU scoped to AI subscriptions
  • Cost boundaries — AI compute costs (GPU, OpenAI tokens) tracked separately from general workloads
  • Responsible AI guardrails — policies to enforce content safety, model evaluation, and data classification

Conceptual Architecture — AI Landing Zone

graph TB
    subgraph MGMT_SUB["Management Subscription"]
        LAW["Log Analytics<br/>Workspace"]
        AUTO["Automation<br/>Account"]
        DEF["Defender for<br/>Cloud"]
    end
    subgraph CONN_SUB["Connectivity Subscription"]
        subgraph HubVNet["Hub VNet 10.0.0.0/16"]
            FW["Azure Firewall<br/>Premium"]
            BASTION["Azure Bastion"]
            GW["VPN / ER<br/>Gateway"]
            DNS_RES["DNS Private<br/>Resolver"]
        end
        DNS_ZONES["Private DNS<br/>Zones (8+)"]
    end
    subgraph AI_SUB["AI Landing Zone Subscription"]
        subgraph AIVNet["AI Spoke VNet 10.10.0.0/16"]
            subgraph AI_SVC["AI Services Subnet /24"]
                AOAI["Azure OpenAI"]
                AIS["AI Search"]
                AML["Azure ML"]
            end
            subgraph COMPUTE["Compute Subnet /20"]
                AKS["AKS / App Svc"]
                DBX["Databricks<br/>VNet-injected"]
            end
            subgraph DATA_SUB["Data Subnet /24"]
                ADLS["ADLS Gen2<br/>(x multiple)"]
                ACR["Container<br/>Registry"]
            end
            subgraph PE_SUB["PE Subnet /24"]
                PE["Private<br/>Endpoints (6-12)"]
            end
        end
        KV["Key Vault"]
        AI_MON["App Insights"]
    end
    GW -->|"ExpressRoute /<br/>VPN"| OnPrem["On-Premises"]
    HubVNet -->|"VNet Peering"| AIVNet
    PE --> AOAI
    PE --> AIS
    PE --> AML
    PE --> ADLS
    AI_SVC --> FW
    COMPUTE --> FW
    AI_MON --> LAW
    DEF --> AI_SUB
    DNS_ZONES --> PE_SUB
              

Conceptual: Hub-Spoke topology with dedicated AI Landing Zone subscription peered to the Connectivity Hub

Network Topology

AI Landing Zones use a hub-spoke network model. The Connectivity subscription owns the Hub VNet (Firewall, DNS, Gateway). Each AI workload spoke peers into the Hub for controlled egress and DNS resolution.

graph LR
    subgraph Hub["Hub VNet 10.0.0.0/16"]
        FW["Firewall<br/>10.0.0.0/26"]
        BASTION["Bastion<br/>10.0.1.0/26"]
        GW["Gateway<br/>10.0.2.0/27"]
        DNS["DNS Resolver<br/>10.0.3.0/28"]
    end
    subgraph Spoke1["AI Spoke 1 — 10.10.0.0/16"]
        S1_AI["AI Services /24"]
        S1_PE["Private Endpoints /24"]
        S1_COMP["Compute /20"]
        S1_DATA["Data /24"]
    end
    subgraph Spoke2["AI Spoke 2 — 10.20.0.0/16"]
        S2_AI["AI Services /24"]
        S2_PE["Private Endpoints /24"]
        S2_COMP["Compute /20"]
    end
    Hub -->|"Peering"| Spoke1
    Hub -->|"Peering"| Spoke2
    S1_COMP -->|"UDR 0.0.0.0/0"| FW
    S2_COMP -->|"UDR 0.0.0.0/0"| FW
              
Egress Control
  • All spoke traffic routed through Azure Firewall via UDR
  • Application rules for OpenAI API, Docker Hub, PyPI
  • Threat intelligence in Deny mode
  • TLS inspection for data exfiltration prevention
  • DNS Proxy enabled on Firewall for private zones
DNS Architecture
  • Private DNS Zones hosted in Connectivity subscription
  • Zones linked to Hub VNet for resolution
  • DNS Private Resolver for on-premises forwarding
  • Auto-registration for spoke VNets disabled (PE-only)
  • 8+ zones for AI: openai, search, blob, dfs, vaultcore, azureml, notebooks, acr

Subnet Design for AI Spoke VNet

A typical AI spoke VNet (10.10.0.0/16 = 65,536 IPs) needs careful subnet planning. Databricks alone consumes 2Ã- /22 subnets (2,048 IPs). Private Endpoints and AKS are the next largest consumers.

SubnetCIDRIPsPurposeDelegation / Notes
snet-ai-pe10.10.0.0/24251Private Endpoints for AI servicesNo delegation; NSG: VNet-only inbound
snet-data-pe10.10.1.0/24251Private Endpoints for data servicesNo delegation; NSG: VNet-only inbound
snet-aks10.10.16.0/204,091AKS node pool (system + user + GPU)No delegation; Azure CNI overlay or kubenet
snet-dbx-public10.10.32.0/221,019Databricks public subnetMicrosoft.Databricks/workspaces delegation
snet-dbx-private10.10.36.0/221,019Databricks private subnetMicrosoft.Databricks/workspaces delegation
snet-app10.10.40.0/24251App Service VNet IntegrationMicrosoft.Web/serverFarms delegation
snet-ml-compute10.10.41.0/24251Azure ML compute instances/clustersNo delegation; NSG required
snet-mgmt10.10.42.0/2659Management / jumpbox / BastionNo delegation
Total Allocated~7,19211% of /16 used — room for growth
IP Planning Tip: Reserve at least a /16 per AI spoke. Databricks VNet injection alone requires 2Ã- /22 (2,048 IPs). With AKS overlay networking, pods get IPs from a virtual CIDR, reducing node subnet pressure.

Identity & Access Control

Managed Identities

Every service uses System-Assigned Managed Identity for zero-credential authentication. No API keys stored in app settings.

ServiceIdentity Grants
AKS / App ServiceCognitive Services OpenAI User, Search Index Data Reader
Azure MLStorage Blob Data Contributor, Key Vault Secrets User
AI SearchStorage Blob Data Reader (indexer), Cognitive Services OpenAI User
DatabricksStorage Blob Data Contributor (on ADLS accounts)
RBAC Model

Follow least-privilege with custom roles where built-in roles are too broad.

PersonaScopeRole
Platform TeamAI MGOwner / Contributor
AI Team LeadAI SubscriptionContributor
Data ScientistResource GroupML Operator + Blob Data Contributor
App DeveloperResource GroupCognitive Services OpenAI User
SecOpsAI MGSecurity Reader + Log Analytics Reader
Key Vault Standards
  • RBAC authorization (no access policies)
  • Purge protection enabled
  • Soft delete: 90-day retention
  • Private endpoint access only
  • Diagnostic settings → Log Analytics
Conditional Access
  • MFA required for all AI resource management
  • Named locations: allow only corp IP ranges
  • Block legacy authentication protocols
  • Session controls on Azure portal for AI MG
  • PIM for elevated roles (Just-in-Time)

Governance & Azure Policy

Azure Policies assigned at the AI & ML management group enforce guardrails that prevent misconfigurations before they reach production.

PolicyEffectPurpose
Cognitive Services should use private linkDenyPrevent OpenAI / AI Search with public endpoints
Storage accounts should use private linkDenyForce ADLS Gen2 behind private endpoints
Key Vault should use RBAC authorizationDenyPrevent access-policy-based Key Vaults
Azure OpenAI should have content filteringAuditEnsure responsible AI guardrails
Diagnostic settings should be enabledDeployIfNotExistsAuto-configure diagnostics to Log Analytics
Deploy Defender for AI servicesDeployIfNotExistsEnable threat detection on cognitive services
AKS clusters should use Azure RBACDenyPrevent local Kubernetes accounts
Allowed locations: Canada CentralDenyData residency compliance
Require resource tags (CostCenter, Owner, Environment)DenyCost tracking and accountability
Storage TLS minimum version 1.2DenyEncryption in transit
Required Tags
CostCenter Owner Environment DataClassification AIWorkload ModelName ResponsibleAIReview

Data Platform Layer

AI workloads are data-intensive. The landing zone must accommodate multiple storage accounts, feature stores, model registries, and data pipelines — all with private access.

graph LR
    subgraph Ingestion
        EH["Event Hubs"]
        ADF["Data Factory"]
        STREAM["Stream<br/>Analytics"]
    end
    subgraph Storage["ADLS Gen2 Medallion"]
        RAW["Raw Zone<br/>landing / raw / archive"]
        CUR["Curated Zone<br/>silver / gold / delta"]
        SRV["Serving Zone<br/>embeddings / vectors"]
    end
    subgraph Processing
        DBX["Databricks<br/>ETL + Feature Eng"]
        AML2["Azure ML<br/>Training Pipelines"]
    end
    subgraph AI_Layer["AI Ready"]
        AIS2["AI Search<br/>Vector Index"]
        AOAI2["Azure OpenAI<br/>Embeddings + LLM"]
    end
    EH --> RAW
    ADF --> RAW
    RAW --> DBX
    DBX --> CUR
    CUR --> DBX
    DBX --> SRV
    SRV --> AIS2
    SRV --> AOAI2
    AML2 --> CUR
    STREAM --> RAW
              
Raw Zone

Immutable landing area. Data arrives here from Event Hubs, Data Factory, or direct upload.

  • ADLS Gen2, HNS enabled
  • Write-once, append-only
  • Lifecycle: archive after 90 days
  • Containers: landing/, raw/, archive/
Curated Zone

Cleansed, transformed, business-ready data. Delta Lake provides ACID transactions.

  • ADLS Gen2, HNS enabled
  • Delta Lake format
  • Silver (cleaned) + Gold (aggregated)
  • Feature store tables
Serving Zone

AI-ready data: embeddings, vector indexes, model artifacts, and documents for RAG.

  • ADLS Gen2, HNS enabled
  • Embeddings + vector indexes
  • Model artifacts + packages
  • Feeds AI Search and OpenAI

AI Services Layer

ServiceRole in LZDeployment ModelKey LimitsPrivate Endpoint Zone
Azure OpenAILLM inference + embeddingsS0, GlobalStandard models30 deployments, 240K TPM (GPT-4o)privatelink.openai.azure.com
Azure AI SearchVector + semantic searchStandard S1 (25GB), S2 (100GB)50 indexes, 12 replicas maxprivatelink.search.windows.net
Azure Machine LearningModel registry, training, endpointsWorkspace (Free) + compute800 endpoints, 200 compute clustersprivatelink.api.azureml.ms
Azure DatabricksETL, feature eng, trainingPremium, VNet-injectedWorkspace-level quotasprivatelink.databricks.azure.net
Container RegistryML model container imagesPremium (geo-replicated)500 repos, 500GB storageprivatelink.azurecr.io
API ManagementGateway for AI APIsStandard v2 / Consumption4,000 req/sec (Std v2)privatelink.azure-api.net

Observability & Monitoring

Application Insights
  • Workspace-based (linked to Log Analytics)
  • Request tracing on OpenAI calls
  • Dependency tracking for AI Search
  • Custom metrics: token usage, latency P95
  • Smart detection for anomalies
Log Analytics
  • Centralized in Management subscription
  • 90-day hot retention (365 for compliance)
  • Diagnostic settings on all AI services
  • KQL queries for OpenAI usage analysis
  • Cost: ~$2.30/GB ingestion
Alerting
  • OpenAI 429 rate-limiting alerts
  • AI Search latency P95 > threshold
  • AKS node CPU/memory pressure
  • Firewall threat intel hits
  • Key Vault access anomalies

Pattern-to-Landing-Zone Mapping

Each of the 7 Azure AI Deployment Patterns maps to a specific landing zone topology. Use this table to choose the right foundation:

PatternLZ TypeSubscriptionsVNet ModelFirewallBest For
1. Centralized HubPlatform LZ (Hub)1 shared AI subHub VNet + spoke peeringOptionalSingle-team, cost-efficientDetails
2. Decentralized SpokeApp LZ (per spoke)N subs (1/team)Isolated VNetsStandardData isolation, autonomyDetails
3. Hybrid Central-SpokePlatform + App LZ1 hub + N spoke subsHub-spoke peeringStandardShared models + spoke RAGDetails
4. AI Factory / MLOpsDedicated AI LZ1 AI Factory subSingle VNet, 5+ subnetsOptionalFull ML lifecycleDetails
5. ServerlessOnline LZ / Sandbox1 subNo VNet (public PaaS)NonePoC, hackathons, MVPDetails
6. Secure PrivateCorp LZ (isolated)1 dedicated subFully private VNetPremiumRegulated (HIPAA, PCI)Details
7. Data-Centric LakehouseDedicated AI LZ1 AI + data subSingle VNet, 6+ subnetsOptionalData platform + AIDetails

Landing Zone Decision Tree

graph TD
    START["Start: New AI Workload"] --> Q1{"Need private<br/>network isolation?"}
    Q1 -->|"No"| Q2{"Production or<br/>PoC / internal?"}
    Q2 -->|"PoC"| P5["Pattern 5<br/>Serverless AI<br/>Online LZ / Sandbox"]
    Q2 -->|"Production"| Q3{"Multiple teams<br/>sharing AI?"}
    Q1 -->|"Yes"| Q4{"Zero Trust /<br/>regulated data?"}
    Q4 -->|"Yes"| P6["Pattern 6<br/>Secure Private AI<br/>Corp LZ"]
    Q4 -->|"No"| Q5{"Data-intensive<br/>with Databricks?"}
    Q5 -->|"Yes"| Q6{"Full ML lifecycle<br/>needed?"}
    Q6 -->|"Yes"| P4["Pattern 4<br/>AI Factory / MLOps<br/>Dedicated AI LZ"]
    Q6 -->|"No"| P7["Pattern 7<br/>Data-Centric Lakehouse<br/>Dedicated AI LZ"]
    Q5 -->|"No"| Q3
    Q3 -->|"Yes"| Q7{"Teams need<br/>full isolation?"}
    Q7 -->|"Yes"| P2["Pattern 2<br/>Decentralized Spoke<br/>App LZ per team"]
    Q7 -->|"No"| Q8{"Teams need own<br/>search / RAG?"}
    Q8 -->|"Yes"| P3["Pattern 3<br/>Hybrid Central-Spoke<br/>Platform + App LZ"]
    Q8 -->|"No"| P1["Pattern 1<br/>Centralized Hub<br/>Platform LZ"]
    Q3 -->|"No"| P1
    style P1 fill:#0d6efd,color:#fff
    style P2 fill:#0dcaf0,color:#000
    style P3 fill:#198754,color:#fff
    style P4 fill:#6f42c1,color:#fff
    style P5 fill:#fd7e14,color:#fff
    style P6 fill:#dc3545,color:#fff
    style P7 fill:#20c997,color:#fff
              

AI Landing Zone Deployment Checklist

Foundation
Networking
Security & Identity
AI Services & Data

References

All Patterns Landing Zone Architecture Guide Pattern 1: Centralized Hub