Each application has its own complete AI stack deployed inside its own landing zone. No dependency on a central AI hub — every workload is fully self-contained with its own Azure OpenAI, storage, search, and compute.
The Decentralized Spoke AI pattern gives each workload team complete autonomy over their AI stack. Every spoke contains its own Azure OpenAI instance, AI Search, storage, Key Vault, and AKS cluster — fully isolated from other workloads.
This is the pattern of choice when workloads have different compliance requirements, need dedicated OpenAI quotas, or require data isolation boundaries (PII, PCI-DSS, PHI).
| GPT-4o | 20K TPM |
| text-embedding-3-large | 80K TPM |
| System Pool | D4s_v5 Ã- 3 |
| GPU Pool | NC6s_v3 (optional) |
graph LR
subgraph Mgmt[" Shared Management
(Centralized)"]
FW[" Azure
Firewall
Control all
outbound traffic"]
MON[" Log Analytics
Monitor metrics
from all spokes"]
end
subgraph SP1[" SPOKE 1
(Team A)
VNet 10.10.0.0/16"]
AKS1["AKS Cluster
D4s_v5 x 3 nodes
Runs AI apps"]
AOAI1["Azure OpenAI
S0 Tier
LLM models"]
AIS1["AI Search
Basic Tier
Vector DB"]
ADLS1["Data Lake
Gen2 Storage
Datasets"]
end
subgraph SP2[" SPOKE 2
(Team B)
VNet 10.20.0.0/16"]
AKS2["AKS Cluster
D4s_v5 x 3 nodes
Runs AI apps"]
AOAI2["Azure OpenAI
S0 Tier
LLM models"]
AIS2["AI Search
Basic Tier
Vector DB"]
ADLS2["Data Lake
Gen2 Storage
Datasets"]
end
subgraph SP3[" SPOKE 3
(Team C)
VNet 10.30.0.0/16"]
AKS3["AKS Cluster
D4s_v5 x 3 nodes
Runs AI apps"]
AOAI3["Azure OpenAI
S0 Tier
LLM models"]
AIS3["AI Search
Basic Tier
Vector DB"]
ADLS3["Data Lake
Gen2 Storage
Datasets"]
end
FW -->|Allow| AKS1
FW -->|Allow| AKS2
FW -->|Allow| AKS3
AKS1 -->|Call API| AOAI1
AKS1 -->|Search| AIS1
AIS1 -->|Read/Write| ADLS1
AOAI1 -->|Fetch| ADLS1
AKS2 -->|Call API| AOAI2
AKS2 -->|Search| AIS2
AIS2 -->|Read/Write| ADLS2
AOAI2 -->|Fetch| ADLS2
AKS3 -->|Call API| AOAI3
AKS3 -->|Search| AIS3
AIS3 -->|Read/Write| ADLS3
AOAI3 -->|Fetch| ADLS3
AKS1 -.->|Logs & Metrics| MON
AOAI1 -.->|Performance| MON
AKS2 -.->|Logs & Metrics| MON
AOAI2 -.->|Performance| MON
AKS3 -.->|Logs & Metrics| MON
AOAI3 -.->|Performance| MON
| # | Service | Qty | SKU / Tier | Purpose | Cost/Unit | Monthly Total |
|---|---|---|---|---|---|---|
| 1 | Azure OpenAI | 3 | S0 | Dedicated LLM per workload team | ~$1,100 | ~$3,300 |
| 2 | Azure AI Search | 3 | Basic | Workload-specific vector search | $73.73 | $221.19 |
| 3 | ADLS Gen2 | 3 | Standard LRS, HNS | Workload datasets & embeddings | ~$30 | ~$90 |
| 4 | Azure Key Vault | 3 | Standard | Per-spoke secrets management | ~$5 | ~$15 |
| 5 | AKS | 3 | D4s_v5 Ã- 3 nodes | Container compute per workload | ~$420 | ~$1,260 |
| 6 | Log Analytics | 1 | PerGB2018 | Centralized monitoring (shared) | ~$35 | ~$35 |
| 7 | Application Insights | 3 | Workspace-based | Per-spoke APM telemetry | Incl. | Incl. |
| 8 | Azure Firewall | 1 | Standard | Centralized egress control | $912 | $912 |
| 9 | VNets | 3 | /16, 3 subnets each | Network isolation per workload | Free | Free |
| 10 | Private DNS Zones | 12 | Global (4/spoke) | Name resolution | $0.50 | ~$6 |
| 11 | Private Endpoints | 12 | 4 per spoke | Private access per service | $7.30 | ~$88 |
| 12 | NSGs | 3 | Default rules | Per-spoke micro-segmentation | Free | Free |
| Estimated Total (3 Spokes, Moderate Production) | ~$7,360/mo | |||||
Costs scale linearly: each additional spoke adds ~$2,150/mo (OpenAI + Search + AKS + Storage + KV + PE).
Dedicated LLM per workload with 20K TPM GPT-4o and 80K TPM embeddings. Full quota isolation ensures no cross-workload throttling.
Kubernetes compute for each workload with system pool and optional GPU pool (NC6s_v3) for inference. Azure RBAC and Calico network policies.
Workload-specific vector and keyword search. Each spoke maintains independent indexes, avoiding cross-workload data exposure.
Centralized egress control for all spokes. Application and network rules control outbound traffic. Threat intelligence in alert mode.
Per-workload data lake with hierarchical namespace. Stores datasets, embeddings, and model artifacts in isolated storage boundary.
Centralized Log Analytics workspace collects diagnostics from all spokes. Per-spoke App Insights instances for APM and request tracing.
Each workload has its own encryption boundary. Perfect for healthcare, finance, and regulated industries.
Teams own their entire AI stack end-to-end with no cross-team dependencies.
Each spoke gets its own 20K TPM, no noisy-neighbor issues when one workload spikes.
Each workload can enforce unique compliance controls independently.
| Constraint | Mitigation |
|---|---|
| Cost duplication — each spoke pays for full stack | Use consumption-based SKUs where possible; right-size per workload |
| Harder to standardize across workloads | Use shared Bicep modules with org-wide defaults |
| No shared model registry | Consider adding a lightweight registry spoke or use AML |
| OpenAI quota per subscription limits | Spread workloads across subscriptions if needed |
| Operational overhead (N clusters to manage) | GitOps + centralized monitoring in Log Analytics |
Generate a complete deployment spec sheet with GitHub Actions workflow, Bicep file structure, and prerequisite checklist.
Generate Deployment Spec Sheet