Pattern 3: Hybrid Central-Spoke AI

Overview

The Hybrid Central-Spoke pattern is a best-of-both-worlds approach. Expensive, shared AI services (Azure OpenAI, model registry) live in the Hub, while application-specific components (AI Search for RAG, document storage, compute) live in each spoke.

This is ideal for organizations with many small AI-powered apps that share base models but need their own search indexes and data.

Key Characteristics

Shared Base Models — Hub owns GPT-4o (50K TPM) + embeddings (120K TPM)
Spoke RAG Components — Each spoke has its own AI Search + ADLS for documents
VNet Peering — Bidirectional hub-to-spoke connectivity
AML Model Registry — Centralized model versioning and staged deployment
Cost Efficient — Share expensive AOAI; duplicate cheap spoke services

Hub OpenAI Models

GPT-4o	50K TPM
text-embedding-3-large	120K TPM

Hub Storage

models datasets artifacts

Spoke Storage

documents embeddings cache

Architecture Diagram

graph TB
    subgraph Hub["Hub VNet — Central AI Platform"]
        AOAI["Azure OpenAI<br/>GPT-4o 50K TPM"]
        AML["Azure ML<br/>Model Registry"]
        HKV["Key Vault"]
        HADLS["ADLS Gen2<br/>Models & Datasets"]
        HMON["Log Analytics"]
    end
    subgraph Spoke["Spoke VNet — App Workload"]
        AIS["Azure AI Search<br/>Vector DB / RAG"]
        APP["App Service or AKS<br/>Application Compute"]
        SADLS["ADLS Gen2<br/>Documents"]
    end
    subgraph Security["Network Layer"]
        PE1["Hub Private Endpoints"]
        PE2["Spoke Private Endpoints"]
        PEER["VNet Peering<br/>Bidirectional"]
    end
    APP -->|"VNet Peering"| AOAI
    APP --> AIS
    AIS --> SADLS
    AML --> HADLS
    AML --> HKV
    AOAI --> HMON
    AIS --> HMON
    APP --> HMON
    PE1 --- AOAI
    PE1 --- HKV
    PE2 --- AIS
    PE2 --- SADLS
    PEER --- Hub
    PEER --- Spoke

Bill of Materials

#	Service	Location	SKU / Tier	Purpose	Monthly Cost
1	Azure OpenAI	Hub	S0	Shared base models (GPT-4o + embeddings)	~$2,750
2	Azure Machine Learning	Hub	Workspace	Model registry, training pipelines	$0*
3	ADLS Gen2 (Hub)	Hub	Standard LRS, HNS	Models, datasets, artifacts	~$35
4	Key Vault	Hub	Standard	Centralized secrets	~$5
5	Azure AI Search	Spoke	Standard S1	Per-spoke vector DB and RAG	$245.28
6	ADLS Gen2 (Spoke)	Spoke	Standard LRS, HNS	Documents, embeddings, cache	~$25
7	App Service or AKS	Spoke	P1v3 / D4s_v5	Application compute	~$140
8	AML Compute (Spoke)	Spoke	2Ã- D4s v3	Spoke-level inference endpoints	$280.32
9	Azure Firewall	Shared	Standard	Egress control	$912
10	Log Analytics	Shared	PerGB2018	Centralized logging	~$35
11	Hub VNet	Hub	/16, 3 subnets	Hub network backbone	Free
12	Spoke VNet	Spoke	/16, 3 subnets	Application network	Free
13	VNet Peering	—	Bidirectional	Hub↔Spoke connectivity	~$10/TB
14	Private DNS Zones	—	5 zones (Global)	Name resolution	~$2.50
15	Private Endpoints	Hub + Spoke	6 endpoints	Private PaaS access	~$44
Estimated Range (Moderate Production)					$15,000 – $22,000/mo

* AML workspace free; compute billed separately. Additional spokes add ~$700/mo (Search + ADLS + App + PE).

Pricing Disclaimer: Estimates start from a $10,000–$15,000/mo base (shared Azure OpenAI, AI Search, AML, networking, monitoring) +$5,000/spoke (per-spoke AI Search, storage, compute, Private Endpoints). Actual costs will vary based on Azure region, reserved instance commitments, Enterprise Agreement discounts, consumption volume, and selected SKU tiers. Use the Azure Pricing Calculator for your specific configuration. These figures are provided for planning purposes only and do not constitute a pricing guarantee.

Service Breakdown

Azure OpenAI (Hub)

S0~$2,750/mo

Central LLM serving all spokes. GPT-4o at 50K TPM for chat completions, text-embedding-3-large at 120K TPM for vector generation. Per-spoke throttling via APIM or app-level rate limiting.

Azure ML (Hub)

WorkspaceFree*

Centralized model registry for versioning and staged deployments. Training pipelines, experiment tracking, and A/B testing across all spoke applications.

AI Search (Spoke)

Standard S1$245.28/mo

Each spoke owns its AI Search instance for RAG. Maintains vector indexes, semantic ranking, and keyword search specific to its workload data.

Azure Firewall

Standard$912/mo

Centralized egress control between hub and spokes. Application rules, network rules, and threat intelligence. DNS proxy for private DNS resolution.

App Service / AKS

P1v3~$140/mo

Spoke compute for the application workload. Uses VNet Integration for outbound to private endpoints. Connects to Hub OpenAI via VNet Peering.

ADLS Gen2 (Hub + Spoke)

Standard LRS~$60/mo

Hub ADLS stores models, datasets, artifacts. Spoke ADLS stores documents, embeddings, and cache for the RAG application. Both HNS-enabled.

Security & Networking

Network

Hub AI services behind Private Endpoints
Spoke connects via bidirectional VNet Peering
Private DNS Zones linked to Hub VNet (shared resolution)
App Service uses VNet Integration for outbound
Azure Firewall for centralized egress

Identity

Key Vault uses RBAC authorization
App identity injected as App Settings
Managed Identities for cross-service auth
AML model registry with staged deployments
Purge protection enabled on Key Vault

Use Cases

Vertical Applications

Custom RAG logic per business unit, shared base models. Each BU owns its search indexes and documents.

RAG Applications

Spoke owns the vector DB + documents, hub owns the LLM. Efficient for knowledge-base-per-department patterns.

Cost-Efficient Multi-App

Single OpenAI deployment, per-spoke search indexes. Save 60%+ on LLM costs vs. full decentralization.

Controlled Model Reuse

AML model registry enables versioning and staged rollout across all spoke applications.

Constraints & Considerations

Constraint	Mitigation
Model versioning governance needed	Use AML model registry with staged deployments
Data movement between hub and spokes	Use managed identities + Private Link for secure access
Shared OpenAI quota across spokes	Implement per-spoke throttling via APIM or app-level rate limiting
Network latency (peering)	Minimal within same region; monitor P95 latency
Spoke depends on Hub availability	Monitor Hub services; consider read replicas for AI Search

Hybrid Central-Spoke AI